Light source calibration method, surface reconstruction method and system based on photometric stereoscopic vision

Abstract

The invention provides a light source calibration method, a surface reconstruction method and a system based on photometric stereoscopic vision. The light source calibration method comprises the following steps: based on a small ball calibration algorithm of a light source in any direction, obtaining a near-end point distance and a far-end point distance of a recommended clear working distance range of the light source, and a corresponding near-end inclination angle and a corresponding far-end inclination angle; constructing a mathematical model of the inclination angle of the light source and the working distance by adopting an interpolation algorithm to obtain the inclination angle of the light source in any direction; acquiring a deflection angle of the light source in any direction; calculating an illumination direction vector of the light source in the direction based on the deflection angle and the inclination angle of the light source in the direction; and forming a light source matrix by the illumination direction vectors of the light sources in all directions. According to the light source calibration method, the parameter calibration process is simplified, and the method is suitable for various application scenes and has good universality. The invention further provides a surface reconstruction method based on photometric stereoscopic vision, and surface reconstruction of the to-be-measured object can be simply, conveniently and rapidly achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of photometric stereo vision, and in particular to a light source calibration method, a surface reconstruction method and a system based on photometric stereo vision. Background Art

[0002] Photometric stereo is a non-contact detection method that reconstructs 3D or 2.5D information of an object based on a two-dimensional image. It has important applications in fields where it is necessary to detect scratches, pits, and other objects with slight height differences on the surface.

[0003] The photometric stereo method uses a fixed-position camera to shoot an object at a fixed position under illumination from multiple light sources in different directions. Before performing the photometric stereo algorithm calculation, it is usually necessary to calibrate the angle information of the light sources in different directions.

[0004] However, when photometric stereo is used in practice, ideal conditions for using parallel light sources are often not available. In order to solve the technical problem of light source calibration in photometric stereo, the existing technology often simulates the photometric stereo light source as a point light source and performs relevant theoretical derivations to calibrate the position information of the light source.

[0005] In the prior art, a light source calibration algorithm based on a metal ball is usually used to determine the position information of the light source in the photometric stereo. However, the light source calibration algorithm based on the metal ball has the following problems in practical applications: (1) The light source calibration algorithm based on metal balls requires manual work to find the position mask of the metal balls in advance, so as to eliminate background interference during calibration calculation. When the light source conditions change, recalibration is required, and finding the ball mask will consume a lot of time during the calibration process; (2) The ball may have defects or be affected by stray light spots, which may affect the calibration results; (3) During calibration, multiple calibration calculation parameters need to be input, such as the coordinates of the center of the sphere, the radius of the sphere, and the coordinates of the brightest point. The calculation of the relevant parameters is time-consuming and these parameters have no direct relationship with the actual application. (4) It is inconvenient in actual application of factory assembly lines. It is troublesome to use metal balls for calibration on site when the equipment is deployed. If there are many devices, they need to be calibrated one by one, which is time-consuming and laborious. (5) The calibration algorithm based on metal balls is limited in its application scenarios, such as the lack of calibration conditions for vertical surface detection.

[0006] Based on the possible problems of the traditional metal ball calibration algorithm, Nie Ying et al. proposed a photometric stereo vision system calibration method based on an approximate point light source model (DOI: 10.12146 / j.issn.2095-3135.201605004). The radiance model of the light source was established according to its own luminous characteristics. The camera was used as a reference system, and a multi-sphere calibration method was proposed to calculate the position parameters of each light source. The position of the light source was accurately calculated by combining the sphere contour line and the sphere radius. A calibration method based on a reference plane was also proposed. By analyzing the plane brightness distribution under the illumination of the approximate point light source and combining the equal brightness line fitting strategy, the direction parameters of the main optical axis of the light source were calculated. The calibration of the light source position parameters and the main optical axis parameters was completed. The calibration process recorded in this paper still needs to calculate the position parameters related to the ball, which is too cumbersome and prone to calibration errors.

[0007] To avoid calibration deviations caused by small ball defects and other reasons. Chinese patent CN115631243A proposes a calibration method, related equipment and storage medium for light sources in a light field acquisition system. The method includes: obtaining multiple first images of a reflective object synchronously captured based on different viewing angles under a first ambient brightness; the first ambient brightness is obtained based on the incident light of a target light source in the light field acquisition system, and the target light source is any light source in the light field acquisition system; based on the position of the reflected light spot in each first image, the image position information of the target light source in each first image is determined; based on the spatial position information of the reflective object and the above-mentioned image position information of the target light source, the first reflection position information and the first incident light direction information corresponding to each first image are determined; based on the first reflection position information and the first incident light direction information corresponding to each first image in the multiple first images, the spatial position information of the target light source is determined. This patent uses image position information to implement the calibration process of the light source, which is different from the technical means of this solution.

[0008] China CN 117011559 A proposes a container key identification and pairing system and method, which includes an identification device fixing system, a light source correction system, a key image pre-processing system, a key image processing system and a key identification and pairing system. Among them, the light source correction system is used to collect reference images of different lighting directions, correct the light source, calibrate the position of the light source, and set it to the standard position. In this patent, the positional relationship between the light source and the camera is assumed, and the deflection angle is calculated using the light source height and the horizontal projection distance of the light source from the key to be tested; and the collected image is used as a reference, the center of the image horizontally to the right is called the starting point, and the inclination angle is the angle between the projection light beam and the starting point, which is usually evenly distributed around the object. This patent uses the special positional relationship between the light source and the camera to describe the lighting direction with an angle to achieve light source calibration; the scope of application of this method is limited. Summary of the invention

[0009] The present invention proposes a light source calibration method, a surface reconstruction method and a system based on photometric stereo vision, which can solve at least one of the above technical problems.

[0010] To achieve the above object, the present invention proposes the following technical solutions: A light source calibration method based on photometric stereo vision, wherein the light sources in several directions illuminate the surface of the object to be measured at the working distance to produce brightness changes, and a camera obtains the surface geometric information of the object to be measured, the method comprising: Based on a small ball calibration algorithm of a light source in any direction, obtain the near end point distance and the far end point distance of the recommended clear working distance range of the light source, and the corresponding near end inclination angle and far end inclination angle; An interpolation algorithm is used to construct a mathematical model of the inclination angle of the light source and the working distance to obtain the inclination angle of the light source in any direction; wherein the applicable range of the working distance in the model is the recommended clear working distance range; Get the deflection angle of the light source in any direction; Based on the deflection angle and inclination angle of the light source in that direction, the lighting direction vector of the light source in that direction is calculated; the lighting direction vectors of the light sources in all directions are combined into a light source matrix; The inclination angle is the angle between the light from the light source in any direction and the positive direction of the z-axis of the camera coordinate system; The deflection angle is the angle between the projection of the light from a light source in any direction on the xy plane of the camera coordinate system and the positive direction of the x-axis of the camera coordinate system.

[0011] On the other hand, the present invention also proposes a surface reconstruction method based on photometric stereo vision, comprising: Using the above-mentioned light source calibration method based on photometric stereo vision, obtain the light source matrix L; Collect grayscale images of the object to be tested in different light source directions; Vectorize the grayscale images of different light source directions to obtain the grayscale value vector matrix M; Based on the lighting direction matrix L and the gray value vector matrix M, calculate the surface normal vector N=(L T L) -1 L T M; and reconstruct the surface of the object to be measured.

[0012] Based on the same inventive concept, the present invention also proposes a surface reconstruction system, comprising: The calibration module is used to obtain the deflection angle and inclination angle of the light source in any direction; based on the deflection angle and inclination angle, calculate the lighting direction vector of the light source in that direction; and form the lighting direction vectors of the light sources in all directions into a light source matrix as the calibration result; An acquisition module is used to acquire grayscale images of the object to be tested under light sources in different directions; The reconstruction module is used to vectorize the grayscale images under different directions of light sources and obtain the grayscale value vector matrix; based on the lighting direction matrix and the grayscale value vector matrix, it is used to calculate the surface normal vector of the object to be measured; and reconstruct the surface of the object to be measured.

[0013] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the above-mentioned light source calibration method based on photometric stereo vision or surface reconstruction method based on photometric stereo vision.

[0014] The present invention further proposes a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the light source calibration method based on photometric stereo vision or the surface reconstruction method based on photometric stereo vision is implemented.

[0015] The beneficial effects of the present invention are as follows: The light source calibration method based on photometric stereo vision disclosed in the present invention adopts the angle between the light of the light source in any direction and the direction of the camera optical axis as the inclination angle, and the angle between the projection of the light of the light source in any direction in the camera coordinate system on the xy plane and the positive direction of the x-axis as the deflection angle to describe the angular information of the light source in this direction.

[0016] The light source calibration method based on photometric stereo vision proposed in the present invention can use the installation method of the light source and the camera to determine the value of the deflection angle in the scenario where the light source and the camera have a specific installation method, thereby simplifying the calibration parameter process. In addition, for actual scenarios where the deflection angle cannot be obtained through the installation method, a mathematical model of working distance and deflection angle is provided. Under the premise that the positions of the camera and the light source remain unchanged, the deflection angle corresponding to any working distance within the current working distance range can be obtained. In addition, the mathematical model for calculating the deflection angle proposed in the present invention is also applicable to the above-mentioned specific installation method scenario.

[0017] Based on the same calculation principle, the present invention also proposes a mathematical model of working distance and inclination angle to simplify the calculation process of the inclination angle.

[0018] The light source calibration method based on photometric stereo vision proposed in the present invention can be applied to various application scenarios such as line light sources, ring light sources, dome light sources, etc., and has good environmental adaptability.

[0019] Based on the surface reconstruction method based on photometric stereo vision proposed in the present invention, in actual application scenarios, the angle information of the light source in any direction is obtained in a simpler and faster way, so as to calculate the light source matrix L. The light source matrix L is used as the basic parameter of the surface reconstruction method, and then the photometric stereo model is used to obtain the surface normal vector of the object to be measured, so as to realize the surface reconstruction of the object to be measured. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of a light source calibration method based on photometric stereo vision in the present invention; Figure 2 is a schematic diagram of a ring light source in Embodiment 1 of the present invention; Figure 3 yes Figure 2 A top view of the middle ring light source; Figure 4 yes Figure 2 Equivalent schematic diagram of the ring light source in the figure; Figure 5 yes Figure 3 Equivalent schematic diagram of the ring light source in the figure; Figure 6 is a schematic diagram of inner and outer ring light sources in Example 1 of the present invention; Figure 7 is a schematic diagram of installation method 1 in embodiment 1 of the present invention; Figure 8 is a schematic diagram of installation method 2 in embodiment 1 of the present invention; Fig. 9 is a schematic diagram of installation method 3 in embodiment 1 of the present invention; Fig.10 is a schematic diagram of installation method 4 in embodiment 1 of the present invention; Fig.11 is a schematic diagram of upper and lower ring light sources in Example 3 of the present invention; Fig.12 is a schematic diagram of a dome light source device in Embodiment 4 of the present invention; Fig.13 is a schematic diagram of a dome light source device in Embodiment 4 of the present invention; Fig.14 is a schematic diagram of a dome light source device in Embodiment 4 of the present invention; Fig.15 It is a flowchart of the surface reconstruction method based on photometric stereo vision in Example 5 of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0022] like Figure 1 As shown, the present invention proposes a light source calibration method based on photometric stereo vision, comprising: Based on the camera's location, the camera coordinate system and image coordinate system are constructed.

[0023] Among them, the camera coordinate system is a three-dimensional rectangular coordinate system, and the origin of the camera coordinate system is located at the optical center of the camera, which is usually defined as the intersection point of the lens.

[0024] Typically, in the camera coordinate system, the x-axis and y-axis are parallel to the two sides of the camera imaging plane (working plane); the z-axis is along the optical axis of the camera lens, that is, the observation direction from the inside of the camera to the outside.

[0025] The image coordinate system is a two-dimensional coordinate system. The origin of the image coordinate system is the intersection of the z-axis of the camera coordinate system and the imaging plane (working plane). The x-axis of the image coordinate system extends in the horizontal direction of the imaging plane, which is consistent with the extension direction of the x-axis on the xy plane of the camera coordinate system; the y-axis of the image coordinate system extends in the vertical direction of the imaging plane.

[0026] Based on a small ball calibration algorithm for a light source in any direction, the near end point distance and the far end point distance of the recommended clear working distance range of the light source, and their corresponding near end inclination angle and far end inclination angle are obtained; an interpolation algorithm is used to construct a mathematical model of the inclination angle of the light source and the working distance to obtain the inclination angle of the light source in any direction; wherein the applicable range of the working distance in the model is the recommended clear working distance range.

[0027] The inclination angle is the angle between the light from the light source in any direction and the positive direction of the z-axis of the camera coordinate system.

[0028] Photometric stereo scenes are generally composed of multiple approximate point light sources in different directions and a camera, where the relative position of the approximate point light source and the camera is fixed. In photometric stereo scenes, the spatial position of the light source is determined by the relative position relationship between the calibrated approximate point light source and the camera. The most common light source calibration method is the small ball calibration algorithm.

[0029] The ball calibration algorithm mainly uses a highly reflective ball (calibration ball) with a known radius for reflection in a photometric stereo scene, uses a camera to capture images of the calibration ball under approximate point light sources in different directions, extracts the contour lines of the highly reflective ball and the coordinates of the highlight points on the surface of the highly reflective ball (the position of the highlight points on the spherical surface) in multiple calibration ball images; based on the mirror reflection properties of the highlight point positions on the spherical surface, the mirror reflection model is used to calculate the position of the center of the ball, thereby calibrating the position of the light source.

[0030] The recommended clear working distance is when the camera can capture enough details so that the highlights and outlines of the calibration sphere in the calibration sphere image are still clearly visible.

[0031] Get the deflection angle of the light source in any direction; where the deflection angle is the angle between the projection of the light from the light source in any direction and the positive direction of the x-axis of the camera coordinate system in the xy plane of the camera coordinate system.

[0032] There are two ways to obtain the deflection angle, including: Acquisition method 1: In the results of the ball calibration algorithm of the camera at the same position and the light source in any direction, at least two working distances of the light source in any direction and their corresponding two deflection angles are obtained; an interpolation algorithm is used to construct a mathematical model of the light source deflection angle and the light source working distance, and the deflection angle of the light source in any direction is obtained; wherein the applicable range of the working distance in the model is the range between the two working distances. The working distance is the vertical distance between the light source in any direction and the working plane.

[0033] Acquisition method 2: If it is an area array camera, the angular relationship between the area array camera and the light source in any direction is obtained according to the installation method of the illumination direction vector of the light source in any direction being rotationally symmetrically distributed with the camera optical axis as the rotation axis, and the camera image acquisition direction, so as to obtain the deflection angle of the light source in any direction.

[0034] If it is a line array camera, the angular relationship between the line array camera and the light source in any direction is obtained according to the installation method in which the illumination direction vector of the light source in any direction is rotationally symmetrically distributed with the camera optical axis as the rotation axis, as well as the camera image acquisition direction and the object movement direction, and the deflection angle of the light source in any direction is obtained.

[0035] Based on the deflection angle and inclination angle of the light source in this direction, the illumination direction vector of the light source in this direction is calculated.

[0036] l1=[sin(slant1)cos(tilt1) sin(slant1)sin(tilt1) cos(slant1)]; where slant1 is the inclination angle of the light source in that direction, and tilt1 is the deflection angle of the light source in that direction.

[0037] The illumination direction vectors of light sources in all directions are combined into the light source matrix L. Example

[0038] like Figure 2-5 As shown, this embodiment takes a ring light source containing light sources in four directions as an example, assuming that the ring light source is a photometric stereoscopic lighting source uniformly distributed in a circle, and the light emitted by the light source in any direction is uniform, and the light source in any direction can be approximated as a point light source.

[0039] The light sources of the ring light source in four directions are arc light source ①, arc light source ②, arc light source ③, and arc light source ④. For the above ring light source, there are two parameters required for calibration: ① the working distance D of the ring light source; ② the deflection angle of the light source in any direction of the ring light source.

[0040] The calibration method of the above-mentioned ring light source includes: Get the deflection angle of the light source in any direction; where the deflection angle is the angle between the projection of the light from the light source in any direction and the positive direction of the x-axis of the camera coordinate system in the xy plane of the camera coordinate system.

[0041] Preferably, in this embodiment, the deflection angle is defined with the counterclockwise direction being positive.

[0042] In this embodiment, a light source in any direction is equivalent to a point light source.

[0043] like Figure 2-5 As shown, the four arc-shaped light sources of the annular light source in this embodiment are located in the same plane, and the annular optical axis coincides with the optical axis of the camera. The camera and the annular light source are installed in a manner that: the illumination direction vector of the light source in any direction is rotationally symmetrically distributed with the camera optical axis as the rotation axis; the angle relationship between the camera and the light source in any direction can be used to obtain the deflection angle of the light source in any direction.

[0044] In this embodiment, a ring light source composed of four evenly distributed arc light sources is provided, and the optical axis of the ring light coincides with the optical axis of the camera. In particular, on the xy plane, the center of the ring light source is the projection of the optical center of the camera.

[0045] Preferably, if Figure 6 As shown, the light sources of the ring light source in each direction of this embodiment are located in the same plane, and the ring optical axis coincides with the camera optical axis. The ring light source can be divided into an inner ring light source and an outer ring light source, and the illumination direction vectors of the inner ring light source and the outer ring light source are both rotationally symmetrically distributed with the camera optical axis as the rotation axis.

[0046] In this embodiment, the inner ring light source is provided with two arc-shaped light sources, which are evenly distributed around the circumference; the outer ring light source is provided with two arc-shaped light sources, which are evenly distributed around the circumference.

[0047] Preferably, the number of arc-shaped light sources of the inner ring light source and the outer ring light source can be adjusted according to actual needs, and the number recorded in this embodiment is only for reference. The number of layers of the inner ring and the outer ring can also be determined according to actual needs.

[0048] like Figure 7-10 As shown, this embodiment proposes four camera installation methods to further explain the angular relationship between the camera and the light source in any direction.

[0049] In the schematic diagram of the camera installation method ( Figure 7-10 ), in the image coordinate system, four arc-shaped light sources are evenly distributed around the center of the circle with the projection of the camera's optical center as the center to form a ring-shaped light source.

[0050] The light sources in the four directions are arc light source ①, arc light source ②, arc light source ③, and arc light source ④. The arc light source is equivalent to a point light source. Arc light source ① and arc light source ② are located in the schematic diagram ( Figure 7-10), arc light source ③ and arc light source ④ are located in the schematic diagram ( Figure 7-10 ) below.

[0051] The camera is located at the center of the ring light source, and the camera is marked with a camera plug port to explain the camera's image acquisition direction and the direction of the image coordinate system. The object's motion direction is also marked in the schematic diagram.

[0052] The origin of the image coordinate system is the intersection of the camera optical axis and the imaging plane (working plane). The x-axis of the camera coordinate system is parallel to the x-axis of the image coordinate system, and the y-axis of the camera coordinate system is parallel to the y-axis of the image coordinate system. The origin of the camera coordinate system is the location of the camera.

[0053] Based on the relationship between the camera coordinate system and the image coordinate system, the xy plane of the camera coordinate system can be equivalent to the image coordinate system. And because the deflection angle is the angle between the projection of the light of the light source in any direction in the camera coordinate system on the xy plane and the positive direction of the x-axis, it can also be equivalent to the angle between the light of the light source in any direction in the image coordinate system and the positive direction of the x-axis.

[0054] The positive direction of the x-axis of the image coordinate system is consistent with the positive direction of the x-axis of the xy plane of the camera coordinate system. The y-axis of the image coordinate system extends perpendicular to the imaging plane, and the positive direction of the y-axis is vertically upward or vertically downward.

[0055] like Figure 7 As shown, in installation mode 1, in the image coordinate system, the horizontal left direction is the positive direction of the x-axis, and the vertical downward direction is the positive direction of the y-axis. The object moves vertically downward.

[0056] Assume that arc light sources ①-④ are lit in sequence, and the camera is installed in such a way that the camera plug port is close to arc light source ① and arc light source ②.

[0057] like Figure 8 As shown, in installation mode 2, in the image coordinate system, the horizontal rightward direction is the positive direction of the x-axis, and the vertical downward direction is the positive direction of the y-axis. The object moves vertically downward.

[0058] Assume that arc light sources ①-④ are lit in sequence, and the camera is installed in such a way that the camera plug port is close to arc light source ③ and arc light source ④.

[0059] like Fig. 9 As shown, in installation mode 3, in the image coordinate system, the horizontal rightward direction is the positive direction of the x-axis, and the vertical upward direction is the positive direction of the y-axis. The object moves vertically upward.

[0060] Assume that arc light sources ①-④ are lit in sequence, and the camera is installed in such a way that the camera plug port is close to arc light source ③ and arc light source ④.

[0061] like Fig.10As shown, in installation mode 4, in the image coordinate system, the horizontal left direction is the positive direction of the x-axis, and the vertical upward direction is the positive direction of the y-axis. The object moves vertically upward.

[0062] Assume that arc light sources ①-④ are lit in sequence, and the camera is installed in such a way that the camera plug port is close to arc light source ① and arc light source ②.

[0063] In summary, the camera installation method is that the camera cable port is close to the top of the diagram ( Figure 7 , 10) or below ( Figure 8 , 9), causing the position of the image sensor to deflect, causing the positive direction of the x-axis of the image coordinate system to change, that is, Figure 7 and Fig.10 The positive x-axis of the image coordinate system is horizontally to the left. Figure 8 and Fig. 9 The positive y-axis of the image coordinate system is horizontal and points to the right.

[0064] According to the installation method in which the illumination direction vector of the light source in any direction is rotationally symmetrically distributed with the camera optical axis as the rotation axis, and the camera image acquisition direction, the angular relationship between the area array camera and the light source in any direction is obtained, and the deflection angle of the light source in any direction is obtained.

[0065] If it is an area array camera, the factors affecting the light source deflection angle in any direction include the installation position of the ring light source and the installation method of the area array camera. Among them, the installation method of the area array camera can be Figure 8 (Installation method 2) or Fig.10 Any of the installation methods shown in (Installation method 4).

[0066] According to the installation method in which the illumination direction vector of the light source in any direction is rotationally symmetrically distributed with the camera optical axis as the rotation axis, as well as the camera image acquisition direction and the object movement direction, the angle relationship between the camera and the light source in any direction is obtained, and the deflection angle of the light source in any direction is obtained. Among them, the camera is a linear array camera.

[0067] If it is a line array camera, the factors affecting the light source deflection angle in any direction include the installation position of the ring light source, the installation method of the line array camera, and the object movement direction (the object movement direction affects the positive direction of the image coordinate y axis). Among them, the installation method of the line array camera is Figure 7-10 Any of the installation methods shown, namely, installation methods 1-4, are applicable to the application scenarios of the line array camera.

[0068] Assume that the deflection angle of the arc light source ③ in installation method 1 is , the deflection angle distribution of other installation methods is shown in the following table.

[0069] Table 1

[0070] The camera and the ring light source are installed in a manner that: the illumination direction vector of the light source in any direction is rotationally symmetrically distributed with the camera optical axis as the rotation axis; the deflection angle of the light source in any direction can be obtained by using the angular relationship between the camera and the light source in any direction.

[0071] A mathematical model of the light source inclination angle and the light source working distance is constructed to obtain the inclination angle of the light source in any direction.

[0072] Specifically, it includes: based on a small ball calibration algorithm of a light source in any direction, obtaining a near end point distance D1 and a far end point distance D2 of a recommended clear working distance range of the light source, and their corresponding near end tilt angle slantD1 and far end tilt angle slantD2; An interpolation algorithm is used to construct a mathematical model of the inclination angle of the light source and the working distance to obtain the inclination angle of the light source in any direction; wherein the applicable range of the working distance in the model is the recommended clear working distance range.

[0073] The inclination angle is the angle between the light from the light source in any direction and the positive direction of the z-axis of the camera coordinate system.

[0074] In such Figure 2 In the installation method of the ring light source and the camera shown, at least two sets of data of the working distance of the ring light source from D1 to D2 and the corresponding light source inclination angle are obtained from the ball calibration algorithm results of the installation method.

[0075] Taking arc light source ① as an example, the working distance D1 and its corresponding light source inclination angle slantD1 are obtained, and the working distance D2 and its corresponding light source inclination angle slantD2 are obtained.

[0076] When the arc light source ① is at the working distance D (D1 <= D <= D2), the corresponding light source inclination angle slant1 is:

[0077] The working distance D is the vertical distance between the light source and the working plane in any direction.

[0078] In this embodiment, the working distance may be the vertical distance from the light source surface to the working plane in any direction. In this embodiment, the working plane is the plane where the object to be measured is located, and is also the focus plane of the camera.

[0079] Preferably, the surface of the light source in any direction can also be corrected to the surface of the light bead. In this case, the working distance is the vertical distance from the surface of the light bead of the light source in any direction to the working plane.

[0080] Based on the deflection angle and inclination angle of the light source in this direction, the illumination direction vector of the light source in this direction is calculated.

[0081] According to the above calculation process, the deflection angle tilt1 and the inclination angle slant1 of the arc light source ① can be obtained; then the unit direction vector of the arc light source ① is calculated, and the unit direction vector is used as the illumination direction vector l1 of the arc light source ①.

[0082] l1=[sin(slant1)cos(tilt1) sin(slant1)sin(tilt1) cos(slant1)] According to the acquisition process of the lighting direction vector of the arc light source ①, the deflection angle tilt2 and the inclination angle slant2 corresponding to the arc light source ② are obtained; the deflection angle tilt3 and the inclination angle slant3 corresponding to the arc light source ③; and the deflection angle tilt4 and the inclination angle slant4 corresponding to the arc light source ④ are obtained.

[0083] Calculate the lighting direction vector l2 of arc light source ②, the lighting direction vector l3 of arc light source ③, and the lighting direction vector l4 of arc light source ④ respectively.

[0084] The illumination direction vectors of light sources in all directions are combined into the light source matrix L, specifically:

[0085] Example 2 On the basis of Example 1, this embodiment proposes a new calculation method for calculating the deflection angle of a light source in any direction, so that the deflection angle of a light source in any direction can still be obtained under the condition that the installation method of the camera and the light source recorded in Example 1 is not satisfied. At the same time, the deflection angle calculation formula proposed in this embodiment is also applicable to the application scenario of Example 1. The construction process of the mathematical model of the deflection angle and the working distance can refer to the construction process of the inclination angle and the working distance in Example 1.

[0086] Get the deflection angle of the light source in any direction, including: In the ball calibration algorithm results of the camera and light source at the same position, at least two working distances of the light source in any direction and their corresponding two deflection angles are obtained; an interpolation algorithm is used to construct a mathematical model of the light source deflection angle and the light source working distance to obtain the deflection angle of the light source in any direction; wherein the applicable range of the working distance in the model is the range between the two working distances.

[0087] The calculation process of the deflection angle is described by taking the annular light source including four arc-shaped light sources in the same plane in Example 1 as an example.

[0088] In such Figure 2 In the installation method of the ring light source and the camera shown, at least two sets of data of the working distance of the ring light source from D1 to D2 and the corresponding light source deflection angle are obtained from the ball calibration algorithm results of the installation method.

[0089] Taking arc light source ① as an example, the working distance D1 and its corresponding light source deflection angle tiltD1 are obtained, and the working distance D2 and its corresponding light source deflection angle tiltD2 are obtained.

[0090] When the arc light source ① is at the working distance D (D1 <= D <= D2), the corresponding light source deflection angle tilt1 is:

[0091] The working distance D is the vertical distance between the light source and the working plane in any direction.

[0092] In this embodiment, the working distance may be the vertical distance from the light source surface to the working plane in any direction. In this embodiment, the working plane is the plane where the object to be measured is located, and is also the focus plane of the camera.

[0093] Preferably, the surface of the light source in any direction can also be corrected to the surface of the light bead. In this case, the working distance is the vertical distance from the surface of the light bead of the light source in any direction to the working plane.

[0094] According to the acquisition process of the light source deflection angle tilt1 corresponding to the above arc light source ①, a deflection angle calculation formula corresponding to arc light sources ②-4 is constructed, and the deflection angle tilt2 corresponding to arc light source ②, the deflection angle tilt3 corresponding to arc light source ③, and the deflection angle tilt4 corresponding to arc light source ④ are obtained.

[0095] Example 3 Based on the embodiment 1, this embodiment provides the following explanation on the installation method of the camera and the light source proposed in the embodiment 1.

[0096] The camera and the light source are installed in a manner that: the illumination direction vector of the light source in any direction is rotationally symmetrically distributed with the camera optical axis as the rotation axis; the deflection angle of the light source in any direction can be obtained by using the angular relationship between the camera and the light source in any direction.

[0097] like Fig.11 As shown, this embodiment proposes a new light source to meet the above installation method.

[0098] In this embodiment, the annular light source is divided into two layers, the upper and lower layers, and the illumination direction vectors of the upper and lower light sources are both rotationally symmetrically distributed with the camera optical axis as the rotation axis.

[0099] In this embodiment, the upper light source is provided with three arc-shaped light sources, which are evenly distributed around the circumference; the lower light source is provided with three arc-shaped light sources, which are evenly distributed around the circumference.

[0100] Preferably, the number of arc-shaped light sources of the upper light source and the lower light source can be adjusted according to actual needs, and the number recorded in this embodiment is only for reference. Similarly, the number of layers can also be confirmed according to actual needs.

[0101] In order to satisfy the light source calibration method based on photometric stereo vision described in this application, the arc light source in this embodiment is equivalent to a point light source.

[0102] Example 4 Based on the embodiment 1, this embodiment provides the following explanation on the installation method of the camera and the light source proposed in the embodiment 1.

[0103] The camera and the light source are installed in a manner that: the illumination direction vector of the light source in any direction is rotationally symmetrically distributed with the camera optical axis as the rotation axis; the deflection angle of the light source in any direction can be obtained by using the angular relationship between the camera and the light source in any direction.

[0104] like Fig.12 As shown, this embodiment proposes a new light source to meet the above installation method.

[0105] In this embodiment, the camera is arranged at the top of the dome light source device, and a line light source is arranged below the dome light source device. The line light source is symmetrically distributed along the optical axis of the camera.

[0106] In order to satisfy the light source calibration method based on photometric stereo vision described in this application, the line light source in this embodiment is equivalent to a point light source.

[0107] like Fig.13 As shown, preferably, the camera is arranged at the vertex of the dome light source device, and a plurality of point light sources are arranged below the dome light source device, and the point light sources are symmetrically distributed along the optical axis of the camera.

[0108] like Fig.14 As shown, preferably, the camera is arranged at the vertex of the dome light source device, and a plurality of point light sources are arranged in the dome, and the point light sources are symmetrically distributed along the optical axis of the camera.

[0109] Example 5 like Fig.15 As shown, the present invention also proposes a surface reconstruction method based on photometric stereo vision, comprising: The light source calibration method based on photometric stereo vision described in the above embodiment is used to calibrate the deflection angle, inclination angle and working distance of the light source in any direction to obtain the light source matrix L; in,

[0110] Where f is the number of light sources.

[0111] Collect grayscale images of the object to be tested in different light source directions, vectorize the grayscale images in different light source directions, and obtain the grayscale value vector matrix M; in, ; Where m is the vector of the grayscale image collected under the light source in any direction, f is the number of light sources, and p is the total number of image pixels.

[0112] Based on the lighting direction matrix L and the grayscale value vector matrix M, according to the photometric stereo normal vector model: M=LN, the surface normal vector N=(L T L) -1 L T M; and reconstruct the surface of the object to be measured.

[0113] Based on the same inventive concept, the present application also proposes a surface reconstruction system to execute the method described in the above embodiment, the system comprising: The calibration module is used to obtain the deflection angle and inclination angle of the light source in any direction; based on the deflection angle and inclination angle, calculate the lighting direction vector of the light source in that direction; and form the lighting direction vectors of the light sources in all directions into a light source matrix as the calibration result; An acquisition module is used to acquire grayscale images of the object to be tested in different light source directions; The reconstruction module is used to vectorize the grayscale images of different light source directions to obtain the grayscale value vector matrix; based on the lighting direction matrix and the grayscale value vector matrix, it is used to calculate the surface normal vector of the object to be measured; and reconstruct the surface of the object to be measured.

[0114] The present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the above method.

[0115] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method described above is implemented.

[0116] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0117] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A light source calibration method based on photometric stereo vision, wherein the light sources in several directions illuminate the surface of the object to be measured at the working distance to produce brightness changes, and a camera obtains the surface geometric information of the object to be measured, characterized in that: The method includes: Based on a small ball calibration algorithm of a light source in any direction, obtain the near end point distance and the far end point distance of the recommended clear working distance range of the light source, and the corresponding near end inclination angle and far end inclination angle; An interpolation algorithm is used to construct a mathematical model of the inclination angle of the light source and the working distance to obtain the inclination angle of the light source in any direction; wherein the applicable range of the working distance in the mathematical model is the recommended clear working distance range; Get the deflection angle of the light source in any direction; Based on the deflection angle and inclination angle of the light source in the direction, the lighting direction vector of the light source in the direction is calculated; the lighting direction vectors of the light sources in all directions are combined into a light source matrix; The inclination angle is the angle between the light from the light source in any direction and the positive direction of the z-axis of the camera coordinate system; The deflection angle is the angle between the projection of the light from a light source in any direction on the xy plane of the camera coordinate system and the positive direction of the x-axis of the camera coordinate system.

2. The light source calibration method based on photometric stereo vision according to claim 1, characterized in that: The method of obtaining the deflection angle of the light source in any direction includes: obtaining the angular relationship between the area array camera and the light source in any direction according to the installation method in which the illumination direction vector of the light source in any direction is rotationally symmetrically distributed with the camera optical axis as the rotation axis, and the camera image acquisition direction, thereby obtaining the deflection angle of the light source in any direction.

3. The light source calibration method based on photometric stereo vision according to claim 1, characterized in that: The method of obtaining the deflection angle of the light source in any direction includes: obtaining the angular relationship between the line array camera and the light source in any direction according to the installation method in which the illumination direction vector of the light source in any direction is rotationally symmetrically distributed with the camera optical axis as the rotation axis, as well as the camera image acquisition direction and the object movement direction, thereby obtaining the deflection angle of the light source in any direction.

4. The light source calibration method based on photometric stereo vision according to claim 1, characterized in that: The obtaining of the deflection angle of a light source in any direction comprises: Based on the result of the small ball calibration algorithm, at least two working distances of the light source in any direction and their corresponding two deflection angles are obtained; an interpolation algorithm is used to construct a mathematical model of the light source deflection angle and the light source working distance to obtain the deflection angle of the light source in any direction; wherein, in the mathematical model of the light source deflection angle and the light source working distance, the applicable range of the working distance is the range between the two working distances.

5. The light source calibration method based on photometric stereo vision according to claim 1, characterized in that: The working distance is the vertical distance between the light source and the working plane in any direction.

6. The light source calibration method based on photometric stereo vision according to claim 1, characterized in that: The illumination direction vector l1 of the light source in this direction is l1=[sin(slant1)cos(tilt1) sin(slant1)sin(tilt1) cos(slant1)]; wherein slant1 is the inclination angle of the light source in this direction, and tilt1 is the deflection angle of the light source in this direction.

7. A surface reconstruction method based on photometric stereo vision, characterized in that: include: Using the light source calibration method based on photometric stereo vision as described in any one of claims 1 to 6 to obtain the light source matrix L; Collect grayscale images of the object to be tested in different light source directions; Vectorize the grayscale images of different light source directions to obtain the grayscale value vector matrix M; Based on the lighting direction matrix L and the gray value vector matrix M, calculate the surface normal vector N=(L T L) -1 L T M; and reconstruct the surface of the object to be measured.

8. A surface reconstruction system, characterized in that: include: The calibration module is used to obtain the deflection and inclination of the light source in any direction; Based on the deflection angle and the inclination angle, the illumination direction vector of the light source in this direction is calculated; The illumination direction vectors of light sources in all directions are combined into a light source matrix as the calibration result; An acquisition module is used to acquire grayscale images of the object to be tested under light sources in different directions; The reconstruction module is used to vectorize the grayscale images under different directions of light sources and obtain the grayscale value vector matrix; based on the lighting direction matrix and the grayscale value vector matrix, it is used to calculate the surface normal vector of the object to be measured; and reconstruct the surface of the object to be measured.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the light source calibration method based on photometric stereo vision as described in any one of claims 1 to 6 or the surface reconstruction method based on photometric stereo vision as described in claim 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for calibrating a light source based on photometric stereo vision as described in any one of claims 1 to 6 or the method for reconstructing a surface based on photometric stereo vision as described in claim 7 is implemented.

Images