Photometric three-dimensional deformation monitoring method oriented to natural light source condition

Abstract

The invention belongs to the technical field of optical measurement and computer vision, and particularly relates to a photometric three-dimensional deformation monitoring method oriented to a natural light source condition. The method comprises the steps that targets are designed, and a target array is arranged according to the imaging range of a camera and the morphology characteristics of an observation object; measuring three-dimensional coordinates of feature points of a target pattern, and calculating a normal vector of a target surface and internal and external orientation elements of an image; calculating multi-period light source directions and light source intensities based on the target three-dimensional coordinates, the normal and the internal and external orientation elements of the image; calculating a surface normal vector by using a luminosity stereoscopic model based on the multi-period light source direction and intensity; and realizing object surface continuous reconstruction based on the surface normal vector and the imaging geometric constraint. Compared with the traditional method, the method can break through the problem that the traditional photometric stereo depends on an artificial light source and is difficult to adapt to the dynamic change of a natural light source so that the method is not applicable, can realize deformation monitoring in an outdoor environment without the artificial light source, and has the advantages of low cost and wide coverage range.

Description

Technical Field

[0001] The present invention belongs to the technical field of optical measurement and computer vision, and in particular relates to a photometric stereo deformation monitoring method for natural light source conditions. Background Art

[0002] Photometric stereo technology, based on the variations in the reflective properties of an object's surface under a combination of multi-angle light sources, enables non-contact, high-precision monitoring of submillimeter deformations through three-dimensional reconstruction and normal distribution analysis. Compared to traditional methods (such as laser scanning or localized discrete measurements using point sensors), photometric stereo enables continuous monitoring over a wide area. Through multi-light source image analysis, photometric stereo technology enables high-resolution, non-contact surface deformation monitoring, accurately capturing submillimeter deformations such as crack propagation and surface spalling. Its continuous surface deformation field reconstruction capability helps comprehensively reflect the overall deformation trend and local damage distribution of a structure, making it particularly suitable for health monitoring of large-scale infrastructure such as bridges and dams. Furthermore, traditional measurement methods require high equipment costs and complex deployment, while photometric stereo requires only a camera for data acquisition, offering a low-cost advantage.

[0003] However, photometric stereo methods rely on analyzing changes in surface brightness to infer surface morphology. In practical applications, traditional photometric stereo relies on fixed light sources, which limits its application under natural light conditions due to uncontrollable or uneven illumination. Especially in outdoor environments, the intensity, angle, and direction of sunlight change with time and weather, resulting in unstable brightness information in the image. Natural light is affected by factors such as weather, season, and time, resulting in unstable light source direction and intensity during photometric stereo analysis, which in turn affects the accuracy of the calculation results. Compared to artificial light sources, whose direction and intensity are easy to control, it is impossible to directly control the angle and distribution of the light source when using natural light for photometric stereo monitoring, which makes the calculation results far less stable than under artificial light conditions. Summary of the Invention

[0004] In order to solve the problems existing in the prior art, the purpose of the present invention is to provide a method for reconstructing the surface shape of an object and then realizing deformation monitoring by using the photometric stereo method under natural light conditions, by pre-deploying customized targets that can accurately measure the coordinates and normal directions to obtain the accurate direction and light intensity of the light source, and then realizing the surface shape reconstruction of the object based on multiple natural light sources combined with photometric stereo and imaging geometry models. The method provided by the present invention can solve the problem of difficult calculation of the direction and intensity of natural light sources, and can effectively expand the application scenarios of photometric stereo.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions: The present invention provides a photometric stereoscopic deformation monitoring method for natural light source conditions, comprising the following steps: Step 1: Design the target and arrange the target array according to the camera imaging range and the morphological characteristics of the observed object; Step 2: Measure the three-dimensional coordinates of the target pattern feature points, calculate the target surface normal vector and the internal and external orientation elements of the image; Step 3: Calculate the direction and intensity of the light source in multiple time periods based on the target's three-dimensional coordinates, normal direction, and image internal and external orientation elements; Step 4: Calculate the surface normal vector point by point using the photometric stereo model based on the direction and intensity of the light source in multiple time periods; Step 5: Based on the surface normal vector and imaging geometric constraints, continuous reconstruction of the object surface is achieved.

[0006] Furthermore, the target design and target array layout in step 1 include the following steps: Based on pinhole imaging and photometric stereo principles, the target shape, material, and surface pattern are determined. The target shape is a square plane; the material meets the characteristics of Lambertian reflectance, weather resistance, and anti-fouling; the target surface pattern is conducive to target calibration and feature pixel positioning; Determine the target position and number based on the camera's imaging field of view. Each target surface should be easy to image in the camera. In addition, the plane normal direction of the target should be different to avoid the target planes being coplanar. After the target is installed, the position and angle remain unchanged, that is, the plane normal direction of the target remains unchanged.

[0007] Furthermore, the calculation of the target surface normal direction and the internal and external orientation elements of the image in step 2 includes the following steps: Measure the spatial position of the target and the normal direction of the target plane. First, measure the 3D coordinates of the four corner markers in the plane pattern. Based on the 3D coordinates of the four points, calculate the 3D coordinates of the plane center point, and then calculate the normal direction of the plane center point. Measure the three-dimensional coordinates of the camera, and on this basis calculate the internal and external orientation elements of the image based on the target measurement data. After successful installation and debugging, keep the camera position and posture stable, and the internal and external orientation elements of the sequence images obtained by the corresponding camera remain unchanged.

[0008] Furthermore, the calculation of the light source direction and light source intensity in multiple time periods in step 3 includes the following steps: The image plane coordinates of the center of each target plane in the image are calculated using the image acquired under a certain light source, and then the image brightness values ​​(i.e. pixel values) of all targets under the light source are obtained; Assuming the target plane is a Lambertian surface, the brightness value of the image is determined by the reflectivity of the target plane, the intensity of the light source, and the direction of the light source; For a certain light source, the brightness values ​​of all target center points are known. A set of equations is constructed by combining the brightness values ​​of all target center points with the light source direction, light source intensity, target plane reflectivity, and target plane normal vector. The direction and intensity of the light source are solved using the least squares method. Based on the images obtained from other light sources, the directions and intensities of other light sources are solved in turn using the same method.

[0009] Furthermore, the calculation of the surface normal vector of the object in step 4 includes the following steps: Based on the images of all time periods and the direction and intensity values ​​of the light source calculated in step 3, for a certain pixel, the pixel brightness values ​​of the images of all time periods and the direction vectors and light intensity of the light sources of all time periods are constructed into an equation system, and the surface normal vector corresponding to the pixel point is solved using the least squares method; Based on the images of all time periods and the direction and intensity of the light source calculated in step 4, the surface normal vectors corresponding to other pixel points are solved point by point using the same method.

[0010] Furthermore, the surface shape reconstruction in step 5 includes the following steps: Calculate the depth gradient based on the normal vector solved in step 4, and convert the normal vector into a depth gradient; Reconstruct the depth map from the depth gradient by solving the Poisson equation; According to the plane coordinates and depth value of each pixel, the three-dimensional coordinates of the object point corresponding to the pixel in the camera coordinate system are calculated point by point; Based on the internal and external orientation elements of the image, the three-dimensional coordinates of the object point in the camera coordinate system are converted into the world coordinate system point by point.

[0011] Compared with the prior art, the present invention has the following beneficial effects: This invention breaks through the technical bottleneck of photometric stereo using natural light sources, and can achieve continuous reconstruction of object surfaces in outdoor environments without artificial light sources. It has the advantages of low cost and wide coverage, and can provide reliable technical support for infrastructure health diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 and Figure 2 A flow chart of the method provided in Example 1 of the present invention; Figure 3 A schematic diagram of a target pattern provided in Example 1 of the present invention; Figure 4 This is a schematic diagram of target layout provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0013] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0014] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0015] It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof. Those skilled in the art can make various modifications or improvements based on the basic idea of ​​the present invention, but as long as they do not depart from the basic idea of ​​the present invention, they are all within the scope of protection of the present invention.

[0016] Example 1 To solve this technical problem, the present invention proposes a photometric stereoscopic deformation monitoring method for natural light source conditions. The method flow is as follows: Figure 1-2 shown.

[0017] The specific steps of the photometric stereo deformation monitoring method for natural light source conditions are as follows: Step 1: Design the target and arrange the target array according to the camera imaging range and the morphological characteristics of the observed object; Specifically, based on the pinhole imaging and photometric stereo principle, the target shape, material and surface pattern are determined. The target shape is a square plane, and the material meets the characteristics of Lambertian reflection, weather resistance and anti-fouling. Alumina ceramic or matte aluminum alloy is selected. The target surface pattern is conducive to target calibration and feature pixel positioning. The target pattern is as follows: Figure 3 As shown; Determine the target size based on camera imaging parameters such as camera focal length, pixel size, and photography parameters such as camera station position and object distance; The target positions and number are determined according to the camera imaging field of view. The target positions are evenly distributed within the imaging field of view, with a layout of 6 to 8 targets. The angle between the normal of each target plane center point and the line connecting the photographic center and the target plane center point is less than 60°. The absolute value of the difference in the three coordinate axis angles of the plane normal between each target in the target array is greater than 30°. The target layout diagram is shown in the figure below. Figure 4 shown.

[0018] Step 2: Measure the three-dimensional coordinates of the target pattern feature points, calculate the target surface normal vector and the internal and external orientation elements of the image; Specifically, the three-dimensional coordinates of the four corner markers in the plane pattern are first measured, and the three-dimensional coordinates of the plane center point are calculated based on the three-dimensional coordinates of the four points. Then, the three-dimensional coordinates of the four corner markers and the three-dimensional coordinates of the plane center point are used to calculate the normal direction of the plane center point; The three-dimensional coordinates of the center points of all target planes and their corresponding image plane coordinates are used to calculate the internal and external orientation elements of the image based on the collinearity equation and the least squares method. After successful installation and debugging, the camera position and posture are kept stable, and the internal and external orientation elements of the sequence images obtained by the corresponding camera remain unchanged.

[0019] Step 3: Calculate the direction and intensity of the light source in multiple time periods based on the target's three-dimensional coordinates, normal direction, and image internal and external orientation elements; Specifically, the target center image plane coordinates are calculated based on the image of target i, and the image brightness value (pixel value) of target i under light source k is obtained: i,k。 Assuming the target is a Lambertian surface, the brightness I i,k由式(1)决定： (1) Where: i is the reflectivity of target i (the target material is uniform, let ρ i =1, i=1,2,…,m); is the intensity of light source k; is the unit direction vector of light source k.

[0020] For each light source k, the brightness data of m targets are used to establish an equation, such as formula (2): ,i=1, 2, …, m (2) in: is the plane normal vector of target i.

[0021] Formula (2) is expanded into: (3) set up , formula (3) is simplified to (4) All targets are: (5) Right now: (6).

[0022] Solve l by least squares method k : (7) The decomposition of light source direction and light source intensity is shown in formula (6) and formula (7): (6) (7) Follow the above steps to calculate the direction and intensity of each light source.

[0023] Step 4: Calculate the surface normal vector point by point using the photometric stereo model based on the direction and intensity of the light source in multiple time periods; Specifically, based on the directions and intensity values ​​of multiple light sources, a surface normal vector of each pixel is calculated.

[0024] (8) Assuming that the reflectivity ρ is constant, the surface normal vector is obtained by the least squares method : (9) Among them, the matrix A is composed of k light source directions, and the elements of the vector b are normalized brightness values.

[0025] Step 5: Based on the surface normal vector and imaging geometric constraints, sub-millimeter surface continuous reconstruction is achieved.

[0026] Specifically, the depth gradient is calculated based on the normal direction value, and the normal vector is converted into a depth gradient. (10) (11) Where X and Y represent the local two-dimensional plane coordinates of the object surface, and Z is the depth direction; Reconstruct the depth map from the depth gradient by solving the Poisson equation, (12) After discretization, a system of linear equations is constructed and solved using the conjugate gradient method; For each pixel (u, v) and its depth Z, calculate the three-dimensional coordinates in the camera coordinate system, (13) Convert to world coordinate system, (14) Where u and v are pixel coordinates on the image plane, f x 、f y is the focal length of the camera, c x 、c y is the coordinate of the principal point, R is the camera rotation matrix, T is the camera translation vector, P cam The three-dimensional coordinates in the camera coordinate system, P world The three-dimensional coordinates in the world coordinate system.

[0027] The advantage of the above scheme is that it proposes a method for solving light source parameters and reconstructing deformation fields based on customized target constraints, which solves the problem of poor applicability of traditional photometric stereo technology, which relies on fixed artificial light sources and is difficult to adapt to dynamic changes in natural lighting. Compared with traditional methods, it has the advantages of low cost and wide coverage in aspects such as the health diagnosis of important infrastructure.

[0028] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical principles disclosed herein are intended to be covered by the claims of the present invention.

Claims

1. A photometric stereo deformation monitoring method for natural light source conditions, characterized in that: The steps include: Step 1: Design the target and arrange the target array according to the camera imaging range and the morphological characteristics of the observed object; Step 2: Measure the three-dimensional coordinates of the target pattern feature points, calculate the target surface normal vector and the internal and external orientation elements of the image; Step 3: Calculate the direction and intensity of the light source in multiple time periods based on the target's three-dimensional coordinates, normal direction, and image internal and external orientation elements; Step 4: Calculate the surface normal vector point by point using the photometric stereo model based on the direction and intensity of the light source in multiple time periods; Step 5: Based on the surface normal vector and imaging geometric constraints, continuous reconstruction of the object surface is achieved.

2. The photometric stereo deformation monitoring method for natural light source conditions according to claim 1, characterized in that: In step 1, the target shape is a square plane, the target surface material has Lambertian reflectance, weather resistance, and anti-fouling properties, and the target surface pattern is a three-color, nine-rectangular grid.

3. The photometric stereo deformation monitoring method for natural light source conditions according to claim 1, characterized in that: In step 1, when the targets are arranged, the angle between the normal line of the target plane center point and the line connecting the photographic center and the target plane center point is less than 60°, and the sum of the absolute values ​​of the differences in the three coordinate axis angles of the plane normal lines of each pair of targets in the target array is greater than 30°.

4. The photometric stereo deformation monitoring method for natural light source conditions according to claim 1, characterized in that: In the step 2: the three-dimensional coordinates of the four corner marker points are measured according to the target plane pattern, and the three-dimensional coordinates of the target plane center point and the plane normal direction are calculated based on the three-dimensional coordinates of the four points; the target and the camera are fixed, and the target center coordinates, the plane normal direction and the internal and external orientation elements of the image are fixed.

5. The photometric stereo deformation monitoring method for natural light source conditions according to claim 1, characterized in that: In step 3: the image plane coordinates of the center point of the target plane in the image are calculated according to the pattern features of each target, and then the image brightness values ​​of all targets under all light sources are obtained respectively; based on the photometric stereo model, equations are constructed for each light source in turn using the image brightness values ​​of all targets and the corresponding target plane normal direction values, and the direction values ​​and intensity values ​​of all light sources are solved using the least squares method.

6. The method for photometric stereo deformation monitoring under natural light source conditions according to claim 1, characterized in that: In step 4: the direction values ​​and intensity values ​​of the multiple light sources are taken as known values, an equation is constructed pixel by pixel based on the photometric stereo model, and the surface normal vector of each pixel is solved by the least squares method.

7. The photometric stereo deformation monitoring method for natural light source conditions according to claim 1, characterized in that: In step 5: the depth gradient is calculated and the normal vector is converted into a depth gradient; the depth map is then integrated and the depth map is reconstructed from the depth gradient by solving the Poisson equation; the three-dimensional coordinates are then back-projected and the three-dimensional coordinates in the camera coordinate system are calculated based on the plane coordinates of each pixel and its depth value, and then converted into world coordinates.

Images