A hybrid binocular vision measurement and reconstruction method for 3D shape and temperature field of high-temperature objects

Abstract

The present invention discloses a hybrid binocular vision measurement and reconstruction method for the three-dimensional shape and temperature field of a high-temperature object. The method adopts a monocular structured light system for measurement. The measuring device is simpler than the binocular structured light system. During the measurement, an image of the high-temperature object to be measured modulated by structured light stripes is taken, and the noise in the image is filtered and denoised by a designed algorithm to make the image quality higher, thereby laying a good foundation for subsequent three-dimensional reconstruction. The structured light image is dephased to obtain the absolute phase, and a mapping relationship is established through the same phase. The three-dimensional reconstruction of the high-temperature object is completed by the triangulation principle to obtain point cloud data. At the same time, the emissivity of the high-temperature object is calibrated by a colorimetric thermometer, and a mapping relationship from temperature to color is established by a colorimetric temperature measurement method, thereby completing the synchronous measurement of the three-dimensional shape and surface temperature. The present invention has strong robustness and strong adaptability to the experimental environment. It can complete the measurement work in a narrow observation window. While completing the three-dimensional shape measurement of the high-temperature object, the temperature information of the high-temperature object is synchronously obtained to enrich the measurement data.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional shape and surface temperature measurement, and more specifically, relates to a hybrid binocular vision measurement and reconstruction method for the three-dimensional shape and temperature field of a high-temperature object. Background Art

[0002] In the aerospace field, the turbine blades of aircraft engines located in the combustion chamber often work at a high temperature of about 2000 degrees Celsius. Their surface morphology often undergoes large deformation due to temperature, and their physical dimensions are very different from those at normal temperature. For aircraft engines, a slight deformation under high temperature and high speed conditions may have a significant impact on their working state, and even cause damage, resulting in extremely serious consequences. Therefore, in order to ensure that the deformation of objects at work is controlled, it is necessary to simultaneously measure the three-dimensional morphology and surface temperature of the objects under high temperature conditions, accurately obtain their working temperature, and control them so that they will not produce excessive deformation at the working temperature, so as to ensure the safe and stable operation of equipment and facilities.

[0003] At present, the three-dimensional shape measurement of high-temperature objects is mostly based on binocular structured light systems, which are mainly completed by binocular cameras and projectors. When the projector projects a structured light pattern with a specific coding scheme, the binocular camera captures the deformed fringes formed by these patterns on the object to be measured. The structured light phase information in these deformed fringes contains important clues about the depth of the object. By matching the conjugate point pairs on the imaging planes of the two cameras, combining the matching information and the calibration parameters of the two cameras, and using the principle of trigonometric relationship, a direct connection between phase and depth is established, and the three-dimensional height information of the object is derived based on the captured phase information, and finally the three-dimensional shape characteristics of the object are obtained. However, this method has certain shortcomings:

[0004] 1. The measuring device is complex, has high requirements for the measuring environment, and has poor adaptability to different test scenarios; the experimental process is complex, and it is necessary to ensure that the object to be measured is exactly in the center of the field of view of the binocular camera and the projector. It is difficult to carry out experiments in a small and restricted environment. In addition, the existing high-temperature object heating device has a smaller and smaller observation window to ensure the temperature stability in the cavity, which leads to severe limitations on the measurement viewing angle, making it difficult to ensure that the object to be measured is in the measurement center of the binocular camera and the projector at the same time;

[0005] 2. For high-temperature objects to be tested, their own radiation is strong, which interferes with the collected images, resulting in poor quality of the collected images. They contain more noise and interference, which will cause the loss of key information.

[0006] 3. For high-temperature objects, the existing measurement technology can only obtain the geometric features of the target surface, but ignores the important temperature information of the high-temperature object, resulting in the loss of measurement information and the inability to synchronously obtain the overall temperature information of the high-temperature object; and if it is necessary to synchronously obtain the temperature information of the high-temperature object, it is necessary to add temperature measurement equipment, which will further increase the complexity of the experimental device. In a small and restricted measurement environment, it is even more difficult to ensure that the object to be measured is in the center of the field of view of the measuring device, and the experimental effect is difficult to guarantee. Summary of the invention

[0007] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a hybrid binocular vision measurement and reconstruction method for the three-dimensional shape and temperature field of a high-temperature object, so as to enhance the robustness of the measurement, overcome the interference caused by high temperature, complete the three-dimensional shape measurement of the high-temperature object, and at the same time, enhance the adaptability of the experimental environment so as to complete the measurement work in a narrow observation window, and more easily ensure that the object to be measured is in the center of multiple fields of view. While completing the three-dimensional shape measurement of the high-temperature object, the temperature information of the high-temperature object is synchronously obtained to enrich the measurement data.

[0008] To achieve the above-mentioned purpose, the hybrid binocular vision measurement and reconstruction method of the three-dimensional shape and temperature field of a high-temperature object of the present invention is characterized by comprising:

[0009] (1) Measurement device debugging and data collection

[0010] 1.1) Fix the projection device in the center, install the grayscale camera and the color camera adjacent to each other and fix them on the left side of the projection device, where the resolution of the grayscale camera is M*N and the resolution of the color camera is m*n;

[0011] 1.2) Adjust the positions of the grayscale camera, the color camera and the projection device so that both the grayscale camera and the color camera can clearly capture the high-temperature object to be measured, and the structured light projected by the projection device just completely covers the high-temperature object to be measured, that is, the high-temperature object to be measured is in the common field of view of the grayscale camera, the color camera and the projection device;

[0012] S1.3) Place the calibration plate at the center of the common field of view of the grayscale camera and the projection device, use the grayscale camera to take multiple grayscale images of the calibration plate in different postures, take num calibration plate images in each posture, use Zhang's calibration method to complete the calibration of the grayscale camera and the projection device based on the calibration plate images in different postures, and obtain the grayscale camera internal parameter K c And the rotation and translation matrix [R c ,T c ] and the projection device internal parameter K p And the rotation and translation matrix [R p ,T p ];

[0013] 1.4) The projection device projects Gray code and phase shift code structured light images in sequence, triggering the grayscale camera to synchronously collect the image of the high-temperature object under test. The obtained Gray code structured light image is recorded as There are a total of a images, and the acquired phase-shifted code structured light image is There are b images in total, with a resolution of M*N. The phase-shifted coded structured light image is They are collectively referred to as grayscale images and are denoted as

[0014] Use a color camera to collect a color image of the high-temperature object being measured, denoted as: ColorImg m*n , the resolution is m*n, and the data collection work is completed;

[0015] (2) High-frequency noise removal of grayscale images

[0016] 2.1) Grayscale image acquired by grayscale camera Perform the first level wavelet decomposition Get grayscale image The low frequency component LL1 and the high frequency components HL1, LH1, HH1 in the horizontal, vertical and diagonal directions;

[0017] 2.2) Continue to decompose the first-layer low-frequency component LL1, [LL2, LH2, HL2, HH2] = wavedec (LL1), and obtain the second-layer low-frequency signal LL2 and the high-frequency components HL2, LH2, HH2 in three directions after wavelet decomposition;

[0018] 2.3) Perform threshold determination on the obtained high-frequency signals HL1, LH1, HH1, HL2, LH2, and HH2. If the data in the high-frequency signal is less than the threshold, it is set to 0, otherwise it is retained, where the threshold is the standard deviation of each high-frequency signal:

[0019]

[0020] Among them, HL1(i,j), LH1(i,j), HH1(i,j), HL2(i,j), HH1(i,j), HH2(i,j) are the data of high-frequency signals HL1, LH1, HH1, HL2, LH2, HH2 located at coordinate (i,j), μ_HL1, μ_LH1, μ_HH1μ_HL2μ_LH2, μ_HH2 are the means of high-frequency signals HL1, LH1, HH1, HL2, LH2, HH2, N_HL1, N_LH1, N_HH1, N_HL2, N_LH2 N_HH2 are the numbers of high-frequency signals HL1, LH1, HH1, HL2, LH2, HH2;

[0021] Traverse the data of each high-frequency signal HL1, LH1, HH1, HL2, LH2, HH2, and compare and judge with the corresponding thresholds, i.e., standard deviations stdHL1, stdHL1, stdHL1, stdLH2, stdLH2, stdHH2, to obtain the high-frequency signals HL1′, LH1′, HH1′, HL2′, LH2′, HH2′ after noise removal;

[0022] 2.4) First, perform inverse wavelet transform on each wavelet subband and low-frequency signal of the denoised second layer to obtain the reconstructed low-frequency component of the first layer LL1′=waverec[LL2,LH2′,HL2′,HH2′], and then reconstruct the reconstructed low-frequency component and high-frequency component of the first layer Get the reconstructed image after denoising

[0023] (3) Grayscale image detail texture restoration after denoising

[0024] 3.1) For the reconstructed image after denoising Use Butterworth low-pass filter G(i,j) and Gaussian low-pass filter H(i,j) to process them respectively Get the image ImgG respectively k ,ImgH k ;

[0025] 3.2) Use the low-pass filtered image ImgG k ,ImgH k With the original image Perform differential processing respectively to obtain the high-frequency signal part DG of the image k DH k ;

[0026] 3.3) Use the high-frequency signal part DG of the image k DH k and reconstructed image Perform superposition fusion processing to obtain the enhanced image

[0027]

[0028] Among them, sgn represents the sign function, according to the high-frequency signal part DG of the image k The data in returns 1 or -1 if it is positive or negative, w1 and w2 are weights, and w1+w2=1;

[0029] (4) Absolute phase acquisition based on Gray code combined with phase-shift code structured light

[0030] 4.1) The enhanced image Divided into Gray code structured light image And phase-shifted structured light images

[0031] Phase-shifted structured light image Decode and get the package phase

[0032]

[0033] 4.2) For the auxiliary Gray level G1, a total of a-1 Gray codes are used for phase deconvolution to obtain:

[0034] First, calculate the original level V1:

[0035]

[0036] Then, calculate the auxiliary Gray level G1:

[0037] G1(i,j)=Sort(V1(i,j))

[0038] Among them, Sort represents the sorting operation;

[0039] 4.3) For the auxiliary level G2, use a Gray code to perform phase deconvolution to obtain:

[0040] First, calculate the original level V2:

[0041]

[0042] Then, calculate the auxiliary Gray level G2:

[0043] G2(i,j)=Sort(V2(i,j))

[0044] Among them, Sort represents the sorting operation;

[0045] 4.4) Calculate the absolute phase Φ(i,j):

[0046]

[0047] (5) Point pair matching based on absolute phase

[0048] Using the fact that the absolute phase of the same point is the same in different coordinate systems, the imaging position (u c ,v c ) and its corresponding imaging position on the reverse camera of the projection device (u p ,v p ), and obtain the point pair matching relationship: Φ c (u c,v c )=Φ p (u p ,v p );

[0049] (6) Complete 3D reconstruction of high-temperature objects based on matching point pairs

[0050] The world coordinate system Coordinate world Coordinate with the camera coordinate system camera The mapping relationship between the camera pixel coordinate system and the three-dimensional point in the space world is simplified, called P c , P p is the projection matrix of the grayscale camera and projection device, which is determined by calibration work, then:

[0051]

[0052] Among them, s c 、s p represents the scale factor, X, Y, Z represent the camera pixel (u c ,v c ) and its corresponding projector pixel (u p ,v p ) The three-dimensional coordinates of a spatial point P determined between two corresponding points;

[0053] Solve to get the solution of three-dimensional points in space:

[0054]

[0055] Traverse all pixel points, calculate their corresponding three-dimensional points P, and merge all three-dimensional points to obtain point cloud data Cloud ori (X,Y,Z);

[0056] (7) Use colorimetric temperature measurement through color image ColorImg m*n Get the temperature information of high temperature objects

[0057] 7.1) Use a colorimetric thermometer to measure the emissivity K of an object b Calibration, establishing the mapping relationship between object color information and temperature;

[0058] 7.2) Color Image ColorImg m*n The temperature T(i,j) corresponding to a single pixel (i,j) is:

[0059]

[0060] Where c2 represents the second constant of Planck's law, λ r and λg Represents the wavelengths represented by red and green respectively, R(i,j) and G(i,j) represent the color image ColorImg m*n The red channel and green channel values ​​at (i, j);

[0061] (8) Merge the 3D point cloud and temperature information of high-temperature objects

[0062] 8.1) Image ColorImg m*n Process it and convert its perspective into a grayscale image with a resolution of M*N and unify the resolution of the two.

[0063] 8.2) Use the canny operator to perform color image ColorImg m*n With a grayscale image with a resolution of M*N The edge contour is extracted to obtain the image ColorImage edge , Img edge ;

[0064] 8.3) For the contour image ColorImage edge , Img edge Use dilation and erosion operations to eliminate the two defects of holes and discreteness, and get the processed image ColorImage e ' dge , Img e ' dge ;

[0065] 8.4) Image ColorImage e ' dge With Img e ' dge The SURF algorithm is used to extract feature points and establish matching relationships. Redundant feature matches are removed, and only the registration point pairs with higher confidence are retained, recorded as: MatchPairs[p1.p2,...pn], and it is ensured that there is only a one-to-one correspondence between point pairs;

[0066] 8.5) Based on the obtained high confidence matching point pairs, the RANSAC algorithm is used to calculate the final transformation matrix H between the matching point pairs. trans ;

[0067] 8.6) Color image ColorImg m*n Transform to grayscale image Img enhanced From the perspective, we get ColorImage tran :ColorImage tran =ColorImg m*n*H trans ;

[0068] 8.7) For color image ColorImage tran Perform interpolation and expansion processing to obtain high-resolution color images

[0069] 8.8) According to the color image Combined with the temperature mapping formula obtained based on the colorimetric temperature measurement method in step (7), the temperature value corresponding to each pixel is obtained, and a two-dimensional temperature matrix Temp is constructed. M*N :

[0070]

[0071] in, Represents color images The red channel and green channel values ​​at (i, j);

[0072] 8.9) The obtained high temperature object 3D point cloud data Cloud ori (X,Y,Z) and temperature information Temp M*N To perform the merge process:

[0073] Using the mapping relationship between two-dimensional image coordinates and three-dimensional point cloud coordinates, the two-dimensional temperature matrix Temp M*N Assign values ​​to the three-dimensional point cloud to obtain new point cloud data Cloud (X, Y, Z, T), where T represents the temperature information of the point, and XYZ represents the three-dimensional coordinates of each point in the point cloud. The point cloud after the temperature information is merged is obtained to complete the synchronous measurement of the three-dimensional morphology and surface temperature of the high-temperature object.

[0074] The object of the invention of the present invention is achieved in this way:

[0075] The hybrid binocular vision measurement and reconstruction method of the three-dimensional shape and temperature field of a high-temperature object of the present invention adopts a monocular structured light system for measurement. The measurement device is simpler than the binocular structured light system. It is simpler to place the object to be measured in the center of multiple fields of view during the measurement process. It has higher robustness to different experimental environments and is more suitable for measurement work in a high-temperature restricted environment. The synchronous measurement device adopted includes: a projection device for projecting structured light images, a grayscale camera for collecting grayscale images, a color camera for collecting color images, and a colorimetric thermometer. During the measurement, an image of the high-temperature object to be measured modulated by structured light stripes is taken, and the noise in the image is filtered and denoised by designing an algorithm to make the image quality higher, laying a good foundation for subsequent three-dimensional reconstruction. The structured light image is dephased to obtain the absolute phase, and a mapping relationship is established by the same phase. The three-dimensional reconstruction of the high-temperature object is completed by the triangulation principle to obtain point cloud data. At the same time, the emissivity of the high-temperature object is calibrated by a colorimetric thermometer, and a mapping relationship from temperature to color is established by the colorimetric temperature measurement method, so that the temperature information of the object can be obtained by the color image collected by the color camera, and the synchronous measurement of the three-dimensional shape and surface temperature is completed. The present invention has strong robustness, can overcome the interference caused by high temperature, and complete the three-dimensional morphology measurement of high-temperature objects; it has strong adaptability to the experimental environment, can complete the measurement work in a narrow observation window, and can easily ensure that the object to be measured is in the center of multiple fields of view; it can simultaneously obtain the temperature information of the high-temperature object while completing the three-dimensional morphology measurement of the high-temperature object, thereby enriching the measurement data. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 It is a flow chart of a specific implementation method of the hybrid binocular vision measurement and reconstruction method of the three-dimensional shape and temperature field of a high-temperature object of the present invention;

[0077] Figure 2 It is the principle schematic diagram of the whole measuring device;

[0078] Figure 3 : is a comparison diagram of a noisy grayscale image before and after filtering denoising and detail texture restoration, wherein (a) is the image before filtering denoising and detail texture restoration, and (b) is the image after filtering denoising and detail texture restoration;

[0079] Figure 4 It is a schematic diagram of the absolute phase obtained by de-phase after projecting an image on a high-temperature object using an improved Gray code combined with phase-shift code structured light;

[0080] Figure 5 In order to obtain the three-dimensional point cloud data of high-temperature objects by using the improved Gray code combined with phase-shift code structured light scheme;

[0081] Figure 6The image of a high-temperature object taken by a color camera and the resulting image after perspective conversion and interpolation expansion, where (a) is the image of a high-temperature object, and (b) is the resulting image after perspective conversion and interpolation expansion;

[0082] Figure 7 Through the temperature fusion experiment, we obtain the point cloud with temperature information and the normalized point cloud, where (a) the original point cloud data; (b) the point cloud data after temperature information fusion; (c) the fused point cloud data after normalization. DETAILED DESCRIPTION

[0083] The specific implementation of the present invention is described below in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.

[0084] The present invention is based on a monocular structured light system and proposes a method for synchronously measuring the overall morphology and surface temperature information of a high-temperature object based on a grayscale camera, a color camera, and a projection device. A single grayscale camera and a projection device are used to form a monocular structured light system, and Gray code combined with phase-shift code structured light are used to measure the morphology of a high-temperature object; a colorimetric thermometer is used to calibrate the emissivity of the object to establish a mapping relationship between temperature and color, a color camera is used to obtain a color image, and a colorimetric temperature measurement formula is used to obtain the overall temperature information of the object through color information. The experimental equipment and experimental requirements are simplified, and the morphology measurement of the object can be better completed under complex and restricted conditions. In addition, for the noise in the collected high-temperature object image, the cause and type of the noise are analyzed, and a joint filtering algorithm is used to process it, the noise interference in the image is removed, and the blurred details are restored to obtain clear and effective image data, which is conducive to the subsequent three-dimensional reconstruction work. Finally, the purpose of synchronously obtaining the morphology and temperature information of the high-temperature object is achieved.

[0085] Figure 1 It is a flow chart of a specific implementation method of the hybrid binocular vision measurement and reconstruction method of the three-dimensional shape and temperature field of a high-temperature object of the present invention.

[0086] In this embodiment, if Figure 1 As shown, the hybrid binocular vision measurement and reconstruction method of the three-dimensional shape and temperature field of a high-temperature object of the present invention comprises:

[0087] Step S1: Measurement device debugging and data collection

[0088] Step S1.1: Fix the projector device Projector in the center, install the grayscale camera CameraGray and the color camera CameraColor adjacent to each other and fix them on the left side of the projector device, where the resolution of the grayscale camera is M*N and the resolution of the color camera is m*n. The schematic diagram of the whole measuring device is as follows: Figure 2 As described above, under the control of the measurement control system, the three-dimensional morphology and surface temperature of the high-temperature object are synchronously measured.

[0089] Step S1.2: Adjust the positions of the grayscale camera CameraGray, the color camera CameraColor and the projector Projector so that the grayscale camera CameraGray and the color camera CameraColor can clearly capture the high-temperature object under test, and the structured light projected by the projector Projector just completely covers the high-temperature object under test, that is, the high-temperature object under test is in the common field of view of the grayscale camera CameraGray, the color camera CameraColor and the projector Projector.

[0090] Step S1.3: Place the calibration plate at the center of the common field of view of the grayscale camera CameraGray and the projector Projector, use the grayscale camera CameraGray to take multiple grayscale images of the calibration plate in different postures, take num calibration plate images in each posture, use Zhang's calibration method to complete the calibration of the grayscale camera and the projector based on the calibration plate images in different postures, and obtain the grayscale camera internal parameter K c And the rotation and translation matrix [R c ,T c ] and the projection device internal parameter K p And the rotation and translation matrix [R p ,T p ].

[0091] Step S1.4: The projection device projects Gray code and phase shift code structured light images in sequence, triggering the grayscale camera to synchronously collect the image of the high-temperature object under test. The obtained Gray code structured light image is recorded as There are a total of a images, and the acquired phase-shifted code structured light image is There are b images in total, with a resolution of M*N. The phase-shifted coded structured light image is They are collectively referred to as grayscale images and are denoted as

[0092] Use a color camera to collect a color image of the high-temperature object being measured, denoted as: ColorImg m*n , the resolution is m*n, and the data collection work is completed.

[0093] In this example, the test object is a metal specimen, and the eddy current coil is used to heat the specimen. A grayscale camera is used to capture a grayscale image of the high-temperature object to be tested. Use a color camera to capture color images of high-temperature objects ColorImg m*n .

[0094] Step S2: Grayscale image high frequency noise removal processing

[0095] Step S2.1: Collect the grayscale image obtained by the grayscale camera CameraGray Perform the first level wavelet decomposition Get grayscale image The low frequency component LL1 and the high frequency components HL1, LH1, HH1 in the horizontal, vertical and diagonal directions.

[0096] Step S2.2: Continue to decompose the first-layer low-frequency component LL1, [LL2, LH2, HL2, HH2] = wavedec(LL1), and obtain the second-layer low-frequency signal LL2 after wavelet decomposition and the high-frequency components HL2, LH2, HH2 in three directions.

[0097] Step S2.3: To better preserve details, threshold values ​​are determined for the obtained high-frequency signals HL1, LH1, HH1, HL2, LH2, and HH2. If the data in the high-frequency signal is less than the threshold value, it is set to 0, otherwise it is retained, where the threshold value is the standard deviation of each high-frequency signal:

[0098]

[0099] Among them, HL1(i,j), LH1(i,j), HH1(i,j), HL2(i,j), HH1(i,j), HH2(i,j) are the data of high-frequency signals HL1, LH1, HH1, HL2, LH2, HH2 located at coordinate (i,j), μ_HL1, μ_LH1, μ_HH1μ_HL2μ_LH2, μ_HH2 are the means of high-frequency signals HL1, LH1, HH1, HL2, LH2, HH2, respectively, and N_HL1, N_LH1, N_HH1, N_HL2, N_LH2 N_HH2 are the numbers of high-frequency signals HL1, LH1, HH1, HL2, LH2, HH2 respectively.

[0100] Traverse the data of each high-frequency signal HL1, LH1, HH1, HL2, LH2, HH2, and compare them with the corresponding thresholds, i.e., standard deviations stdHL1, stdHL1, stdHL1, stdLH2 stdLH2, stdHH2, to obtain the high-frequency signals HL1′, LH1′, HH1′, HL2′, LH2′, HH2′ after noise removal.

[0101] Step S2.4: First, perform inverse wavelet transform on each denoised second-layer wavelet subband and low-frequency signal to obtain the reconstructed first-layer low-frequency component LL1′=waverec[LL2,LH2′,HL2′,HH2′], and then reconstruct the reconstructed low-frequency component and high-frequency component of the first layer. Get the reconstructed image after denoising

[0102] Step S3: grayscale image detail texture restoration after denoising

[0103] Step S3.1: Reconstruct the denoised image Use Butterworth low-pass filter G(i,j) and Gaussian low-pass filter H(i,j) to process them respectively Get the image ImgG respectively k ,ImgH k .

[0104] Step S3.2: Use the low-pass filtered image ImgG k ,ImgH k With the original image Perform differential processing respectively to obtain the high-frequency signal part DG of the image k DH k .

[0105] Step S3.3: Use the high frequency signal part DG of the image k DH k and reconstructed image Perform superposition fusion processing to obtain the enhanced image

[0106]

[0107] Among them, sgn represents the sign function, according to the high-frequency signal part DG of the image k The data in returns 1 or -1 if it is positive or negative, w1 and w2 are weights, and w1+w2=1.

[0108] In this embodiment, the noisy grayscale image is subjected to filtering denoising and detail texture restoration before and after. Figure 3 As shown, the wavelet decomposition layer number L = 2, and the high-frequency noise threshold is set The frequency channel cutoff frequency is set to D0=80.

[0109] Step S4: Absolute phase acquisition based on Gray code combined with phase-shift code structured light

[0110] Step S4.1: The enhanced image Divided into Gray code structured light image And phase-shifted structured light images

[0111] Phase-shifted structured light image Decode and get the package phase

[0112] Phase value obtained by decoding the phase shift code It is only distributed between (-π, π), which is ambiguous in the entire space and makes it impossible to obtain the absolute phase Φ(x, y).

[0113] Step S4.2: For the auxiliary Gray level G1, use a-1 Gray codes to perform phase deconvolution to obtain:

[0114] First, calculate the original level V1:

[0115]

[0116] Then, calculate the auxiliary Gray level G1:

[0117] G1(i,j)=Sort(V1(i,j))

[0118] Here, Sort represents a sorting operation.

[0119] Step S4.3: For the auxiliary level G2, use a Gray code to perform phase deconvolution to obtain:

[0120] First, calculate the original level V2:

[0121]

[0122] Then, calculate the auxiliary Gray level G2:

[0123] G2(i,j)=Sort(V2(i,j))

[0124] Here, Sort represents a sorting operation.

[0125] Step S4.4: Calculate the absolute phase Φ(i,j):

[0126]

[0127] In this embodiment, the improved Gray code is combined with the phase shift code structured light to project an image on the high temperature object. The absolute phase diagram obtained by de-phase is as follows: Figure 4 As shown, the projection structured light resolution is set to Height = 1080, Width = 1920, and the level of the projection Gray code is G level =4, the period of the projected phase-shifted code structured light is

[0128] Step S5: Point pair matching based on absolute phase

[0129] After decoding the structured light image and obtaining the absolute phase, the camera coordinate system Camera can be established by using the fact that the phase of the same point is the same in different coordinate systems. XYZ Projector coordinate system XYZ The mapping between .

[0130] Using the fact that the absolute phase of the same point is the same in different coordinate systems, the imaging position (u c ,v c ) and its corresponding imaging position on the reverse camera of the projection device (u p ,v p ), and obtain the point pair matching relationship: Φ c (u c ,v c )=Φ p (u p ,v p ).

[0131] Step S6: Complete the 3D reconstruction of the high temperature object based on the matching point pairs

[0132] The world coordinate system Coordinate world Coordinate with the camera coordinate system camera The mapping relationship between the camera pixel coordinate system and the three-dimensional point in the space world is simplified, called P c , P p is the projection matrix of the grayscale camera and projection device, which is determined by calibration work, then:

[0133]

[0134] Among them, s c 、s p represents the scale factor, X, Y, Z represent the camera pixel (u c ,v c ) and its corresponding projector pixel (u p ,v p)The three-dimensional coordinates of a spatial point P determined between two corresponding points.

[0135] Solve to get the solution of three-dimensional points in space:

[0136]

[0137] Traverse all pixel points, calculate their corresponding three-dimensional points P, and merge all three-dimensional points to obtain point cloud data Cloud ori (X,Y,Z).

[0138] The improved Gray code combined with phase-shift code structured light scheme is used to obtain the three-dimensional point cloud data of high-temperature objects. Figure 5 As shown, the three pictures are from three different perspectives.

[0139] Step S7: Using the colorimetric temperature measurement method to obtain the color image ColorImg m*n Get the temperature information of high temperature objects

[0140] Step S7.1: Use a colorimetric thermometer to measure the object's emissivity K b Calibration,establishes the mapping relationship between object color information and temperature.

[0141] Step S7.2: Color image ColorImg m*n The temperature T(i,j) corresponding to a single pixel (i,j) is:

[0142]

[0143] Where c2 represents the second constant of Planck's law, λ r and λ g Represents the wavelengths represented by red and green respectively, R(i,j) and G(i,j) represent the color image ColorImg m*n The red channel and green channel values ​​at (i, j).

[0144] Step S8: Merge the high-temperature object's three-dimensional point cloud with the temperature information

[0145] Step S8.1: ColorImg the image m*n Process it and convert its perspective into a grayscale image with a resolution of M*N and unify the resolution of the two.

[0146] Step S8.2: Use the canny operator to perform color image ColorImg m*n With a grayscale image with a resolution of M*N The edge contour is extracted to obtain the image ColorImage edge, Img edge ;

[0147] Step S8.3: Contour image ColorImage edge , Img edge Use dilation and erosion operations to eliminate the two defects of holes and discreteness, and get the processed image ColorImage e ' dge , Img e ' dge ;

[0148] Step S8.4: ColorImage e ' dge With Img e ' dge The SURF algorithm is used to extract feature points and establish matching relationships. Redundant feature matches are removed, and only the registration point pairs with higher confidence are retained, recorded as: MatchPairs[p1.p2,...pn], and it is ensured that there is only a one-to-one correspondence between point pairs;

[0149] Step S8.5: Based on the obtained high confidence matching point pairs, use the RANSAC algorithm to calculate the final transformation matrix H between the matching point pairs trans ;

[0150] Step S8.6: Convert the color image ColorImg m*n Transform to grayscale image Img enhanced From the perspective, we get ColorImage tran :ColorImage tran =ColorImg m*n *H trans

[0151] Step S8.7: ColorImage tran Perform interpolation and expansion processing to obtain high-resolution color images

[0152] In this embodiment, the image of the high temperature object captured by the color camera and the resulting image after the perspective conversion and difference expansion are as follows: Figure 6 As shown, (a) is the image of the high temperature object, and (b) is the result image after perspective conversion and difference expansion. The image resolution before conversion is Height = 2048, Width = 2448, and the resolution after conversion is Height = 3000, Width = 4096. When using the expansion and corrosion operation, the window size is given as windowsize dilate =5,windowsize erode =3.

[0153] Step S8.8: Based on the color image Combined with the temperature mapping formula obtained based on the colorimetric temperature measurement method in step (7), the temperature value corresponding to each pixel is obtained, and a two-dimensional temperature matrix Temp is constructed. M*N :

[0154]

[0155] in, Represents color images The number of red and green channels at (i, j)

[0156] Step S8.9: Obtain the high temperature object three-dimensional point cloud data Cloud ori (X,Y,Z) and temperature information Temp M*N To perform the merge process:

[0157] Using the mapping relationship between two-dimensional image coordinates and three-dimensional point cloud coordinates, the two-dimensional temperature matrix Temp M*N Assign values ​​to the three-dimensional point cloud to obtain new point cloud data Cloud (X, Y, Z, T), where T represents the temperature information of the point, and XYZ represents the three-dimensional coordinates of each point in the point cloud. The point cloud after the temperature information is merged is obtained to complete the synchronous measurement of the three-dimensional morphology and surface temperature of the high-temperature object.

[0158] In this embodiment, a point cloud with temperature information and a normalized point cloud are obtained through a temperature fusion experiment. Figure 7 , where (a) original point cloud data; (b) point cloud data after temperature information fusion; (c) fused point cloud data after normalization processing.

[0159] Although the above describes an illustrative specific embodiment of the present invention to facilitate the understanding of the present invention by those skilled in the art, it should be clear that the present invention is not limited to the scope of the specific embodiment. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

Claims

1. A hybrid binocular vision measurement and reconstruction method for the three-dimensional shape and temperature field of a high-temperature object, characterized by comprising: (1) Measurement device debugging and data collection 1.1) Fix the projection device in the center, install the grayscale camera and the color camera adjacent to each other and fix them on the left side of the projection device, where the resolution of the grayscale camera is M*N and the resolution of the color camera is m*n; 1.2) Adjust the positions of the grayscale camera, the color camera and the projection device so that both the grayscale camera and the color camera can clearly capture the high-temperature object to be measured, and the structured light projected by the projection device just completely covers the high-temperature object to be measured, that is, the high-temperature object to be measured is in the common field of view of the grayscale camera, the color camera and the projection device; S1.3) Place the calibration plate at the center of the common field of view of the grayscale camera and the projection device, use the grayscale camera to take multiple grayscale images of the calibration plate in different postures, take num calibration plate images in each posture, use Zhang's calibration method to complete the calibration of the grayscale camera and the projection device based on the calibration plate images in different postures, and obtain the grayscale camera internal parameter K c And the rotation and translation matrix [R c ,T c ] and the projection device internal parameter K p And the rotation and translation matrix [R p ,T p ]; 1.4) The projection device projects Gray code and phase shift code structured light images in sequence, triggering the grayscale camera to synchronously collect the image of the high-temperature object under test. The obtained Gray code structured light image is recorded as There are a total of a images, and the acquired phase-shifted code structured light image is There are b images in total, with a resolution of M*N. The phase-shifted coded structured light image is They are collectively referred to as grayscale images and are denoted as Use a color camera to collect a color image of the high-temperature object being measured, denoted as: ColorImg m*n , the resolution is m*n, and the data collection work is completed; (2) High-frequency noise removal of grayscale images 2.1) Grayscale image acquired by grayscale camera Perform the first level wavelet decomposition Get grayscale image The low frequency component LL1 and the high frequency components HL1, LH1, HH1 in the horizontal, vertical and diagonal directions; 2.2) Continue to decompose the first-layer low-frequency component LL1, [LL2, LH2, HL2, HH2] = wavedec (LL1), and obtain the second-layer low-frequency signal LL2 and the high-frequency components HL2, LH2, HH2 in three directions after wavelet decomposition; 2.3) Perform threshold determination on the obtained high-frequency signals HL1, LH1, HH1, HL2, LH2, and HH2. If the data in the high-frequency signal is less than the threshold, it is set to 0, otherwise it is retained. The threshold is the standard deviation of each high frequency signal: Among them, HL1(i,j), LH1(i,j), HH1(i,j), HL2(i,j), HH1(i,j), HH2(i,j) are the data of high-frequency signals HL1, LH1, HH1, HL2, LH2, HH2 located at coordinate (i,j), μ_HL1, μ_LH1, μ_HH1μ_HL2μ_LH2, μ_HH2 are the means of high-frequency signals HL1, LH1, HH1, HL2, LH2, HH2, N_HL1, N_LH1, N_HH1, N_HL2, N_LH2 N_HH2 are the numbers of high-frequency signals HL1, LH1, HH1, HL2, LH2, HH2; Traverse the data of each high-frequency signal HL1, LH1, HH1, HL2, LH2, HH2, and compare and judge with the corresponding thresholds, i.e., standard deviations stdHL1, stdHL1, stdHL1, stdLH2, stdLH2, stdHH2, to obtain the high-frequency signals HL1′, LH1′, HH1′, HL2′, LH2′, HH2′ after noise removal; 2.4) First, perform inverse wavelet transform on each wavelet subband and low-frequency signal of the denoised second layer to obtain the reconstructed low-frequency component of the first layer LL1′=waverec[LL2,LH2′,HL2′,HH2′], and then reconstruct the reconstructed low-frequency component and high-frequency component of the first layer Get the reconstructed image after denoising (3) Grayscale image detail texture restoration after denoising 3.1) For the reconstructed image after denoising Use Butterworth low-pass filter G(i,j) and Gaussian low-pass filter H(i,j) to process them respectively Get the image ImgG respectively k ,ImgH k ; 3.2) Use the low-pass filtered image ImgG k ,ImgH k With the original image Perform differential processing respectively to obtain the high-frequency signal part DG of the image k DH k ; 3.3) Use the high-frequency signal part DG of the image k DH k and reconstructed image Perform superposition fusion processing to obtain the enhanced image Among them, sgn represents the sign function, according to the high-frequency signal part DG of the image k The data in returns 1 or -1 if it is positive or negative, w1 and w2 are weights, and w1+w2=1; (4) Absolute phase acquisition based on Gray code combined with phase-shift code structured light 4.1) The enhanced image Divided into Gray code structured light image And phase-shifted structured light images Phase-shifted structured light image Decode and get the package phase 4.2) For the auxiliary Gray level G1, a total of a-1 Gray codes are used for phase deconvolution to obtain: First, calculate the original level V1: Then, calculate the auxiliary Gray level G1: G1(i,j)=Sort(V1(i,j)) Among them, Sort represents a sorting operation; 4.3) For the auxiliary level G2, use a Gray code to perform phase deconvolution to obtain: First, calculate the original level V2: Then, calculate the auxiliary Gray level G2: G2(i,j)=Sort(V2(i,j)) Among them, Sort represents a sorting operation; 4.4) Calculate the absolute phase Φ(i,j): (5) Point pair matching based on absolute phase Using the fact that the absolute phase of the same point is the same in different coordinate systems, the imaging position (u c ,v c ) and its corresponding imaging position on the reverse camera of the projection device (u p ,v p ), and obtain the point pair matching relationship: Φ c (u c ,v c )=Φ p (u p ,v p ); (6) Complete 3D reconstruction of high-temperature objects based on matching point pairs The world coordinate system Coordinate world Coordinate with the camera coordinate system camera The mapping relationship between the camera pixel coordinate system and the three-dimensional point in the space world is simplified, called P c , P p is the projection matrix of the grayscale camera and projection device, which is determined by calibration work, then: Among them, s c 、s p represents the scale factor, X, Y, Z represent the camera pixel (u c ,v c ) and its corresponding projector pixel (u p ,v p ) The three-dimensional coordinates of a spatial point P determined between two corresponding points; Solve to get the solution of three-dimensional points in space: Traverse all pixel points, calculate their corresponding three-dimensional points P, and merge all three-dimensional points to obtain point cloud data Cloud ori (X,Y,Z); (7) Use colorimetric temperature measurement through color image ColorImg m*n Get the temperature information of high temperature objects 7.1) Use a colorimetric thermometer to measure the emissivity K of an object b Calibration, establishing the mapping relationship between object color information and temperature; 7.2) Color Image ColorImg m*n The temperature T(i,j) corresponding to a single pixel (i,j) is: in, c2 represents the second constant of Planck's law, λ r and λ g Represents the wavelengths represented by red and green respectively, R(i,j) and G(i,j) represent the color image ColorImg m*n The red channel and green channel values ​​at (i, j); (8) Merge the 3D point cloud and temperature information of high-temperature objects 8.1) Image ColorImg m*n Process it and convert its perspective into a grayscale image with a resolution of M*N and unify the resolution of the two. 8.2) Use the canny operator to perform color image ColorImg m*n With a grayscale image with a resolution of M*N The edge contour is extracted to obtain the image ColorImage edge , Img edge ; 8.3) For the contour image ColorImage edge , Img edge Use dilation and erosion operations to eliminate the two defects of holes and discreteness, and get the processed image ColorImage e ' dge , Img e ' dge ; 8.4) Image ColorImage e ' dge With Img e ' dge The SURF algorithm is used to extract feature points and establish matching relationships. Redundant feature matches are removed, and only the registration point pairs with higher confidence are retained, recorded as: MatchPairs[p1.p2,...pn], and it is ensured that there is only a one-to-one correspondence between point pairs; 8.5) Based on the obtained high confidence matching point pairs, the RANSAC algorithm is used to calculate the final transformation matrix H between the matching point pairs. trans ; 8.6) Color image ColorImg m*n Transform to grayscale image Img enhanced From the perspective, we get ColorImage tran :ColorImage tran =ColorImg m*n *H trans ; 8.7) For color image ColorImage tran Perform interpolation and expansion processing to obtain high-resolution color images 8.8) According to the color image Combined with the temperature mapping formula obtained based on the colorimetric temperature measurement method in step (7), the temperature value corresponding to each pixel is obtained, and a two-dimensional temperature matrix Temp is constructed. M*N : in, Represents color images The red channel and green channel values ​​at (i, j); 8.9) The obtained high temperature object 3D point cloud data Cloud ori (X,Y,Z) and temperature information Temp M*N To perform the merge process: Using the mapping relationship between two-dimensional image coordinates and three-dimensional point cloud coordinates, the two-dimensional temperature matrix Temp M*N Assign values ​​to the three-dimensional point cloud to obtain new point cloud data Cloud (X, Y, Z, T), where T represents the temperature information of the point, and XYZ represents the three-dimensional coordinates of each point in the point cloud. The point cloud after the temperature information is merged is obtained to complete the synchronous measurement of the three-dimensional morphology and surface temperature of the high-temperature object.

Images