A 3D profile measurement method and system for sheet metal holes based on fringe projection

Abstract

This invention discloses a fringe projection-based method and system for measuring the three-dimensional contours of sheet metal holes. The method uses a uniform illumination pattern from a left camera to capture an initial hole area image to extract the hole contour curve. The extracted contour curve is then subjected to two constant motions toward the surrounding area. Phase matching is used to reconstruct the amplified three-dimensional contour points. The true three-dimensional contour is then reversely reconstructed based on the corresponding points of the two previously reconstructed amplified contours. This method only requires contour extraction on a single image, thus minimizing the impact of inconsistent contour extraction. Furthermore, by performing high-quality phase matching in non-edge areas of the sheet metal, the accuracy and integrity of the three-dimensional contours of the sheet metal holes can be guaranteed.

Description

Technical Field

[0001] The present invention belongs to the field of three-dimensional measurement, and more specifically, relates to a method and system for measuring the three-dimensional contour of a sheet metal hole based on fringe projection. Background Art

[0002] Sheet metal parts are widely used in fields such as automotive transportation and aerospace due to their excellent properties, such as low density, high strength, and good forming ability. Holes are the most common and important features in sheet metal parts, and they require three-dimensional measurement to ensure forming quality and product safety. Existing mainstream methods for three-dimensional hole contour measurement are mainly divided into the following two categories: 1) Three-dimensional contour reconstruction based on measurement points. This method first uses structured light technology to reconstruct the complete three-dimensional shape of the sheet metal part and then extracts the contour of the hole from the point cloud. 2) Contour reconstruction based on the principle of passive binocular vision. This method directly extracts contours in both directions and on the image, and uses the contours to match and reconstruct the point cloud. Compared with the first method, the method based on passive binocular vision has higher accuracy and stability, making it more suitable for three-dimensional hole measurement in sheet metal parts. However, due to factors such as hole thickness, the accuracy and stability of corresponding point matching, etc., the reconstruction accuracy and completeness of this method are still lacking.

[0003] To address these issues, existing methods primarily focus on improving contour extraction accuracy and stereo matching accuracy, thereby improving 3D contour reconstruction accuracy. To improve contour extraction accuracy, researchers primarily employ methods such as shape fitting (e.g., ellipse) based on prior information and grayscale clustering to mitigate the impact of factors like hole thickness and local reflections on contour extraction accuracy. For stereo matching, existing researchers employ methods such as multi-view contour matching energy functions to address the inconsistencies in contour extraction from multiple views and the large number of matching points between the upper and lower contours. Overall, these methods can improve the 3D reconstruction accuracy of hole contours to a certain extent, but the following issues remain: 1) Due to the thickness of the sheet metal part, the hole contour extraction in the left and right camera images can be inconsistent, leading to errors in 3D reconstruction; 2) The edges of the sheet metal hole contour are prone to local reflections and strong reflections, making it difficult to accurately measure the consistency of corresponding pixels between the left and right cameras, resulting in low contour stereo matching accuracy; 3) Multiple stereo candidate points exist at the upper and lower edges of the hole contour, making accurate matching difficult; and 4) Existing research primarily focuses on 3D measurement of circular hole contours, generally requiring ellipse fitting to improve accuracy, making it difficult to apply to irregular-shaped holes such as waist-shaped holes.

[0004] Therefore, a high-precision 3D reconstruction method for the 3D contours of all types of hole positions on sheet metal parts is urgently needed to reduce the impact of inconsistent contour extraction between the left and right cameras, ensure accurate matching of corresponding contour points, and improve the accuracy of 3D reconstruction of hole contours. Summary of the Invention

[0005] In response to the above-mentioned defects or improvement needs of the prior art, the present invention provides a three-dimensional contour measurement method and system for sheet metal holes based on fringe projection, thereby solving the problem of low accuracy of three-dimensional reconstruction of hole contours in existing three-dimensional contour measurement methods due to factors such as inconsistent image extraction from left and right cameras and multiple candidate points at the upper and lower edges.

[0006] To achieve the above objectives, according to a first aspect of the present invention, a method for measuring the three-dimensional profile of a sheet metal hole based on fringe projection is provided, comprising:

[0007] S1, the left and right cameras synchronously obtain a uniform illumination image of the sheet metal to be tested; project a set of multi-frequency grating fringe images onto the sheet metal to be tested, and the left and right cameras synchronously obtain the fringe projection image formed on the surface of the sheet metal to be tested, and calculate the absolute phase image based on the fringe projection image;

[0008] S2, obtaining a binary hole image based on the uniform illumination map of the left camera, using it as the initial hole area image, and determining each hole area and its minimum circumscribed rectangle;

[0009] S3, sequentially expanding the minimum circumscribed rectangle of each hole region by a preset range to use as a mask to initialize the level set energy function, and iterating the function according to a preset number of iterations to obtain the contour curve of each hole region;

[0010] Wherein, the level set energy function is: E LG =E G +ω L E L , E L is the local grayscale energy, E G is the global grayscale energy, E L =∫ Ω H ε (Φ)(I(x,y)-u(x,y)) 2 dxdy+∫ Ω (1-H ε (Φ))(I(x,y)-v(x,y)) 2 dxdy,E G =∫ Ω H ε (Φ)(I(x,y)-u m ) 2 dxdy+∫ Ω (1-H ε (Φ))(I(x,y)-v m ) 2 dxdy,ω L For E L The weighted index, I(x,y) is the gray value of the pixel with coordinates (x,y), H ε(Φ) is the smoothed Heaviside function of the level set function Φ, u m and v m is the global average intensity of the inner and outer regions, Ω is the entire image area, u(x,y) is the intensity of the local internal area, and v(x,y) is the intensity of the local external area;

[0011] S4, performing two constant motions on the contour curve to obtain two amplified contour curves, searching for matching points of each pixel point on the two amplified contour curves in the absolute phase image of the right camera to form matching point pairs; wherein the matching point pairs have the same absolute phase value, and the row number of the pixel point in the absolute phase image of the left camera is the same as the row number of the matching point in the absolute phase image of the right camera;

[0012] S5, performing three-dimensional reconstruction on the matching point pairs to obtain two reconstructed contours; performing reverse reconstruction based on the three-dimensional contour point clouds corresponding to each other on the two reconstructed contours to obtain the true contour point clouds of each hole area.

[0013] According to a second aspect of the present invention, a method for measuring the three-dimensional profile of a sheet metal hole based on fringe projection is provided, comprising:

[0014] S1, a monocular camera obtains a uniform illumination image of the sheet metal to be tested; a set of multi-frequency grating fringe images are projected onto the sheet metal to be tested, the monocular camera obtains a fringe projection image formed on the surface of the sheet metal to be tested, and an absolute phase image is calculated based on the fringe projection image;

[0015] S2, obtaining a binary hole position image according to the uniform illumination map, using it as an initial hole position region image, and determining each hole position region and its minimum circumscribed rectangle therein;

[0016] S3, sequentially expanding the minimum circumscribed rectangle of each hole region by a preset range to use as a mask to initialize the level set energy function, and iterating the function according to a preset number of iterations to obtain the contour curve of each hole region;

[0017] Wherein, the level set energy function is: E LG =E G +ω L E L , E L is the local grayscale energy, E G is the global grayscale energy, E L =∫ Ω H ε (Φ)(I(x,y)-u(x,y)) 2 dxdy+∫ Ω (1-H ε (Φ))(I(x,y)-v(x,y))2 dxdy,E G =∫ Ω H ε (Φ)(I(x,y)-u m ) 2 dxdy+∫ Ω (1-H ε (Φ))(I(x,y)-v m ) 2 dxdy,ω L For E L The weighted index, I(x,y) is the gray value of the pixel with coordinates (x,y), H ε (Φ) is the smoothed Heaviside function of the level set function Φ, u m and v m is the global average intensity of the inner and outer regions, Ω is the entire image area, u(x,y) is the intensity of the local internal area, and v(x,y) is the intensity of the local external area;

[0018] S4, performing two constant motions on the contour curve to obtain two amplified contour curves, searching for matching points of each pixel point on the two amplified contour curves in the absolute phase image of the projector to form matching point pairs; wherein the absolute phase values ​​of the matching point pairs are the same, and the row numbers of the pixel points on the absolute phase image of the monocular camera are the same as the row numbers of the matching points on the absolute phase image of the projector;

[0019] S5, performing three-dimensional reconstruction on the matching point pairs to obtain two reconstructed contours; performing reverse reconstruction based on the three-dimensional contour point clouds corresponding to each other on the two reconstructed contours to obtain the true contour point clouds of each hole area.

[0020] According to a third aspect of the present invention, there is provided a sheet metal part hole three-dimensional profile measurement system based on fringe projection, comprising: a computer-readable storage medium and a processor;

[0021] The computer-readable storage medium is used to store executable instructions;

[0022] The processor is configured to read the executable instructions stored in the computer-readable storage medium and execute the method according to the first aspect or the method according to the second aspect.

[0023] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0024] 1. When reconstructing the three-dimensional contour of the hole position of a sheet metal part, the thickness of the sheet metal part, the accuracy of the two-dimensional contour extraction, etc. are affected, resulting in inconsistencies in the hole contour extraction in the left and right camera images. The three-dimensional contour measurement method of the sheet metal part hole position based on fringe projection provided by the present invention can complete the contour three-dimensional reconstruction by extracting only the image from one camera, which can effectively reduce the problem of low three-dimensional contour reconstruction accuracy caused by inconsistent contour extraction from the left and right camera images.

[0025] 2. To address the problem of local reflection and strong glare easily occurring at the edges of sheet metal hole contours, making direct phase matching difficult, the present invention provides a fringe projection-based three-dimensional contour measurement method for sheet metal holes. This method, based on a contour reconstruction algorithm using curvilinear motion, does not directly match the original contour. Instead, it uses the two-dimensional contour of the hole extracted by a single camera, obtains two enlarged contours after two curvilinear motions, and then performs phase point matching at the enlarged contours to reconstruct a three-dimensional contour point cloud. Finally, the accurate true hole contour point cloud is reversely reconstructed based on the reconstructed enlarged three-dimensional contour. Therefore, the actual contour matching location is not at the edge of the hole, but in the surrounding sheet metal area, where the phase quality is better. Therefore, direct phase matching can be used to ensure the accuracy of contour stereo matching.

[0026] 3. To address the problem that there are multiple three-dimensional candidate points on the upper and lower edges of the hole contour, which are difficult to accurately match, the curve motion method proposed in the present invention makes it possible to directly use phase for contour matching, thereby eliminating the problem of reconstruction errors caused by the large number of candidate points in traditional grayscale matching methods.

[0027] 4. Existing research mainly focuses on the three-dimensional measurement of circular hole contours, which often requires ellipse fitting to improve accuracy and is difficult to apply to special-shaped holes such as waist-shaped holes. The present invention provides a three-dimensional contour measurement method for sheet metal holes based on fringe projection, which directly uses the level set algorithm for contour extraction and can be applied to contours of all shapes; at the same time, the present invention uses phase matching for stereo contour matching, and the accuracy can be improved to a certain extent compared with traditional methods. Therefore, there is no need to perform corresponding shape fitting to meet actual measurement needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a schematic diagram of a process flow of a three-dimensional profile measurement method for a sheet metal hole based on fringe projection provided by the present invention;

[0029] Figure 2 The second flow chart of the method for measuring the three-dimensional profile of a sheet metal hole based on fringe projection provided by the present invention;

[0030] Figure 3 (a) and (b) are respectively the uniform illumination image and the grating image of the sheet metal provided by the present invention;

[0031] Figure 4 (a) and (b) are respectively the initial hole position area image and the image after hole position area screening provided by the present invention;

[0032] Figure 5 (a), (b), and (c) are respectively the initial contour map, contour extraction iterative process map, and final contour map of the sub-pixel precision contour extraction process provided by the present invention;

[0033] Figure 6 (a) and (b) are respectively a curve motion diagram of a circular hole contour of a sheet metal part to be tested and a local enlarged diagram of the contour matching point provided by the present invention;

[0034] Figure 7 (a), (b), and (c) are respectively a hole position contour extraction diagram of the sheet metal to be tested, a local magnified diagram of hole position one in the hole position contour extraction diagram, and a local magnified diagram of hole position two in the hole position contour extraction diagram of the sheet metal to be tested provided by the present invention;

[0035] Figure 8 The present invention provides a point cloud image of the hole contour reconstruction of the sheet metal part to be tested. DETAILED DESCRIPTION

[0036] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0037] The embodiment of the present invention provides a three-dimensional profile measurement method of a sheet metal hole based on fringe projection, such as Figure 1-2 Shown, including:

[0038] S1, the left and right cameras synchronously obtain a uniform illumination image of the sheet metal to be tested; project a set of multi-frequency grating fringe images onto the sheet metal to be tested, and the left and right cameras synchronously obtain the fringe projection image formed on the surface of the sheet metal to be tested, and calculate the absolute phase image based on the fringe projection image.

[0039] Preferably, after calculating the absolute phase map according to the fringe projection map, the method further includes:

[0040] Distortion correction and epipolar correction are performed on both the uniform illumination image and the absolute phase image.

[0041] Specifically, step S1 is data preprocessing, first obtaining the uniform illumination map of the left and right cameras, such as Figure 3As shown in (a), it can be understood that the uniform illumination image is a grayscale image. Using the grating image (i.e., fringe projection image) as initial data, a relative phase image is calculated. Then, based on distortion correction and epipolar correction, the corrected uniform illumination images (grayscale images) for the left and right cameras and the corrected relative phase images for the left and right cameras are obtained.

[0042] The grating image is a deformed grating image that is synchronously acquired by a camera after a series of original grating fringe images are projected onto the surface of the sheet metal to be measured by a projector, such as Figure 3 As shown in (b) in .

[0043] Furthermore, the series of original grating fringe images have different phase shifts and different step sizes. For example, the original grating fringe image can be a three-frequency grating fringe image, but the step size of each frequency can be different, such as setting the first frequency to 76 and the step size to The phase shifts of the six grating images are as follows: The second frequency is set to 70, four raster images The third frequency is set to 65, four raster images The total number of raster images is 14.

[0044] Specifically, the calculation of the relative phase diagram is as follows:

[0045]

[0046] Where φ is the phase value, I n is the grating image intensity under the projection of the nth fringe image, δ n =2π(n-1) / N is the phase shift, N is the phase shift amount, and x, y are the pixel coordinates in the image.

[0047] The absolute phase map is calculated based on the relative phase maps of the three frequencies. That is, the relative phase maps of the left and right cameras at each frequency are obtained based on the fringe projection map, and the relative phase maps at any frequency are dephased using the relative phase maps at each frequency to obtain the absolute phase map.

[0048] S2, obtaining a binary hole position image according to the uniform illumination map of the left camera, using it as the initial hole position area image, and determining each hole position area and its minimum circumscribed rectangle.

[0049] Preferably, the step of obtaining a binary hole position image according to the uniform illumination map of the left camera includes:

[0050] The Ostu algorithm is used to adaptively identify and segment the initial position of the hole contour on the uniform illumination image of the left camera to obtain the first segmented image;

[0051] Set a threshold to perform binary segmentation on the uniform illumination map of the left camera and fill the holes to obtain the second segmented image;

[0052] A union operation is performed on the first segmented image and the second segmented image to obtain a binary hole position image.

[0053] Preferably, step S2 includes:

[0054] Performing a connected area search on the initial hole position area image to obtain all initial hole position areas;

[0055] According to the hole area pixel number threshold, the minimum bounding rectangle aspect ratio threshold and the hole area to minimum bounding rectangle area ratio threshold, all the initial hole areas are screened to obtain each hole area and its minimum bounding rectangle.

[0056] Specifically, step S2 performs initial contour positioning of the left camera. Using the calibrated uniformly illuminated image of the left camera (and the calibrated grayscale image of the left camera) as a reference, preliminary image segmentation is performed based on grayscale gradient and grayscale intensity to obtain the hole area. Then, preliminary contour screening is performed based on information such as the hole size and ratio, and adjacent rectangles of the hole are calculated.

[0057] The preliminary image segmentation based on grayscale gradient and grayscale intensity to obtain the hole area is specifically as follows:

[0058] (1) Using the Ostu algorithm, the initial position of the hole contour on the corrected left camera grayscale image is adaptively identified and segmented to obtain the first binary segmented image I1;

[0059] (2) Binarize and segment the corrected left camera grayscale image directly based on the grayscale gradient threshold, then fill the holes to obtain the second binary segmented image I2;

[0060] (3) Based on the first segmented image I1 and the second segmented image I2, the initial hole area image is obtained, as shown in Figure 4 As shown in (a) in .

[0061] Specifically, preliminary contour screening is performed based on information such as the size and proportion of the hole position, and the adjacent rectangles of the hole position (i.e., the minimum circumscribed rectangle of the hole position area) are calculated. Specifically,

[0062] (1) Searching for connected regions based on the initial hole region image to obtain all initial hole regions;

[0063] (2) Screening based on the number of pixels in the hole area. For example, if the pixel number threshold is set to 100-1000, then the area with a pixel number less than 100 or greater than 1000 will be considered as a non-hole area;

[0064] (3) Calculate the adjacent rectangles of each hole position and filter them based on the area aspect ratio. For example, if the aspect ratio is set to no more than 10, then the area with an aspect ratio greater than 10 will be considered as a non-hole area.

[0065] (4) Calculate the ratio of the area of ​​each hole region to the area of ​​the adjacent rectangle for screening. For example, if the ratio is set to 0.2, the area ratio less than 0.2 will be regarded as a noise contour and a non-hole region;

[0066] (5) Based on the above screening principles, obtain the hole area and its corresponding adjacent rectangles, such as Figure 4 As shown in (b) in .

[0067] S3, sequentially enlarging the minimum circumscribed rectangle of each hole region by a preset range to use as a mask to initialize the level set energy function, and iterating the function according to a preset number of iterations to obtain the contour curve of each hole region.

[0068] Specifically, step S3 performs sub-pixel precision contour positioning. Based on the screened contours, region segmentation is performed, and then a level set energy function is constructed that couples local grayscale energy with global grayscale energy. Multiple hole contours are calculated in parallel to obtain high-precision sub-pixel contours of the hole contours.

[0069] Region segmentation is performed based on the filtered contour. Specifically, the corresponding pixel matrix is ​​obtained according to the adjacent rectangles of each hole area (that is, the minimum circumscribed rectangular area image is expanded and extracted, and the preset range is expanded, for example, the length and width are increased by 20 pixels). Then, the sub-pixel precision contour is extracted based on the corresponding pixel matrix, and finally transformed into the original image.

[0070] The level set energy function for coupling local grayscale energy and global grayscale energy is constructed as follows:

[0071] E LG =E G +ω L E L (2)

[0072] E L =∫ Ω H ε (Φ)(I(x,y)-u(x,y)) 2 dxdy+∫ Ω (1-H ε (Φ))(I(x,y)-v(x,y)) 2 dxdy (3)

[0073] E G =∫ Ω H ε (Φ)(I(x,y)-um ) 2 dxdy+∫ Ω (1-H ε (Φ))(I(x,y)-v m ) 2 dxdy (4)

[0074] Among them E L is the local grayscale energy, E G is the global grayscale energy, ω L For E L The corresponding "weighted" index, I is the grayscale image, H ε (Φ) is the smoothed Heaviside function of the level set function Φ, ε is the parameter of the Heaviside function, ε is a fixed parameter, u and v are the average intensity of the local area and the inner and outer areas, u m and v m is the global average intensity of the local area inside and outside the region. L Can be set to 2.

[0075] Specifically, if Figure 5 As shown in (a), (b), and (c), the high-precision sub-pixel contour of the hole position is obtained, specifically:

[0076] (1) Construct an initial mask (i.e., the extracted minimum bounding rectangle image) based on the initial hole area image and initialize the level set energy function;

[0077] (2) Based on the constructed level set energy function coupling local grayscale energy and global grayscale energy, iteratively optimize the level set energy function;

[0078] (3) After iterating to the specified number of steps, the final contour is obtained.

[0079] S4: Perform two constant motions on the contour curve to obtain two amplified contour curves, and search for matching points of each pixel point on the two amplified contour curves in the absolute phase image of the right camera to form matching point pairs; wherein the absolute phase values ​​of the matching point pairs are the same, and the row number of the pixel point in the absolute phase image of the left camera is the same as the row number of the matching point in the absolute phase image of the right camera.

[0080] Preferably, before searching for matching points of each pixel point on the two amplified contour curves in the absolute phase image of the right camera, the method further includes:

[0081] The mask matrix obtained by binarizing and segmenting the uniform illumination map of the right camera is multiplied by the absolute phase map of the right camera.

[0082] Specifically, the extracted contour curve is subjected to two constant motions toward the surrounding sheet metal area to obtain two enlarged contours; then, phase point matching is performed at the enlarged contours using the absolute phase map of the right camera to reconstruct a high-quality three-dimensional point cloud; finally, the true contour point cloud is reversely reconstructed based on the corresponding points of the two reconstructed contours.

[0083] like Figure 6 As shown in (a) and (b), the extracted contour curve is further subjected to two constant motions toward the surrounding sheet metal area to obtain two enlarged contours, specifically:

[0084] C m =C r +VN o (5)

[0085] Among them C r is the actual contour point, C m is the contour point after constant motion, V is the constant speed, N o is the normal vector pointing to the outside of the contour curve; when calculating the normal vector, it is necessary to sample the contour points to ensure that the distances between adjacent points are consistent. Preferably, the speeds of the two constant movements can be set to 3.5 and 3.4.

[0086] Furthermore, phase point matching is performed at the magnified contour using the absolute phase image, specifically:

[0087] (1) Based on the corrected grayscale image of the right camera, binary segmentation is performed to remove the areas with smaller grayscale. For example, threshold segmentation can be performed based on a grayscale of 20. The pixel values ​​of pixels with a grayscale greater than 20 are set to 1, and the pixel values ​​of pixels with a grayscale less than or equal to 20 are set to 0, thereby obtaining a mask matrix.

[0088] (2) Multiplying the mask matrix with the corrected absolute phase map of the right camera to obtain the phase map to be processed, so as to reduce the probability of phase mismatch;

[0089] (3) Search the corresponding rows of the phase image to be processed based on the absolute phase values ​​of each pixel point at the two magnified contours on the absolute phase image of the right camera, obtain the corresponding matching points (matching points with the same absolute phase value as the pixel point), and obtain matching point pairs; during the search, if the phase is 0, continue to search for the next point until a suitable phase is found or all points on the corresponding rows are searched.

[0090] S5, performing three-dimensional reconstruction on the matching point pairs to obtain two reconstructed contours; performing reverse reconstruction based on the three-dimensional contour point clouds corresponding to each other on the two reconstructed contours to obtain the true contour point clouds of each hole area.

[0091] Specifically, the matching point pairs are 3D reconstructed to obtain two 3D point clouds of reconstructed contours, and reverse reconstruction is performed based on the corresponding 3D contour point clouds on the two reconstructed contours to obtain the true contour point clouds of each hole area, such as Figure 7 (a), (b), (c) and Figure 8 shown.

[0092] It can be understood that the points on the two reconstructed contours are the same in number and correspond one to one.

[0093] Preferably, in step S5, reverse reconstruction is performed according to the following formula:

[0094] P r =P1-(P1-P2)V1 / (V1-V2) (6)

[0095] Among them, P r is the real 3D contour point, P1 and P2 are the corresponding 3D contour points on the two reconstructed contours respectively.

[0096] It is understandable that in the above method, the left and right cameras can also be interchanged, that is, in step S2, a binary hole position image is obtained according to the uniform illumination map of the right camera, and in step S4, matching is performed according to the absolute phase map of the left camera.

[0097] The following describes the three-dimensional profile measurement device for sheet metal holes based on fringe projection provided by the present invention. The three-dimensional profile measurement device for sheet metal holes based on fringe projection described below and the three-dimensional profile measurement method for sheet metal holes based on fringe projection described above can be referenced to each other.

[0098] An embodiment of the present invention provides a three-dimensional profile measurement device for sheet metal holes based on fringe projection, comprising:

[0099] The image acquisition module is used to synchronize the left and right cameras to obtain a uniform illumination image of the sheet metal part to be tested; project a set of multi-frequency grating fringe images onto the sheet metal part to be tested, and synchronize the left and right cameras to obtain the fringe projection images formed on the surface of the sheet metal part to be tested, and calculate the absolute phase image based on the fringe projection images.

[0100] Specifically, a group of multi-frequency grating fringe images are projected onto the surface of the workpiece to be measured through the projection module, and the fringe projection image formed on the surface of the workpiece to be measured is synchronously acquired by the left and right cameras; the image acquisition module acquires the relative phase images of the left and right cameras at each frequency and the modulation map of the left and right cameras at any frequency based on the fringe projection image; the relative phase image at each frequency is used to dephase the relative phase image at any frequency to acquire the absolute phase image.

[0101] The hole area acquisition module is used to obtain a binary hole image based on the uniform illumination map of the left camera, use it as the initial hole area image, and determine each hole area and its minimum circumscribed rectangle.

[0102] Specifically, the hole area acquisition module is used to adaptively identify and segment the initial position of the hole contour on the grayscale image of the left camera; remove the non-hole area according to information such as the size and proportion of the hole, and calculate the adjacent rectangle based on the selected hole.

[0103] A contour curve acquisition module is used to sequentially expand the minimum circumscribed rectangle of each hole area by a preset range as a mask to initialize the level set energy function, and iterate the function according to a preset number of iterations to obtain the contour curve of each hole area;

[0104] Wherein, the level set energy function is: E LG =E G +ω L E L , E L is the local grayscale energy, E G is the global grayscale energy, E L =∫ Ω H ε (Φ)(I(x,y)-u(x,y)) 2 dxdy+∫ Ω (1-H ε (Φ))(I(x,y)-v(x,y)) 2 dxdy,E G =∫ Ω H ε (Φ)(I(x,y)-u m ) 2 dxdy+∫ Ω (1-H ε (Φ))(I(x,y)-v m ) 2 dxdy,ω L For E L The weighted index, I(x,y) is the gray value of the pixel with coordinates (x,y), H ε (Φ) is the smoothed Heaviside function of the level set function Φ, which is generally expressed as

[0105]

[0106] ε is a fixed parameter, u m and v m The global average intensity of the inner and outer regions, Ω is the entire image area, u(x,y) is the local inner region intensity, and v(x,y) is the local outer region intensity.

[0107] Specifically, the contour curve acquisition module is used to extract the sub-pixel precision contour of the rectangular area containing the hole; to ensure the extraction accuracy, each hole contour is operated in parallel, and the level set algorithm is used to couple the local intensity with the global intensity to extract the sub-pixel contour of the hole position;

[0108] A matching module is configured to perform two constant motions on the contour curve to obtain two amplified contour curves, and search for matching points of each pixel point on the two amplified contour curves in the absolute phase image of the right camera to form matching point pairs; wherein the absolute phase values ​​of the matching point pairs are the same, and the row number of the pixel point in the absolute phase image of the left camera is the same as the row number of the matching point in the absolute phase image of the right camera.

[0109] Specifically, the matching module is used to perform phase point matching on the absolute phase image at the magnified contour to reconstruct a three-dimensional point cloud with better quality.

[0110] The three-dimensional reconstruction module is used to perform three-dimensional reconstruction on the matching point pairs to obtain two reconstructed contours; and perform reverse reconstruction based on the three-dimensional contour point clouds corresponding to each other on the two reconstructed contours to obtain the true contour point clouds of each hole area.

[0111] It is understandable that in the method provided in the above embodiment, the left and right cameras can be replaced with a monocular camera. Accordingly, in step S4, phase point matching is performed at the magnified contour using the absolute phase map of the monocular camera and the absolute phase map of the projector. The other steps are the same. Based on this, an embodiment of the present invention provides a three-dimensional contour measurement method for sheet metal hole positions based on fringe projection, comprising:

[0112] S1: A monocular camera obtains a uniform illumination image of the sheet metal to be tested; a set of multi-frequency grating fringe images is projected onto the sheet metal to be tested, and the monocular camera obtains a fringe projection image formed on the surface of the sheet metal to be tested, and an absolute phase image is calculated based on the fringe projection image.

[0113] S2, obtaining a binary hole position image according to the uniform illumination map, using it as an initial hole position region image, and determining each hole position region and its minimum circumscribed rectangle therein;

[0114] S3, sequentially expanding the minimum circumscribed rectangle of each hole region by a preset range to use as a mask to initialize the level set energy function, and iterating the function according to a preset number of iterations to obtain the contour curve of each hole region;

[0115] Wherein, the level set energy function is: E LG =E G +ω L E L , E Lis the local grayscale energy, E G is the global grayscale energy, E L =∫ Ω H ε (Φ)(I(x,y)-u(x,y)) 2 dxdy+∫ Ω (1-H ε (Φ))(I(x,y)-v(x,y)) 2 dxdy,E G =∫ Ω H ε (Φ)(I(x,y)-u m ) 2 dxdy+∫ Ω (1-H ε (Φ))(I(x,y)-v m ) 2 dxdy,ω L For E L The weighted index, I(x,y) is the gray value of the pixel with coordinates (x,y), H ε (Φ) is the smoothed Heaviside function of the level set function Φ, which is generally expressed as

[0116]

[0117] ε is a fixed parameter, u m and v m The global average intensity of the inner and outer regions, Ω is the entire image area, u(x,y) is the local inner region intensity, and v(x,y) is the local outer region intensity.

[0118] S4, performing two constant motions on the contour curve to obtain two amplified contour curves, searching for matching points of each pixel point on the two amplified contour curves in the absolute phase map of the projector to form matching point pairs; wherein the absolute phase values ​​of the matching point pairs are the same, and the row number of the pixel point on the absolute phase map of the monocular camera is the same as the row number of the matching point on the absolute phase map of the projector.

[0119] S5, performing three-dimensional reconstruction on the matching point pairs to obtain two reconstructed contours; performing reverse reconstruction based on the three-dimensional contour point clouds corresponding to each other on the two reconstructed contours to obtain the true contour point clouds of each hole area.

[0120] The following describes the three-dimensional profile measurement device for sheet metal holes based on fringe projection provided by the present invention. The three-dimensional profile measurement device for sheet metal holes based on fringe projection described below and the three-dimensional profile measurement method for sheet metal holes based on fringe projection described above can be referenced to each other.

[0121] An embodiment of the present invention provides a three-dimensional profile measurement device for sheet metal holes based on fringe projection, comprising:

[0122] The image acquisition module is used to enable the monocular camera to obtain a uniform illumination image of the sheet metal part to be tested; project a set of multi-frequency grating fringe images onto the sheet metal part to be tested, and the monocular camera obtains the fringe projection image formed on the surface of the sheet metal part to be tested, and calculate the absolute phase image based on the fringe projection image.

[0123] A hole area acquisition module is used to obtain a binary hole image according to the uniform illumination map, use it as an initial hole area image, and determine each hole area and its minimum circumscribed rectangle therein;

[0124] A contour curve acquisition module is used to sequentially expand the minimum circumscribed rectangle of each hole area by a preset range as a mask to initialize the level set energy function, and iterate the function according to a preset number of iterations to obtain the contour curve of each hole area;

[0125] Wherein, the level set energy function is: E LG =E G +ω L E L , E L is the local grayscale energy, E G is the global grayscale energy, E L =∫ Ω H ε (Φ)(I(x,y)-u(x,y)) 2 dxdy+∫ Ω (1-H ε (Φ))(I(x,y)-v(x,y)) 2 dxdy,E G =∫ Ω H ε (Φ)(I(x,y)-u m ) 2 dxdy+∫ Ω (1-H ε (Φ))(I(x,y)-v m ) 2 dxdy,ω L For E L The weighted index, I(x,y) is the gray value of the pixel with coordinates (x,y), H ε (Φ) is the smoothed Heaviside function of the level set function Φ, which is generally expressed as

[0126]

[0127] ε is a fixed parameter, u m and v mThe global average intensity of the inner and outer regions, Ω is the entire image area, u(x,y) is the local inner region intensity, and v(x,y) is the local outer region intensity.

[0128] A matching module is configured to perform two constant motions on the contour curve to obtain two amplified contour curves, and search for matching points of each pixel point on the two amplified contour curves in the absolute phase map of the projector to form matching point pairs; wherein the absolute phase values ​​of the matching point pairs are the same, and the row numbers of the pixel points on the absolute phase map of the monocular camera are the same as the row numbers of the matching points on the absolute phase map of the projector.

[0129] Specifically, the matching module is used to move the extracted contour curve twice toward the surrounding sheet metal area to obtain two enlarged contours; then, at the enlarged contours, the absolute phase image of the monocular camera is matched with the absolute phase image of the projector to reconstruct a high-quality three-dimensional point cloud; finally, the true contour point cloud is reversely reconstructed based on the corresponding points of the two reconstructed contours.

[0130] The extracted contour curve is subjected to two constant motions toward the surrounding sheet metal area to obtain two enlarged contours, as shown in formula (5).

[0131] Phase point matching is performed at the magnified contour using the absolute phase map, specifically by searching for corresponding rows on the projector phase map according to the phases of the pixels at the magnified contour to obtain corresponding matching points.

[0132] The three-dimensional reconstruction module is used to perform three-dimensional reconstruction on the matching point pairs to obtain two reconstructed contours; and perform reverse reconstruction based on the three-dimensional contour point clouds corresponding to each other on the two reconstructed contours to obtain the true contour point clouds of each hole area.

[0133] An embodiment of the present invention provides a sheet metal part hole position three-dimensional profile measurement system based on fringe projection, comprising: a computer-readable storage medium and a processor;

[0134] The computer-readable storage medium is used to store executable instructions;

[0135] The processor is configured to read the executable instructions stored in the computer-readable storage medium and execute the method described in any one of the above embodiments.

[0136] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A three-dimensional profile measurement method for sheet metal holes based on fringe projection, characterized in that: include: S1, the left and right cameras synchronously obtain a uniform illumination image of the sheet metal to be tested; project a set of multi-frequency grating fringe images onto the sheet metal to be tested, and the left and right cameras synchronously obtain the fringe projection image formed on the surface of the sheet metal to be tested, and calculate the absolute phase image based on the fringe projection image; S2, obtaining a binary hole image based on the uniform illumination map of the left camera, using it as the initial hole area image, and determining each hole area and its minimum circumscribed rectangle; S3, sequentially expanding the minimum circumscribed rectangle of each hole region by a preset range to use as a mask to initialize the level set energy function, and iterating the function according to a preset number of iterations to obtain the contour curve of each hole region; Wherein, the level set energy function is: E LG =E G +ω L E L , E L is the local grayscale energy, E G is the global grayscale energy, ω L For E L The weighted index, I(x,y) is the gray value of the pixel with coordinates (x,y), H ε (Φ) is the smoothed Heaviside function of the level set function Φ, u m and v m is the global average intensity of the inner and outer regions, Ω is the entire image area, u(x,y) is the intensity of the local internal area, and v(x,y) is the intensity of the local external area; S4, performing two constant motions on the contour curve to obtain two amplified contour curves, searching for matching points of each pixel point on the two amplified contour curves in the absolute phase image of the right camera to form matching point pairs; wherein the matching point pairs have the same absolute phase value, and the row number of the pixel point in the absolute phase image of the left camera is the same as the row number of the matching point in the absolute phase image of the right camera; S5, performing three-dimensional reconstruction on the matching point pairs to obtain two reconstructed contours; performing reverse reconstruction based on the three-dimensional contour point clouds corresponding to each other on the two reconstructed contours to obtain the true contour point clouds of each hole area.

2. The method according to claim 1, wherein After calculating the absolute phase map according to the fringe projection map, the method further includes: Distortion correction and epipolar correction are performed on both the uniform illumination image and the absolute phase image.

3. The method according to claim 1, wherein Before searching for matching points of each pixel point on the two amplified contour curves in the absolute phase image of the right camera, the method further includes: The mask matrix obtained by binarizing and segmenting the uniform illumination map of the right camera is multiplied by the absolute phase map of the right camera.

4. The method according to any one of claims 1 to 3, wherein In step S5, reverse reconstruction is performed according to the following formula: P r =P1-(P1-P2)V1 / (V1-V2); Among them, P r is the real 3D contour point, P1 and P2 are the corresponding 3D contour points on the two reconstructed contours respectively.

5. The method according to claim 1, wherein The step of obtaining a binary hole position image according to the uniform illumination map of the left camera includes: The Ostu algorithm is used to adaptively identify and segment the initial position of the hole contour on the uniform illumination image of the left camera to obtain the first segmented image; Set a threshold to perform binary segmentation on the uniform illumination map of the left camera and fill the holes to obtain the second segmented image; A union operation is performed on the first segmented image and the second segmented image to obtain a binary hole position image.

6. The method according to claim 1, wherein The step S2 comprises: Performing a connected area search on the initial hole position area image to obtain all initial hole position areas; According to the hole area pixel number threshold, the minimum bounding rectangle aspect ratio threshold and the hole area to minimum bounding rectangle area ratio threshold, all the initial hole areas are screened to obtain each hole area and its minimum bounding rectangle.

7. The method according to claim 1, wherein The contour curve is subjected to constant motion according to the following formula to obtain the enlarged contour curve: C m =C r +VN o ; Among them, C r is a point on the contour curve, C m is a point on the enlarged contour curve, V is the constant speed of constant motion, N o is the normal vector pointing outside the contour curve.

8. A three-dimensional profile measurement method for sheet metal holes based on fringe projection, characterized in that: include: S1, a monocular camera obtains a uniform illumination image of the sheet metal to be tested; a set of multi-frequency grating fringe images are projected onto the sheet metal to be tested, the monocular camera obtains a fringe projection image formed on the surface of the sheet metal to be tested, and an absolute phase image is calculated based on the fringe projection image; S2, obtaining a binary hole position image according to the uniform illumination map, using it as an initial hole position region image, and determining each hole position region and its minimum circumscribed rectangle therein; S3, sequentially expanding the minimum circumscribed rectangle of each hole region by a preset range to use as a mask to initialize the level set energy function, and iterating the function according to a preset number of iterations to obtain the contour curve of each hole region; Wherein, the level set energy function is: E LG =E G +ω L E L , E L is the local grayscale energy, E G is the global grayscale energy, ω L For E L The weighted index, I(x,y) is the gray value of the pixel with coordinates (x,y), H ε (Φ) is the smoothed Heaviside function of the level set function Φ, u m and v m is the global average intensity of the inner and outer regions, Ω is the entire image area, u(x,y) is the intensity of the local internal area, and v(x,y) is the intensity of the local external area; S4, performing two constant motions on the contour curve to obtain two amplified contour curves, searching for matching points of each pixel point on the two amplified contour curves in the absolute phase image of the projector to form matching point pairs; wherein the absolute phase values ​​of the matching point pairs are the same, and the row numbers of the pixel points on the absolute phase image of the monocular camera are the same as the row numbers of the matching points on the absolute phase image of the projector; S5, performing three-dimensional reconstruction on the matching point pairs to obtain two reconstructed contours; performing reverse reconstruction based on the three-dimensional contour point clouds corresponding to each other on the two reconstructed contours to obtain the true contour point clouds of each hole area.

9. A sheet metal hole three-dimensional profile measurement system based on fringe projection, characterized in that: include: Computer-readable storage medium and processor; The computer-readable storage medium is used to store executable instructions; The processor is configured to read the executable instructions stored in the computer-readable storage medium, and execute the method according to any one of claims 1 to 7, or execute the method according to claim 8.

Images