A high-precision measurement method based on image enhancement and improved optical flow method
By using image enhancement and improved optical flow methods, the problem of detection accuracy caused by high gloss and weak texture of composite material products was solved, and high-precision three-dimensional reconstruction and fiber bundle trajectory measurement were achieved, with an accuracy of 0.03 mm and an error of less than 0.1%.
Patent Information
- Application Number
- CN202311064807.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-23
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-08-23
AI Technical Summary
Existing technologies for non-contact inspection of composite material products, especially composite material lay-up parts and carbon fiber wound shells, are affected by highlights and weak textures, resulting in incomplete image reconstruction and making it impossible to achieve high-precision morphology inspection.
A method based on image enhancement and improved optical flow is adopted. By calibrating a binocular camera and calculating the fundamental matrix through epipolar geometry, and combining directional initial estimation with limit constraints and sub-pixel matching with feature enhancement, high-precision image point matching and 3D spatial information reconstruction are achieved.
High-precision 3D reconstruction was achieved in high-gloss and low-texture environments, with a reconstruction accuracy error of 0.03 mm and a fiber bundle trajectory measurement error of less than 0.1%, meeting the precision requirements for composite material molding and manufacturing.
Smart Images

Figure CN117232429B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the fields of optical experimental mechanics and digital photogrammetry, and relates to a high-precision measurement method based on image enhancement and improved optical flow method. Background Technology
[0002] In industrial production, some composite material products, due to their large elastic deformation, undergo significant deformation when clamped in fixtures. However, CNC machine tools still process them along ideal paths, leading to assembly errors during subsequent assembly. To ensure assembly quality, non-contact inspection of the material processing area is necessary. Existing non-contact inspection methods mainly utilize laser scanners and machine vision-based image reconstruction methods. The main drawback of laser scanning inspection is its low efficiency and inability to meet the required processing accuracy. Machine vision-based image reconstruction methods offer advantages such as high precision, speed, and complete information in non-contact inspection, making them highly advantageous for inspecting composite material processing. However, many industrial products, such as composite material lay-up parts and carbon fiber wound shells, exhibit significant specular reflection during image acquisition due to uneven distribution of natural light on the surface being processed—a phenomenon known as highlights. Simultaneously, these surfaces often have weak textures, which industrial cameras cannot reconstruct. Both highlights and weak textures result in incomplete image reconstruction, hindering high-precision topography inspection.
[0003] Therefore, in order to meet the actual needs of engineering, there is an urgent need for a three-dimensional reconstruction method for images of objects with both weak textures and highlights, so as to achieve high-precision measurement of the surface of structural components. Summary of the Invention
[0004] Technical problems to be solved
[0005] To avoid the shortcomings of existing technologies, this invention proposes a high-precision measurement method based on image enhancement and improved optical flow. This method utilizes optical detection techniques to perform high-precision measurements on the surface of complex structural parts, which may have weak textures or reflectivity. It effectively detects matching points while ensuring high measurement accuracy.
[0006] Technical solution
[0007] A high-precision measurement method based on image enhancement and improved optical flow method is characterized by the following steps:
[0008] Step 1: Calibrate the stereo camera by acquiring images from the calibration board, and obtain the intrinsic parameter matrix K of the stereo camera. l K r and the extrinsic parameter matrix [R] l T l ]、[R r T r ];
[0009] Then, the surface of the weak textured, highly reflective structural component to be measured with high precision is photographed to obtain a stereoscopic image of the component to be measured, namely the left and right views. The image captured by the left camera is used as the reference image, and the image captured by the right camera is used as the target image.
[0010] Step 2: Based on the intrinsic and extrinsic parameter matrices of the stereo camera obtained after calibration, calculate the fundamental matrices of the left and right views using epipolar geometry. in, The inverse of the intrinsic parameter matrix of the left camera is... Let R be the transpose inverse of the intrinsic parameter matrix of the right camera, and let R be the rotation matrix in the extrinsic parameters of the camera. × ] is the antisymmetric matrix of the translation matrix;
[0011] Step 3: Select a region of interest in the reference image, and then select image points on the surface of the structural component to be measured within that region to form a sequence of image point coordinates;
[0012] Step 4: Perform initial orientation estimation of the image point coordinate sequence based on limit constraints:
[0013] First, using the fundamental matrix calculated in step 2, the corresponding search range of image points in the reference image on the target image is obtained. Then, within this range, template matching is used to make accurate estimations, thereby obtaining the integer pixel-level image point coordinate sequence on the target image corresponding to the reference image.
[0014] Step 5: Perform feature-enhanced sub-pixel matching on the integer pixel-level image point coordinate sequence:
[0015] First, a multi-directional fusion neighboring pixel binarization model is used to enhance the image features of the reference image and the target image to obtain a measurement image with better features. Then, an improved deformation optical flow method is used to perform sub-pixel level fine matching on the integer pixel level image point coordinate sequence of the target image to obtain a high-precision sub-pixel level image point coordinate sequence.
[0016] Step 6: Reconstruct three-dimensional spatial information using the image point coordinate sequence obtained in Step 3 and the sub-pixel level image point coordinate sequence obtained in Step 5: Match the two sets of image point coordinate sequences to obtain image point pair sequences. Perform spatial triangulation calculation on the image point pair sequences to obtain spatial three-dimensional point coordinate information. Calculate the high-precision surface parameters of the weak-texture high-reflectivity structural component based on the three-dimensional point coordinates. Reconstruct the high-precision surface of the weak-texture high-reflectivity structural component using the calculated high-precision surface parameters.
[0017] The accuracy error of the reconstructed high-precision surface is 0.03 mm.
[0018] The epipolar constraint relationship in step 2 is as follows: Where F is the fundamental matrix of the binocular vision system, [x L y L 1] and [x] R y R [1] represents the homogeneous coordinates of the left image and the corresponding homogeneous coordinates of the right image, respectively.
[0019] The specific process of the orientation initial estimation based on limit constraints in step 4 is as follows: The epipolar constraint relationship is derived using the fundamental matrix F calculated in step 2, and an auxiliary search is performed on the target image; then, the initial point is evaluated using the zero-mean normalized cross-correlation function to obtain the integer-pixel level coordinate point with the highest matching degree; the zero-mean normalized cross-correlation function is: Where f(x,y) represents the gray value of any pixel in the (2M-1)×(2M-1) pixel reference sub-region, and g(x',y') represents the sub-pixel gray value of the corresponding point in the deformed image sub-region. m and g m These represent the average grayscale value of pixels within a sub-region of the image.
[0020] The specific process of sub-pixel matching based on feature enhancement in step 5 is as follows: Feature enhancement is performed on the reference image and the target image, denoted as I1 and I2 respectively; the pixels in images I1 and I2 are binarized according to the magnitude of the gray values of their neighboring pixels relative to the gray value of the center pixel, resulting in the binarized b(I... i -I c The binarized results of the eight nearest pixels are encoded to obtain the BMNP grayscale value of the center point. Referring to the original BMNP method, the single-direction pixel processing is generalized to the operation of superimposing grayscale values from multiple directions and then averaging them. By processing the region of interest using this model, an image with good feature quality can be obtained, denoted as . and exist and Based on the image, the image point coordinate sequence estimated in step 4 is used to perform sub-pixel-level high-precision matching calculation to obtain sub-pixel-level matching point pair coordinates.
[0021] The binarization is adopted Among them I i Let I be the grayscale value of the i-th neighboring pixel. c This represents the grayscale value of the center pixel.
[0022] The sub-pixel level high-precision matching calculation formula The
[0023]
[0024] Where x1 is the coordinate point on the left image, x2 is the coordinate point on the right image, Δx is the difference between the initial matching points on the left and right images, W(Δx,p) is a first-order shape function, and u and v are the displacement values of the center point p of the sub-region of the left image in the X and Y directions, respectively. x u y v x v y This represents the displacement gradient of a sub-region of the image.
[0025] The high-precision surface parameters of the weak textured high reflective structural component in step 6 are calculated as follows: using the binocular camera calibration parameters calculated in step 1, combined with the matching point pair coordinates calculated in step 5, the three-dimensional spatial coordinate points are reconstructed using the spatial triangulation method: x = PX, x' = P'X, where x(x,y) and x'(x',y') are the corresponding points of the left and right images, and their corresponding spatial points are X;
[0026] Expanding from x×(PX)=0, we get:
[0027]
[0028] Where P iT In the i-th row of P, the three equations are linearly dependent. Taking the first two equations, we can list two more equations to form an equation about X:
[0029]
[0030] Finally, the high-precision spatial coordinate point X is calculated, and the three-dimensional surface parameters of the structure are measured.
[0031] A computer device includes a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 8.
[0032] A computer-readable storage medium having an executable computer program stored thereon, characterized in that the executable computer program, when executed by a processor, implements the measurement method according to any one of claims 1-8.
[0033] Beneficial effects
[0034] This invention proposes a high-precision measurement method based on image enhancement and an improved optical flow method. This method effectively detects matching point pairs from images acquired by a binocular camera and reconstructs spatial three-dimensional coordinate points. The method includes the following steps: Step 1: Calibrate the binocular camera. Step 2: Calculate the fundamental matrix using intrinsic and extrinsic parameter matrices. Step 3: Perform BMNP model preprocessing on the acquired images. Step 4: Estimate initial points at the integer pixel level. Step 5: Calculate matching points at the sub-pixel level. Step 6: Calculate spatial three-dimensional points. This invention can effectively detect corresponding points in extreme environments such as weak textures and high highlights while maintaining high measurement accuracy, with a reconstruction accuracy error of 0.03 mm.
[0035] The beneficial effects of this invention are as follows: This invention proposes a high-precision measurement method based on image enhancement and improved optical flow, addressing the problem of poor measurement accuracy for industrial structural parts with weak texture or high reflectivity. By modifying traditional image matching methods, a "coarse-to-fine" image matching strategy is proposed. First, this paper proposes an initial matching point search estimation method, overcoming the low efficiency and poor accuracy caused by the need for global non-directional matching in traditional methods. It clarifies the search direction and search range of matching points, thereby improving the calculation accuracy and search efficiency of initial points. Second, this paper employs a feature enhancement method to reduce the influence of interference, while improving the iterative optical flow method by considering the affine transformation problem of sub-region images and adding shape functions for characterization, further improving the accuracy of sub-pixel matching points. In the fiber-reinforced composite shell winding measurement experiment, high-precision measurement of fiber bundle trajectories is performed. The three-dimensional trajectory reconstructed through matching points has a small error compared to the actual measured fiber bundle trajectory, with a relative error of 0.1%. These results demonstrate that the proposed method can be applied with high accuracy in the composite material molding and manufacturing process. Attached Figure Description
[0036] Figure 1 This is a flowchart of a high-precision measurement method based on image enhancement and improved optical flow method according to the present invention;
[0037] Figure 2 This is a schematic diagram of the epipolar constraint relationship of a binocular camera;
[0038] Figure 3 This is a reference image of the carbon fiber winding trajectory; the gray lines in this image represent a pre-selected sequence of trajectory image points.
[0039] Figure 4 This is a schematic diagram of limit constraint-assisted search;
[0040] Figure 5 This is a comparison of the image processing results before and after multi-directional BMNP model processing. In the left image, the shell surface texture is poor and reflective, making fiber trajectory recognition difficult; in the right image, after feature enhancement, the fiber trajectory has texture features that are easier to recognize and detect.
[0041] Figure 6 This is a schematic diagram of image point matching using the deformation optical flow method. The left image is the reference image, and the right image is the target image.
[0042] Figure 7 This is a schematic diagram of the subpixel-level matching process; image points at the same spatial location in the reference image and the target image have a high-precision matching effect;
[0043] Figure 8 These are the measurement results of the carbon fiber winding trajectory;
[0044] Figure 9 Experimental diagram of carbon fiber reinforced composite shell winding. Detailed Implementation
[0045] The present invention will now be further described in conjunction with the embodiments and accompanying drawings:
[0046] This invention is a high-precision measurement method based on image enhancement and improved optical flow, specifically for high-precision measurement of fiber bundle trajectories in carbon fiber reinforced composite shell winding experiments. The specific steps are as follows: The embodiments should provide specific examples of the tests you have performed. Figure 5 The image shown is an example of the carbon fiber winding trajectory detected using this algorithm.
[0047] Step 1: Calibrate the binocular cameras. First, acquire calibration images using the binocular industrial camera. Then, calibrate the cameras using Zhang Zhengyou's planar calibration method. Finally, calculate the intrinsic parameter matrix K for each binocular camera. l K r and the extrinsic parameter matrix [R] l T l ]、[R r T r ].
[0048] Step 2: Calculate the fundamental matrix using the intrinsic and extrinsic parameter matrices. This is combined with the derivation process of epipolar geometry, such as... Figure 2 As shown, the fundamental matrix F is obtained by using the intrinsic and extrinsic parameter matrices of the stereo camera calculated in step one, as shown in the following formula:
[0049]
[0050] in, The inverse of the intrinsic parameter matrix of the left camera is... Let R be the transpose inverse of the intrinsic parameter matrix of the right camera, and let R be the rotation matrix in the extrinsic parameters of the camera. × Let ] be the antisymmetric matrix of the translation matrix, expressed as follows:
[0051]
[0052] Step 3: Select the image point sequence to be measured. In the reference image, select each image point to be measured and obtain its two-dimensional coordinate information in the image coordinate system, such as... Figure 3 As shown.
[0053] Step 4: Initial Directional Estimation Based on Limit Constraints. An initial estimation of the target image is performed based on the sequence of points in the reference image. The epipolar constraint relationship is derived using the fundamental matrix F calculated in Step 2. An auxiliary search is performed on the right image based on this geometric relationship. The initial point is evaluated using a zero-mean normalized cross-correlation function to obtain the integer-pixel coordinates of the point with the highest matching degree. The epipolar constraint relationship is as follows: Figure 4 As shown, the following formula is used:
[0054]
[0055] Where F is the fundamental matrix of the binocular vision system, [x L y L 1] and [x] R y R [1] represents the homogeneous coordinates of the left image and the corresponding homogeneous coordinates of the right image, respectively.
[0056] The zero-mean normalized cross-correlation function is as follows:
[0057]
[0058] Where f(x,y) represents the gray value of any pixel in the (2M-1)×(2M-1) pixel reference sub-region, and g(x',y') represents the sub-pixel gray value of the corresponding point in the deformed image sub-region. m and g m These represent the average grayscale value of pixels within a sub-region of the image.
[0059] Step 5: Sub-pixel matching based on feature enhancement. Feature enhancement is performed on the reference image and the target image, denoted as I1 and I2 respectively. Pixels in the original images I1 and I2 are binarized according to the magnitude of their neighboring pixel grayscale values relative to the center pixel grayscale value, using b(I... i -I c To describe the result of this binarization:
[0060]
[0061] Among them I i Let I be the grayscale value of the i-th neighboring pixel. cThe gray value of the center pixel is given. After binarizing the eight neighboring pixels, the BMNP gray value of the center point is obtained by encoding the results of these eight binarizations. The gray value generated by the BMNP method is only related to the surrounding gray values, but the contribution of the binarized points calculated by the formula in the above method to the gray value of the center point is not uniform. For example, in the above process, the gray value provided by the top left pixel is 0 or 1, and the gray value provided by the bottom right pixel is 0 or 128. Such a large difference in contribution makes it difficult for the processed image to reflect the true correlation. To solve this problem, a multi-directional BMNP fusion estimation method is proposed. This method refers to the original BMNP method and extends the single-directional pixel processing process to the operation of superimposing gray values in multiple directions and then averaging them. By processing the region of interest with this model, an image with better feature quality can be obtained, denoted as . and like Figure 5 As shown.
[0062] exist and Based on the image, the image point coordinate sequence estimated in step 4 is used to perform sub-pixel-level high-precision matching calculations, such as... Figure 6 As shown. The calculation formula is as follows:
[0063]
[0064]
[0065] Where x1 is the coordinate point on the left image, x2 is the coordinate point on the right image, Δx is the difference between the initial matching points on the left and right images, W(Δx,p) is a first-order shape function, and u and v are the displacement values of the center point p of the sub-region of the left image in the X and Y directions, respectively. x u y v x v y This represents the displacement gradient of a sub-region of the image. The sub-pixel level coordinates of the matching point pairs can be calculated using the above formula, such as... Figure 7 As shown.
[0066] Step Six: High-precision calculation of the three-dimensional profile of the structure surface;
[0067] Using the binocular camera calibration parameters calculated in step one, and combining them with the coordinates of the matching point pairs calculated in step five, the 3D spatial coordinate points are reconstructed using spatial triangulation. The calculation formula is as follows:
[0068] x = PX, x' = P'X
[0069] Here, x(x,y) and x'(x',y') are the corresponding points in the left and right images, and their corresponding spatial points are X.
[0070] Expanding x×(PX)=0, we get:
[0071]
[0072] Where P iT This is the i-th row of P. The three equations are linearly dependent, so we only need to take the first two equations. Similarly, we can also list two more equations to form an equation about X:
[0073]
[0074] Finally, high-precision spatial coordinates X are calculated, and high-precision spatial trajectory information of the carbon fiber is obtained, such as... Figure 8 As shown.
[0075] To verify the effectiveness of this method, an experimental verification of the carbon fiber reinforced composite shell winding process was conducted.
[0076] Figure 9 The image shows the carbon fiber winding trajectory detected using this algorithm.
[0077] In summary, this invention discloses a high-precision measurement method based on image enhancement and improved optical flow, which solves the problems in extreme optical measurement environments such as weak texture or reflection. On the one hand, image enhancement is used to improve image quality, and on the other hand, improved optical flow is used to improve the measurement accuracy of corresponding coordinate point pairs in the image. This effectively solves the problem of high-precision 3D reconstruction in weak texture scenes, improves detection capability, and ensures the accuracy of point pair matching.
[0078] The examples described above illustrate the implementation steps of the present invention and should not be construed as limiting the present invention. Any improvements made based on the spirit of the present invention should be within the protection scope of the present invention.
Claims
1. A high-precision measurement method based on image enhancement and improved optical flow method, characterized in that... The steps are as follows: Step 1: Calibrate the stereo camera by acquiring images from the calibration board, and obtain the intrinsic parameter matrix K of the stereo camera. l K r and the extrinsic parameter matrix [R] l T l ]、[R r T r ]; Then, the surface of the weak textured, highly reflective structural component to be measured with high precision is photographed to obtain a stereoscopic image of the component to be measured, namely the left and right views. The image captured by the left camera is used as the reference image, and the image captured by the right camera is used as the target image. Step 2: Based on the intrinsic and extrinsic parameter matrices of the stereo camera obtained after calibration, calculate the fundamental matrices of the left and right views using epipolar geometry. in, The inverse of the intrinsic parameter matrix of the left camera is... Let R be the transpose inverse of the intrinsic parameter matrix of the right camera, and let R be the rotation matrix in the extrinsic parameters of the camera. × ] is the antisymmetric matrix of the translation matrix; Step 3: Select a region of interest in the reference image, and then select image points on the surface of the structural component to be measured within that region to form a sequence of image point coordinates; Step 4: Perform initial orientation estimation of the image point coordinate sequence based on limit constraints: First, using the fundamental matrix calculated in step 2, the corresponding search range of image points in the reference image on the target image is obtained. Then, within this range, template matching is used to make accurate estimations, thereby obtaining the integer pixel-level image point coordinate sequence on the target image corresponding to the reference image. Step 5: Perform feature-enhanced sub-pixel matching on the integer pixel-level image point coordinate sequence: First, a multi-directional fusion neighboring pixel binarization model is used to enhance the image features of the reference image and the target image to obtain a measurement image with better features. Then, an improved deformation optical flow method is used to perform sub-pixel level fine matching on the integer pixel level image point coordinate sequence of the target image to obtain a high-precision sub-pixel level image point coordinate sequence. Step 6: Reconstruct three-dimensional spatial information using the image point coordinate sequence obtained in Step 3 and the sub-pixel level image point coordinate sequence obtained in Step 5: Match the two sets of image point coordinate sequences to obtain image point pair sequences. Perform spatial triangulation calculation on the image point pair sequences to obtain spatial three-dimensional point coordinate information. Calculate the high-precision surface parameters of the weak-texture high-reflectivity structural component based on the three-dimensional point coordinates. Reconstruct the high-precision surface of the weak-texture high-reflectivity structural component using the calculated high-precision surface parameters.
2. The high-precision measurement method based on image enhancement and improved optical flow method according to claim 1, characterized in that: The accuracy error of the reconstructed high-precision surface is 0.03 mm.
3. The high-precision measurement method based on image enhancement and improved optical flow method according to claim 1, characterized in that: The epipolar constraint relationship in step 2 is as follows: Where F is the fundamental matrix of the binocular vision system, [x L y L 1] and [x] R y R [1] represents the homogeneous coordinates of the left image and the corresponding homogeneous coordinates of the right image, respectively.
4. The high-precision measurement method based on image enhancement and improved optical flow method according to claim 1, characterized in that: The specific process of the orientation initial estimation based on limit constraints in step 4 is as follows: The epipolar constraint relationship is derived using the fundamental matrix F calculated in step 2, and an auxiliary search is performed on the target image; then, the initial point is evaluated using the zero-mean normalized cross-correlation function to obtain the integer-pixel level coordinate point with the highest matching degree; the zero-mean normalized cross-correlation function is: Where f(x,y) represents the gray value of any pixel in the (2M-1)×(2M-1) pixel reference sub-region, and g(x',y') represents the sub-pixel gray value of the corresponding point in the deformed image sub-region. m and g m These represent the average grayscale value of pixels within a sub-region of the image.
5. The high-precision measurement method based on image enhancement and improved optical flow method according to claim 1, characterized in that: The specific process of sub-pixel matching based on feature enhancement in step 5 is as follows: Feature enhancement is performed on the reference image and the target image, denoted as I1 and I2 respectively; the pixels in images I1 and I2 are binarized according to the magnitude of the gray values of their neighboring pixels relative to the gray value of the center pixel, resulting in the binarized b(I... i -I c The binarized results of the eight nearest pixels are encoded to obtain the BMNP grayscale value of the center point. Referring to the original BMNP method, the single-direction pixel processing is generalized to the operation of superimposing grayscale values in multiple directions and then averaging them. By processing the region of interest using this model, an image with better feature quality can be obtained, denoted as... and exist and Based on the image, the image point coordinate sequence estimated in step 4 is used to perform sub-pixel-level high-precision matching calculation to obtain sub-pixel-level matching point pair coordinates.
6. The high-precision measurement method based on image enhancement and improved optical flow method according to claim 5, characterized in that: The binarization is adopted Among them I i Let I be the grayscale value of the i-th neighboring pixel. c This represents the grayscale value of the center pixel.
7. The high-precision measurement method based on image enhancement and improved optical flow method according to claim 5, characterized in that: The sub-pixel level high-precision matching calculation formula The Where x1 is the coordinate point on the left image, x2 is the coordinate point on the right image, Δx is the difference between the initial matching points on the left and right images, W(Δx,p) is a first-order shape function, and u and v are the displacement values of the center point p of the sub-region of the left image in the X and Y directions, respectively. x u y v x v y This represents the displacement gradient of a sub-region of the image.
8. The high-precision measurement method based on image enhancement and improved optical flow method according to claim 5, characterized in that: The high-precision surface parameters of the weak textured high reflective structural component in step 6 are calculated as follows: using the binocular camera calibration parameters calculated in step 1, combined with the matching point pair coordinates calculated in step 5, the three-dimensional spatial coordinate points are reconstructed using the spatial triangulation method: x = PX, x' = P'X, where x(x,y) and x'(x',y') are the corresponding points of the left and right images, and their corresponding spatial points are X; Expanding from x×(PX)=0, we get: Where P iT In the i-th row of P, the three equations are linearly dependent. Taking the first two equations, we can list two more equations to form an equation about X: Finally, the high-precision spatial coordinate point X is calculated, and the three-dimensional surface parameters of the structure are measured.
9. A computer device, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.
10. A computer-readable storage medium having an executable computer program stored thereon, characterized in that, When the executable computer program is executed by a processor, it implements the measurement method according to any one of claims 1-8.
Citation Information
Patent Citations
Polar correction based sub-pixel level phase three-dimensional matching method
CN104331897A
Binocular-vision-based rapid detection method for three-dimensional information of space object
CN105894499A
