Optical measurement method and system based on camera calibration and distortion model, and medium
By performing parameter calibration and distortion correction on the camera, the problem of camera calibration and distortion model being unable to suppress distortion errors was solved, thus improving the measurement accuracy of the optical measurement system.
Patent Information
- Application Number
- CN202510711035.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-29
- Publication Date
- 2025-10-21
AI Technical Summary
In existing technologies, camera calibration and distortion models are difficult to effectively suppress distortion errors, which affects the measurement accuracy of optical measurement systems.
By calibrating and correcting camera parameters, including taking multiple calibration images and distortion measurement images from different angles and positions, using calibration algorithms to analyze camera parameters and distortion models, extracting and correcting image features of the object to be measured, and finally performing 3D reconstruction to obtain accurate measurement results.
It reduces distortion error and improves the measurement accuracy of the optical measurement system.
Smart Images

Figure CN120820062A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of optical measurement technology, and in particular to an optical measurement method, system, and medium based on camera calibration and distortion model. Background Art
[0002] Industrial robots are increasingly used in precision manufacturing, but their global positioning accuracy and position drift impact manufacturing quality. Traditional optical measurement systems, such as laser trackers, offer high accuracy but are expensive and susceptible to environmental interference. Photogrammetry systems, which use multiple cameras for 3D measurement, offer advantages such as low cost and ease of operation. However, their accuracy is significantly affected by camera calibration and distortion models. Existing technologies struggle to effectively suppress distortion errors, impacting measurement accuracy. Summary of the Invention
[0003] The purpose of the embodiments of the present application is to provide an optical measurement method, system and medium based on camera calibration and distortion model, which reduces distortion error and improves measurement accuracy by performing parameter calibration and distortion correction on the camera.
[0004] The present application also provides an optical measurement method based on camera calibration and distortion model, including:
[0005] Based on the 3D calibration workpiece, multiple calibration images and distortion measurement images at different angles and positions are taken;
[0006] Analyze the calibration image and distortion measurement image based on the calibration algorithm to obtain the camera parameters and distortion model;
[0007] Acquire an image of the object to be measured based on a calibrated camera and extract image features of the object to be measured;
[0008] Performing distortion correction on the image features of the object to be measured based on the distortion model to obtain the corrected image features of the object to be measured;
[0009] The corrected image features of the object to be measured are reconstructed in three dimensions to obtain a measurement image, and the parameter information of the object to be measured is analyzed based on the measurement image to obtain a measurement result.
[0010] Optionally, in the optical measurement method based on camera calibration and distortion model described in the embodiment of the present application, capturing multiple calibration images and distortion measurement images at different angles and positions based on a three-dimensional calibration workpiece specifically includes:
[0011] Select a three-dimensional calibration workpiece with standard geometric features, the three-dimensional calibration workpiece includes a cube, a checkerboard, and the standard geometric features include regular corners or edges;
[0012] Set shooting lighting parameters, camera position and camera parameters;
[0013] Adjust the camera position to establish multiple shooting angles, sort the shooting angles, and adjust the shooting angles and camera positions in sequence according to the sorting results to shoot the 3D calibration workpiece;
[0014] Synchronously adjust camera parameters to obtain multiple calibration images and distortion measurement images;
[0015] Lighting parameters include light brightness and light uniformity, camera position includes horizontal position and vertical position, and camera parameters include camera focal length, aperture and exposure time.
[0016] Optionally, in the optical measurement method based on camera calibration and distortion model described in the embodiment of the present application, the calibration image and the distortion measurement image are analyzed based on a calibration algorithm to obtain camera parameters and the distortion model, specifically including:
[0017] Acquire the calibration image and extract the corner coordinates of the 3D calibration workpiece based on the corner detection algorithm;
[0018] Establish image coordinates and analyze the relationship between corner point coordinates and image coordinates based on the calibration algorithm;
[0019] Solve the camera's intrinsic parameter matrix and extrinsic parameter matrix based on the least squares method to obtain the initial values of the camera parameters;
[0020] According to the initial parameters and corner point coordinates, a distortion model is established, and the distortion parameters are solved through a nonlinear optimization algorithm;
[0021] Taking the reprojection error as the objective function, a nonlinear optimization algorithm is used for global optimization to obtain the calibrated camera parameters and distortion model.
[0022] Optionally, in the optical measurement method based on camera calibration and distortion model described in the embodiment of the present application, acquiring an image of the object to be measured based on a calibrated camera and extracting image features of the object to be measured specifically include:
[0023] Acquire an image of the object to be measured, and perform filtering processing on the image of the object to be measured based on a filtering algorithm, wherein the filtering algorithm includes median filtering, Gaussian filtering, or bilateral filtering;
[0024] Performing contrast enhancement on the filtered image of the object to be measured to obtain an enhanced image;
[0025] Extract edge features of the object to be measured in the enhanced image based on edge detection algorithm;
[0026] Extract the corner features of the object to be measured based on the corner detection algorithm;
[0027] The texture features of the object to be measured are extracted based on texture analysis method.
[0028] Optionally, in the optical measurement method based on camera calibration and distortion model described in the embodiment of the present application, performing distortion correction on the image features of the object to be measured based on the distortion model to obtain the corrected image features of the object to be measured specifically includes:
[0029] Analyze the image features of the object to be measured based on the distortion model to obtain the distortion parameters;
[0030] Establish an objective function based on the reprojection error and calculate the error value of the image feature of the object to be measured;
[0031] Determine whether the error value is greater than or equal to the set error threshold;
[0032] If it is greater than or equal to the set error threshold, a correction coefficient is generated, and the distortion parameter is corrected based on the correction coefficient to obtain the corrected image features of the object to be measured.
[0033] Optionally, in the optical measurement method based on camera calibration and distortion model described in the embodiment of the present application, three-dimensionally reconstructing the corrected image features of the object to be measured to obtain a measurement image specifically includes:
[0034] Obtain the corrected image features of the object to be measured, and calculate the similarity between feature points in different images based on feature point descriptors;
[0035] Based on the similarity analysis, the correspondence between feature points of different images is obtained to obtain matching feature point pairs;
[0036] Calculate the three-dimensional coordinates of all feature point pairs based on the triangulation method to generate a point cloud model;
[0037] Convert the point cloud model into a mesh model and optimize the mesh model to obtain a three-dimensional model. The optimization methods include removing redundant faces, repairing holes, and smoothing the surface.
[0038] The texture features are extracted based on the image of the object to be measured. According to the correspondence between the feature points, the texture features are mapped to the surface of the three-dimensional model for feature reconstruction to obtain the measurement image.
[0039] In a second aspect, an embodiment of the present application provides an optical measurement system based on camera calibration and distortion modeling, the system comprising: a memory and a processor, wherein the memory includes a program for an optical measurement method based on camera calibration and distortion modeling, and when the program for the optical measurement method based on camera calibration and distortion modeling is executed by the processor, the following steps are implemented:
[0040] Based on the 3D calibration workpiece, multiple calibration images and distortion measurement images at different angles and positions are taken;
[0041] Analyze the calibration image and distortion measurement image based on the calibration algorithm to obtain the camera parameters and distortion model;
[0042] Acquire an image of the object to be measured based on a calibrated camera and extract image features of the object to be measured;
[0043] Performing distortion correction on the image features of the object to be measured based on the distortion model to obtain the corrected image features of the object to be measured;
[0044] The corrected image features of the object to be measured are reconstructed in three dimensions to obtain a measurement image, and the parameter information of the object to be measured is analyzed based on the measurement image to obtain a measurement result.
[0045] Optionally, in the optical measurement system based on camera calibration and distortion model described in the embodiment of the present application, capturing multiple calibration images and distortion measurement images at different angles and positions based on a three-dimensional calibration workpiece specifically includes:
[0046] Select a three-dimensional calibration workpiece with standard geometric features, the three-dimensional calibration workpiece includes a cube, a checkerboard, and the standard geometric features include regular corners or edges;
[0047] Set shooting lighting parameters, camera position and camera parameters;
[0048] Adjust the camera position to establish multiple shooting angles, sort the shooting angles, and adjust the shooting angles and camera positions in sequence according to the sorting results to shoot the 3D calibration workpiece;
[0049] Synchronously adjust camera parameters to obtain multiple calibration images and distortion measurement images;
[0050] Lighting parameters include light brightness and light uniformity, camera position includes horizontal position and vertical position, and camera parameters include camera focal length, aperture and exposure time.
[0051] Optionally, in the optical measurement system based on camera calibration and distortion model described in the embodiment of the present application, the calibration image and the distortion measurement image are analyzed based on a calibration algorithm to obtain camera parameters and the distortion model, specifically including:
[0052] Acquire the calibration image and extract the corner coordinates of the 3D calibration workpiece based on the corner detection algorithm;
[0053] Establish image coordinates and analyze the relationship between corner point coordinates and image coordinates based on the calibration algorithm;
[0054] Solve the camera's intrinsic parameter matrix and extrinsic parameter matrix based on the least squares method to obtain the initial values of the camera parameters;
[0055] According to the initial parameters and corner point coordinates, a distortion model is established, and the distortion parameters are solved through a nonlinear optimization algorithm;
[0056] Taking the reprojection error as the objective function, a nonlinear optimization algorithm is used for global optimization to obtain the calibrated camera parameters and distortion model.
[0057] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, which includes an optical measurement method program based on camera calibration and distortion model. When the optical measurement method program based on camera calibration and distortion model is executed by a processor, the steps of the optical measurement method based on camera calibration and distortion model as described in any one of the above items are implemented.
[0058] As can be seen from the above, the embodiments of the present application provide an optical measurement method, system, and medium based on camera calibration and distortion model, which shoots multiple calibration images and distortion measurement images at different angles and positions based on a three-dimensional calibration workpiece; analyzes the calibration images and distortion measurement images based on a calibration algorithm to obtain camera parameters and a distortion model; acquires an image of the object to be measured based on the calibrated camera, and extracts the image features of the object to be measured; performs distortion correction on the image features of the object to be measured based on the distortion model to obtain corrected image features of the object to be measured; reconstructs the corrected image features of the object to be measured in three dimensions to obtain a measurement image, and analyzes parameter information of the object to be measured based on the measurement image to obtain a measurement result; by performing parameter calibration and distortion correction on the camera, the distortion error is reduced and the measurement accuracy is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.
[0060] Figure 1 A flowchart of an optical measurement method based on camera calibration and distortion model provided in an embodiment of the present application;
[0061] Figure 2 A structural diagram of an optical measurement system provided in an embodiment of the present application;
[0062] Figure 3 A schematic diagram of long-distance calibration provided in an embodiment of the present application;
[0063] Figure 4 A schematic diagram of close-range calibration provided in an embodiment of the present application;
[0064] Figure 5 A schematic diagram of distortion measurement provided in an embodiment of the present application;
[0065] Figure 6 This is a diagram of the distortion correction effect provided in an embodiment of the present application. DETAILED DESCRIPTION
[0066] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work fall within the scope of protection of the present application.
[0067] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.
[0068] Please refer to Figure 1 , Figure 1 This is a flowchart of an optical measurement method based on camera calibration and distortion model in some embodiments of the present application. The optical measurement method based on camera calibration and distortion model is used in a terminal device and includes the following steps:
[0069] S101, capturing multiple calibration images and distortion measurement images at different angles and positions based on a three-dimensional calibration workpiece;
[0070] S102, analyzing the calibration image and the distortion measurement image based on a calibration algorithm to obtain camera parameters and a distortion model;
[0071] S103, acquiring an image of the object to be measured based on the calibrated camera, and extracting image features of the object to be measured;
[0072] S104, performing distortion correction on the image features of the object to be measured based on the distortion model to obtain corrected image features of the object to be measured;
[0073] S105 , performing three-dimensional reconstruction on the corrected image features of the object to be measured to obtain a measurement image, and analyzing parameter information of the object to be measured based on the measurement image to obtain a measurement result.
[0074] It's important to note that when setting shooting parameters, you should consider the characteristics of the object being measured (such as size, color, and surface material) and the shooting environment (such as light intensity and distance). For example, for highly reflective objects, you might need to lower the camera's exposure; for smaller objects, you might need to adjust the focus to obtain a clear image.
[0075] Capture images of the object to be measured from different angles and positions to obtain comprehensive information about the object. For objects with complex shapes, it may be necessary to capture images from multiple perspectives to ensure that all parts of the object are clearly recorded.
[0076] Use a calibrated camera to shoot and ensure that the image resolution, clarity, and color reproduction meet the requirements. During the shooting process, try to keep the camera stable to avoid image blur.
[0077] According to an embodiment of the present invention, capturing multiple calibration images and distortion measurement images at different angles and positions based on a three-dimensional calibration workpiece specifically includes:
[0078] Select a three-dimensional calibration workpiece with standard geometric features, the three-dimensional calibration workpiece includes a cube, a checkerboard, and the standard geometric features include regular corners or edges;
[0079] Set shooting lighting parameters, camera position and camera parameters;
[0080] Adjust the camera position to establish multiple shooting angles, sort the shooting angles, and adjust the shooting angles and camera positions in sequence according to the sorting results to shoot the 3D calibration workpiece;
[0081] Synchronously adjust camera parameters to obtain multiple calibration images and distortion measurement images;
[0082] Lighting parameters include light brightness and light uniformity, camera position includes horizontal position and vertical position, and camera parameters include camera focal length, aperture and exposure time.
[0083] It should be noted that distortion measurement uses a tensioned nylon line as a straight reference. Shooting angles and positions are planned, and a detailed shooting plan is developed to ensure coverage of the camera's entire field of view and various possible working angles. For example, shooting can be performed from different horizontal angles (0°, 45°, 90°, etc.), vertical angles (elevation, depression), and distances. For a horizontally placed 3D calibration workpiece, one can first capture a set of images directly above it, then gradually change the camera's horizontal angle and height to capture multiple sets of images.
[0084] Take a sufficient number of images: To obtain accurate camera parameters, you generally need to take at least 10 to 20 images at different angles and positions. During the shooting process, try to make the calibration artifact occupy different positions and proportions in the images to fully reflect the camera's imaging characteristics.
[0085] When shooting, pay attention to adjusting the camera's parameters such as focal length, aperture and exposure time to make the image clear and with moderate contrast, and avoid overexposure or underexposure, as well as blur or ghosting.
[0086] According to an embodiment of the present invention, the calibration image and the distortion measurement image are analyzed based on a calibration algorithm to obtain camera parameters and a distortion model, which specifically includes:
[0087] Acquire the calibration image and extract the corner coordinates of the 3D calibration workpiece based on the corner detection algorithm;
[0088] Establish image coordinates and analyze the relationship between corner point coordinates and image coordinates based on the calibration algorithm;
[0089] Solve the camera's intrinsic parameter matrix and extrinsic parameter matrix based on the least squares method to obtain the initial values of the camera parameters;
[0090] According to the initial parameters and corner point coordinates, a distortion model is established, and the distortion parameters are solved through a nonlinear optimization algorithm;
[0091] Taking the reprojection error as the objective function, a nonlinear optimization algorithm is used for global optimization to obtain the calibrated camera parameters and distortion model.
[0092] It should be noted that the distortion model includes radial distortion and tangential distortion, and its formula is:
[0093] x corrected =x(1+k1r 2 +k2r 4 +k3r 6 )+2p1xy+p2(r 2 +2x 2 )
[0094] y corrected =y(1+k1r 2 +k2r 4 +k3r 6 )+p1(r 2 +2y 2 )+2p2xy
[0095] x corrected with y corrected Indicates the horizontal (X) and vertical (Y) coordinates of the image pixel after distortion correction.
[0096] Among them, xcorrected Indicates the horizontal coordinate of the corrected pixel in the image coordinate system; y corrected Indicates the vertical coordinate of the corrected pixel point in the image coordinate system; x, y are image coordinates, k1, k2, k3 are radial distortion coefficients, p1, p2 are tangential distortion coefficients, r is the distance from the pixel to the center of the image.
[0097] By measuring the corrected image, the distortion correction error is calculated using the following formula:
[0098]
[0099] Where RMS represents the distortion correction error; x i 、y i is the corrected coordinate, is the ideal straight line coordinate, and N is the number of measurement points.
[0100] Specifically, common calibration algorithms include Zhang's calibration method and Tsai's two-step method. Zhang's calibration method is widely used due to its ease of operation and high accuracy, and is suitable for most common camera calibration scenarios. Tsai's two-step method is more accurate in modeling lens distortion and is often used for industrial camera calibration with high accuracy requirements. Corner detection algorithms include Harris corner detection or Shi-Tomasi corner detection; nonlinear optimization algorithms include Levenberg-Marquardt algorithm.
[0101] According to an embodiment of the present invention, acquiring an image of an object to be measured based on a calibrated camera and extracting image features of the object to be measured specifically include:
[0102] Acquire an image of the object to be measured, and perform filtering processing on the image of the object to be measured based on a filtering algorithm, wherein the filtering algorithm includes median filtering, Gaussian filtering, or bilateral filtering;
[0103] Performing contrast enhancement on the filtered image of the object to be measured to obtain an enhanced image;
[0104] Extract edge features of the object to be measured in the enhanced image based on edge detection algorithm;
[0105] Extract the corner features of the object to be measured based on the corner detection algorithm;
[0106] The texture features of the object to be measured are extracted based on texture analysis method.
[0107] It should be noted that image preprocessing includes the following:
[0108] Denoising: Denoising is performed on the acquired image to reduce the impact of noise on subsequent feature extraction. Common denoising methods include median filtering, Gaussian filtering, and bilateral filtering. For example, median filtering can effectively remove salt and pepper noise, while Gaussian filtering is suitable for removing Gaussian noise.
[0109] Enhance contrast: Through image enhancement techniques such as histogram equalization and adaptive histogram equalization, the contrast of the image is enhanced to make the details of the object clearer and facilitate feature extraction.
[0110] Normalization: Normalize the pixel values of an image to a certain range (such as [0, 1] or [0, 255]) to ensure consistency between different images and improve the accuracy of subsequent processing.
[0111] The specific method of feature extraction is as follows:
[0112] Edge-based feature extraction: Use edge detection algorithms (such as the Canny operator, Sobel operator, and Laplacian operator) to extract edge features of an object. Edges are important features of an object, reflecting its shape and outline. For example, the Canny operator is a commonly used edge detection algorithm that finds edge points in an image by calculating the gradient magnitude and direction.
[0113] Corner-based feature extraction: Corner detection algorithms (such as Harris corner detection and Shi-Tomasi corner detection) are used to extract the corner features of an object. Corners are distinct points in an image, typically located at the corners of an object or at the intersection of edges. These corners play an important role in tasks such as image matching and object tracking.
[0114] Region-based feature extraction: Use region segmentation algorithms (such as threshold segmentation, region growing, watershed algorithm, etc.) to segment the image into different regions, and then extract the features of each region, such as area, perimeter, center of mass, shape descriptor, etc. These regional features can be used for object classification and recognition.
[0115] Texture-based feature extraction: This method uses texture analysis methods (such as gray-level co-occurrence matrix and local binary pattern) to extract texture features of an object. Texture is an important characteristic of an object's surface, reflecting its material and surface structure. For example, the gray-level co-occurrence matrix can be used to extract texture features by calculating the co-occurrence relationship between different grayscale values in an image.
[0116] Deep learning-based feature extraction: This approach uses deep learning models such as convolutional neural networks (CNNs) to automatically extract object features. By feeding pre-processed images into a pre-trained CNN model, the model can learn high-level semantic features of the image, which have excellent performance in tasks such as object recognition and classification.
[0117] According to an embodiment of the present invention, performing distortion correction on the image features of the object to be measured based on the distortion model to obtain the corrected image features of the object to be measured specifically includes:
[0118] Analyze the image features of the object to be measured based on the distortion model to obtain the distortion parameters;
[0119] Establish an objective function based on the reprojection error and calculate the error value of the image feature of the object to be measured;
[0120] Determine whether the error value is greater than or equal to the set error threshold;
[0121] If it is greater than or equal to the set error threshold, a correction coefficient is generated, and the distortion parameter is corrected based on the correction coefficient to obtain the corrected image features of the object to be measured.
[0122] According to an embodiment of the present invention, three-dimensional reconstruction of the corrected image features of the object to be measured to obtain a measurement image specifically includes:
[0123] Obtain the corrected image features of the object to be measured, and calculate the similarity between feature points in different images based on feature point descriptors;
[0124] Based on the similarity analysis, the correspondence between feature points of different images is obtained to obtain matching feature point pairs;
[0125] Calculate the three-dimensional coordinates of all feature point pairs based on the triangulation method to generate a point cloud model;
[0126] Convert the point cloud model into a mesh model and optimize the mesh model to obtain a three-dimensional model. The optimization methods include removing redundant faces, repairing holes, and smoothing the surface.
[0127] The texture features are extracted based on the image of the object to be measured. According to the correspondence between the feature points, the texture features are mapped to the surface of the three-dimensional model for feature reconstruction to obtain the measurement image.
[0128] It should be noted that after calibration, feature points are extracted from the multiple images of the object to be measured using a feature point extraction algorithm (such as SIFT, SURF, ORB, etc.). These feature points should be unique and stable, and can be accurately identified under different viewing angles and lighting conditions.
[0129] The similarity between feature points in different images is calculated using feature point descriptors (such as SIFT descriptors). The corresponding relationship between feature points in different images is found through matching algorithms (such as nearest neighbor matching and bidirectional matching). To improve matching accuracy, the RANSAC (Random Sample Consensus) algorithm can be used to eliminate incorrect matching points.
[0130] Based on the matched feature point pairs, the relative position and pose relationship between the images is calculated using epipolar geometry principles (such as the essential matrix and the fundamental matrix). The essential matrix describes the rotation and translation relationship between the two camera coordinate systems, while the fundamental matrix establishes the correspondence between the two images in the pixel coordinate system.
[0131] Triangulation involves using known camera intrinsic parameters and calculated extrinsic parameters, combined with matched feature point pairs, to calculate the 3D coordinates of feature points in space. Specifically, this involves using the projection relationship of feature points from different perspectives to infer their position in the world coordinate system.
[0132] Common methods for converting point cloud models into mesh models include Delaunay triangulation and Poisson reconstruction. Delaunay triangulation forms a triangular mesh by connecting points in the point cloud, while Poisson reconstruction constructs a smoother surface model based on the normal vector information of the point cloud.
[0133] Based on the measurement requirements, select the appropriate virtual perspective to render the 3D reconstructed model and generate a measurement image. During the rendering process, annotations and dimension information can be added to facilitate object measurement and analysis.
[0134] like Figure 2-Figure 6 As shown, in a second aspect, an embodiment of the present application provides an optical measurement system based on camera calibration and distortion model, the system comprising: a memory and a processor, the memory comprising a program for an optical measurement method based on camera calibration and distortion model, and the program for the optical measurement method based on camera calibration and distortion model, when executed by the processor, implementing the following steps:
[0135] Based on the 3D calibration workpiece, multiple calibration images and distortion measurement images at different angles and positions are taken;
[0136] Analyze the calibration image and distortion measurement image based on the calibration algorithm to obtain the camera parameters and distortion model;
[0137] Acquire an image of the object to be measured based on a calibrated camera and extract image features of the object to be measured;
[0138] Performing distortion correction on the image features of the object to be measured based on the distortion model to obtain the corrected image features of the object to be measured;
[0139] The corrected image features of the object to be measured are reconstructed in three dimensions to obtain a measurement image, and the parameter information of the object to be measured is analyzed based on the measurement image to obtain a measurement result.
[0140] It should be noted that the optical measurement system includes the following components:
[0141] Infrared camera: It consists of two wide-angle infrared cameras and is used to photograph the measurement target.
[0142] Calibration device: includes a three-dimensional calibration workpiece with multiple reflective balls arranged on it for camera calibration.
[0143] Robot: Used to move the calibration device and measurement targets.
[0144] Laser tracker: used to verify the accuracy of the measurement system. The laser tracker is used to verify the accuracy of the measurement system and further optimize the calibration process and distortion model based on the verification results.
[0145] Data processing unit: used to process image data, calculate camera parameters and distortion models.
[0146] Specifically, the calibration method includes the following steps:
[0147] Calibration process optimization: Optimize the calibration process by adjusting the density and distribution of the calibration image. Specifically, it includes:
[0148] Front Weighted Rectification (FWR): Increases image density close to the camera.
[0149] Uniform Distribution Rectification (ESR): distributes the image uniformly across the camera's field of view.
[0150] Post-weighted calibration process (BWR): Increases image density at locations far from the camera.
[0151] Distortion model optimization: Optimize the distortion model by measuring image distortion.
[0152] Distortion measurement device: Use tensioned nylon thread as a straight line reference to measure image distortion.
[0153] Distortion correction: According to the measurement results, the distortion model parameters are adjusted to reduce the distortion correction error.
[0154] The data processing unit processes the image data through the following steps:
[0155] Image preprocessing: Apply anti-vignetting filter to enhance image edge brightness.
[0156] Contour extraction: Use a binary mask to extract the contour of the reflective tape.
[0157] Line center extraction: The center point of the nylon line is extracted by inverting the pixel intensity signal.
[0158] Distortion Correction: Correct image distortion based on the camera calibration matrix and distortion parameters.
[0159] It's important to note that when setting shooting parameters, you should consider the characteristics of the object being measured (such as size, color, and surface material) and the shooting environment (such as light intensity and distance). For example, for highly reflective objects, you might need to lower the camera's exposure; for smaller objects, you might need to adjust the focus to obtain a clear image.
[0160] Capture images of the object to be measured from different angles and positions to obtain comprehensive information about the object. For objects with complex shapes, it may be necessary to capture images from multiple perspectives to ensure that all parts of the object are clearly recorded.
[0161] Use a calibrated camera to shoot and ensure that the image resolution, clarity, and color reproduction meet the requirements. During the shooting process, try to keep the camera stable to avoid image blur.
[0162] According to an embodiment of the present invention, capturing multiple calibration images and distortion measurement images at different angles and positions based on a three-dimensional calibration workpiece specifically includes:
[0163] Select a three-dimensional calibration workpiece with standard geometric features, the three-dimensional calibration workpiece includes a cube, a checkerboard, and the standard geometric features include regular corners or edges;
[0164] Set shooting lighting parameters, camera position and camera parameters;
[0165] Adjust the camera position to establish multiple shooting angles, sort the shooting angles, and adjust the shooting angles and camera positions in sequence according to the sorting results to shoot the 3D calibration workpiece;
[0166] Synchronously adjust camera parameters to obtain multiple calibration images and distortion measurement images;
[0167] Lighting parameters include light brightness and light uniformity, camera position includes horizontal position and vertical position, and camera parameters include camera focal length, aperture and exposure time.
[0168] It should be noted that distortion measurement uses a tensioned nylon line as a straight reference. Shooting angles and positions are planned, and a detailed shooting plan is developed to ensure coverage of the camera's entire field of view and various possible working angles. For example, shooting can be performed from different horizontal angles (0°, 45°, 90°, etc.), vertical angles (elevation, depression), and distances. For a horizontally placed 3D calibration workpiece, one can first capture a set of images directly above it, then gradually change the camera's horizontal angle and height to capture multiple sets of images.
[0169] Take a sufficient number of images: To obtain accurate camera parameters, you generally need to take at least 10 to 20 images at different angles and positions. During the shooting process, try to make the calibration artifact occupy different positions and proportions in the images to fully reflect the camera's imaging characteristics.
[0170] When shooting, pay attention to adjusting the camera's parameters such as focal length, aperture and exposure time to make the image clear and with moderate contrast, and avoid overexposure or underexposure, as well as blur or ghosting.
[0171] According to an embodiment of the present invention, the calibration image and the distortion measurement image are analyzed based on a calibration algorithm to obtain camera parameters and a distortion model, which specifically includes:
[0172] Acquire the calibration image and extract the corner coordinates of the 3D calibration workpiece based on the corner detection algorithm;
[0173] Establish image coordinates and analyze the relationship between corner point coordinates and image coordinates based on the calibration algorithm;
[0174] Solve the camera's intrinsic parameter matrix and extrinsic parameter matrix based on the least squares method to obtain the initial values of the camera parameters;
[0175] According to the initial parameters and corner point coordinates, a distortion model is established, and the distortion parameters are solved through a nonlinear optimization algorithm;
[0176] Taking the reprojection error as the objective function, a nonlinear optimization algorithm is used for global optimization to obtain the calibrated camera parameters and distortion model.
[0177] It should be noted that the distortion model includes radial distortion and tangential distortion, and its formula is:
[0178] x corrected =x(1+k1r 2 +k2r 4 +k3r 6 )+2p1xy+p2(r 2 +2x 2 )
[0179] y corrected =y(1+k1r 2 +k2r 4 +k3r 6 )+p1(r 2 +2y 2 )+2p2xy
[0180] Among them, x, y are image coordinates, k1, k2, k3 are radial distortion coefficients, p1, p2 are tangential distortion coefficients, r is the distance from the pixel to the center of the image.
[0181] By measuring the corrected image, the distortion correction error is calculated using the following formula:
[0182]
[0183] Where RMS represents the distortion correction error; x i 、y i is the corrected coordinate, is the ideal straight line coordinate, and N is the number of measurement points.
[0184] Specifically, common calibration algorithms include Zhang's calibration method and Tsai's two-step method. Zhang's calibration method is widely used due to its ease of operation and high accuracy, and is suitable for most common camera calibration scenarios. Tsai's two-step method is more accurate in modeling lens distortion and is often used for industrial camera calibration with high accuracy requirements. Corner detection algorithms include Harris corner detection or Shi-Tomasi corner detection; nonlinear optimization algorithms include Levenberg-Marquardt algorithm.
[0185] According to an embodiment of the present invention, acquiring an image of an object to be measured based on a calibrated camera and extracting image features of the object to be measured specifically include:
[0186] Acquire an image of the object to be measured, and perform filtering processing on the image of the object to be measured based on a filtering algorithm, wherein the filtering algorithm includes median filtering, Gaussian filtering, or bilateral filtering;
[0187] Performing contrast enhancement on the filtered image of the object to be measured to obtain an enhanced image;
[0188] Extract edge features of the object to be measured in the enhanced image based on edge detection algorithm;
[0189] Extract the corner features of the object to be measured based on the corner detection algorithm;
[0190] The texture features of the object to be measured are extracted based on texture analysis method.
[0191] It should be noted that image preprocessing includes the following:
[0192] Denoising: Denoising is performed on the acquired image to reduce the impact of noise on subsequent feature extraction. Common denoising methods include median filtering, Gaussian filtering, and bilateral filtering. For example, median filtering can effectively remove salt and pepper noise, while Gaussian filtering is suitable for removing Gaussian noise.
[0193] Enhance contrast: Through image enhancement techniques such as histogram equalization and adaptive histogram equalization, the contrast of the image is enhanced to make the details of the object clearer and facilitate feature extraction.
[0194] Normalization: Normalize the pixel values of an image to a certain range (such as [0, 1] or [0, 255]) to ensure consistency between different images and improve the accuracy of subsequent processing.
[0195] The specific method of feature extraction is as follows:
[0196] Edge-based feature extraction: Use edge detection algorithms (such as the Canny operator, Sobel operator, and Laplacian operator) to extract edge features of an object. Edges are important features of an object, reflecting its shape and outline. For example, the Canny operator is a commonly used edge detection algorithm that finds edge points in an image by calculating the gradient magnitude and direction.
[0197] Corner-based feature extraction: Corner detection algorithms (such as Harris corner detection and Shi-Tomasi corner detection) are used to extract the corner features of an object. Corners are distinct points in an image, typically located at the corners of an object or at the intersection of edges. These corners play an important role in tasks such as image matching and object tracking.
[0198] Region-based feature extraction: Use region segmentation algorithms (such as threshold segmentation, region growing, watershed algorithm, etc.) to segment the image into different regions, and then extract the features of each region, such as area, perimeter, center of mass, shape descriptor, etc. These regional features can be used for object classification and recognition.
[0199] Texture-based feature extraction: This method uses texture analysis methods (such as gray-level co-occurrence matrix and local binary pattern) to extract texture features of an object. Texture is an important characteristic of an object's surface, reflecting its material and surface structure. For example, the gray-level co-occurrence matrix can be used to extract texture features by calculating the co-occurrence relationship between different grayscale values in an image.
[0200] Deep learning-based feature extraction: This approach uses deep learning models such as convolutional neural networks (CNNs) to automatically extract object features. By feeding pre-processed images into a pre-trained CNN model, the model can learn high-level semantic features of the image, which have excellent performance in tasks such as object recognition and classification.
[0201] According to an embodiment of the present invention, performing distortion correction on the image features of the object to be measured based on the distortion model to obtain the corrected image features of the object to be measured specifically includes:
[0202] Analyze the image features of the object to be measured based on the distortion model to obtain the distortion parameters;
[0203] Establish an objective function based on the reprojection error and calculate the error value of the image feature of the object to be measured;
[0204] Determine whether the error value is greater than or equal to the set error threshold;
[0205] If it is greater than or equal to the set error threshold, a correction coefficient is generated, and the distortion parameter is corrected based on the correction coefficient to obtain the corrected image features of the object to be measured.
[0206] According to an embodiment of the present invention, three-dimensional reconstruction of the corrected image features of the object to be measured to obtain a measurement image specifically includes:
[0207] Obtain the corrected image features of the object to be measured, and calculate the similarity between feature points in different images based on feature point descriptors;
[0208] Based on the similarity analysis, the correspondence between feature points of different images is obtained to obtain matching feature point pairs;
[0209] Calculate the three-dimensional coordinates of all feature point pairs based on the triangulation method to generate a point cloud model;
[0210] Convert the point cloud model into a mesh model and optimize the mesh model to obtain a three-dimensional model. The optimization methods include removing redundant faces, repairing holes, and smoothing the surface.
[0211] The texture features are extracted based on the image of the object to be measured. According to the correspondence between the feature points, the texture features are mapped to the surface of the three-dimensional model for feature reconstruction to obtain the measurement image.
[0212] It should be noted that after calibration, feature points are extracted from the multiple images of the object to be measured using a feature point extraction algorithm (such as SIFT, SURF, ORB, etc.). These feature points should be unique and stable, and can be accurately identified under different viewing angles and lighting conditions.
[0213] The similarity between feature points in different images is calculated using feature point descriptors (such as SIFT descriptors). The corresponding relationship between feature points in different images is found through matching algorithms (such as nearest neighbor matching and bidirectional matching). To improve matching accuracy, the RANSAC (Random Sample Consensus) algorithm can be used to eliminate incorrect matching points.
[0214] Based on the matched feature point pairs, the relative position and pose relationship between the images is calculated using epipolar geometry principles (such as the essential matrix and the fundamental matrix). The essential matrix describes the rotation and translation relationship between the two camera coordinate systems, while the fundamental matrix establishes the correspondence between the two images in the pixel coordinate system.
[0215] Triangulation involves using known camera intrinsic parameters and calculated extrinsic parameters, combined with matched feature point pairs, to calculate the 3D coordinates of feature points in space. Specifically, this involves using the projection relationship of feature points from different perspectives to infer their position in the world coordinate system.
[0216] Common methods for converting point cloud models into mesh models include Delaunay triangulation and Poisson reconstruction. Delaunay triangulation forms a triangular mesh by connecting points in the point cloud, while Poisson reconstruction constructs a smoother surface model based on the normal vector information of the point cloud.
[0217] Based on the measurement requirements, select the appropriate virtual perspective to render the 3D reconstructed model and generate a measurement image. During the rendering process, annotations and dimension information can be added to facilitate object measurement and analysis.
[0218] A third aspect of the present invention provides a computer-readable storage medium, which includes an optical measurement method program based on camera calibration and distortion model. When the optical measurement method program based on camera calibration and distortion model is executed by a processor, the steps of the optical measurement method based on camera calibration and distortion model as described above are implemented.
[0219] The present invention discloses an optical measurement method, system and medium based on camera calibration and distortion model. The method comprises the following steps: shooting a plurality of calibration images and distortion measurement images at different angles and positions based on a three-dimensional calibration workpiece; analyzing the calibration images and distortion measurement images based on a calibration algorithm to obtain camera parameters and a distortion model; acquiring an image of an object to be measured based on the calibrated camera, and extracting image features of the object to be measured; performing distortion correction on the image features of the object to be measured based on the distortion model to obtain corrected image features of the object to be measured; performing three-dimensional reconstruction on the corrected image features of the object to be measured to obtain a measurement image, and analyzing parameter information of the object to be measured based on the measurement image to obtain a measurement result; and performing parameter calibration and distortion correction on the camera, thereby reducing distortion error and improving measurement accuracy.
[0220] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.
[0221] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.
[0222] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.
[0223] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.
[0224] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the existing technology, can be embodied in the form of a software product. The software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program code, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.
Claims
1. An optical measurement method based on camera calibration and distortion model, characterized in that: include: Based on the 3D calibration workpiece, multiple calibration images and distortion measurement images at different angles and positions are taken; Analyze the calibration image and distortion measurement image based on the calibration algorithm to obtain the camera parameters and distortion model; Acquire an image of the object to be measured based on a calibrated camera and extract image features of the object to be measured; Performing distortion correction on the image features of the object to be measured based on the distortion model to obtain the corrected image features of the object to be measured; The corrected image features of the object to be measured are reconstructed in three dimensions to obtain a measurement image, and the parameter information of the object to be measured is analyzed based on the measurement image to obtain a measurement result.
2. The optical measurement method based on camera calibration and distortion model according to claim 1, characterized in that: Based on the 3D calibration workpiece, multiple calibration images and distortion measurement images are taken at different angles and positions, including: Select a three-dimensional calibration workpiece with standard geometric features, the three-dimensional calibration workpiece includes a cube, a checkerboard, and the standard geometric features include regular corners or edges; Set shooting lighting parameters, camera position and camera parameters; Adjust the camera position to establish multiple shooting angles, sort the shooting angles, and adjust the shooting angles and camera positions in sequence according to the sorting results to shoot the 3D calibration workpiece; Synchronously adjust camera parameters to obtain multiple calibration images and distortion measurement images; Lighting parameters include light brightness and light uniformity, camera position includes horizontal position and vertical position, and camera parameters include camera focal length, aperture and exposure time.
3. The optical measurement method based on camera calibration and distortion model according to claim 2, characterized in that: The calibration image and the distortion measurement image are analyzed based on the calibration algorithm to obtain the camera parameters and distortion model, including: Acquire the calibration image and extract the corner coordinates of the 3D calibration workpiece based on the corner detection algorithm; Establish image coordinates and analyze the relationship between corner point coordinates and image coordinates based on the calibration algorithm; Solve the camera's intrinsic parameter matrix and extrinsic parameter matrix based on the least squares method to obtain the initial values of the camera parameters; According to the initial parameters and corner point coordinates, a distortion model is established, and the distortion parameters are solved through a nonlinear optimization algorithm; Taking the reprojection error as the objective function, a nonlinear optimization algorithm is used for global optimization to obtain the calibrated camera parameters and distortion model.
4. The optical measurement method based on camera calibration and distortion model according to claim 3, characterized in that: Acquire an image of the object to be measured based on a calibrated camera and extract image features of the object to be measured, specifically including: Acquire an image of the object to be measured, and perform filtering processing on the image of the object to be measured based on a filtering algorithm, wherein the filtering algorithm includes median filtering, Gaussian filtering, or bilateral filtering; Performing contrast enhancement on the filtered image of the object to be measured to obtain an enhanced image; Extract edge features of the object to be measured in the enhanced image based on edge detection algorithm; Extract the corner features of the object to be measured based on the corner detection algorithm; The texture features of the object to be measured are extracted based on texture analysis method.
5. The optical measurement method based on camera calibration and distortion model according to claim 4, characterized in that: Based on the distortion model, the image features of the object to be measured are subjected to distortion correction to obtain the corrected image features of the object to be measured, specifically including: Analyze the image features of the object to be measured based on the distortion model to obtain the distortion parameters; Establish an objective function based on the reprojection error and calculate the error value of the image feature of the object to be measured; Determine whether the error value is greater than or equal to the set error threshold; If it is greater than or equal to the set error threshold, a correction coefficient is generated, and the distortion parameter is corrected based on the correction coefficient to obtain the corrected image features of the object to be measured.
6. The optical measurement method based on camera calibration and distortion model according to claim 5, characterized in that: The corrected image features of the object to be measured are reconstructed in three dimensions to obtain a measurement image, specifically including: Obtain the corrected image features of the object to be measured, and calculate the similarity between feature points in different images based on feature point descriptors; Based on the similarity analysis, the correspondence between feature points of different images is obtained to obtain matching feature point pairs; Calculate the three-dimensional coordinates of all feature point pairs based on the triangulation method to generate a point cloud model; Convert the point cloud model into a mesh model and optimize the mesh model to obtain a three-dimensional model. The optimization methods include removing redundant faces, repairing holes, and smoothing the surface. The texture features are extracted based on the image of the object to be measured. According to the correspondence between the feature points, the texture features are mapped to the surface of the three-dimensional model for feature reconstruction to obtain the measurement image.
7. An optical measurement system based on camera calibration and distortion model, characterized in that: The system includes: a memory and a processor, wherein the memory includes a program for an optical measurement method based on camera calibration and a distortion model, and when the program for the optical measurement method based on camera calibration and a distortion model is executed by the processor, the following steps are implemented: Based on the 3D calibration workpiece, multiple calibration images and distortion measurement images at different angles and positions are taken; Analyze the calibration image and distortion measurement image based on the calibration algorithm to obtain the camera parameters and distortion model; Acquire an image of the object to be measured based on a calibrated camera and extract image features of the object to be measured; Performing distortion correction on the image features of the object to be measured based on the distortion model to obtain the corrected image features of the object to be measured; The corrected image features of the object to be measured are reconstructed in three dimensions to obtain a measurement image, and the parameter information of the object to be measured is analyzed based on the measurement image to obtain a measurement result.
8. The optical measurement system based on camera calibration and distortion model according to claim 7, characterized in that: Based on the 3D calibration workpiece, multiple calibration images and distortion measurement images are taken at different angles and positions, including: Select a three-dimensional calibration workpiece with standard geometric features, the three-dimensional calibration workpiece includes a cube, a checkerboard, and the standard geometric features include regular corners or edges; Set shooting lighting parameters, camera position and camera parameters; Adjust the camera position to establish multiple shooting angles, sort the shooting angles, and adjust the shooting angles and camera positions in sequence according to the sorting results to shoot the 3D calibration workpiece; Synchronously adjust camera parameters to obtain multiple calibration images and distortion measurement images; Lighting parameters include light brightness and light uniformity, camera position includes horizontal position and vertical position, and camera parameters include camera focal length, aperture and exposure time.
9. The optical measurement system based on camera calibration and distortion model according to claim 8, characterized in that: The calibration image and the distortion measurement image are analyzed based on the calibration algorithm to obtain the camera parameters and distortion model, including: Acquire the calibration image and extract the corner coordinates of the 3D calibration workpiece based on the corner detection algorithm; Establish image coordinates and analyze the relationship between corner point coordinates and image coordinates based on the calibration algorithm; Solve the camera's intrinsic parameter matrix and extrinsic parameter matrix based on the least squares method to obtain the initial values of the camera parameters; According to the initial parameters and corner point coordinates, a distortion model is established, and the distortion parameters are solved through a nonlinear optimization algorithm; Taking the reprojection error as the objective function, a nonlinear optimization algorithm is used for global optimization to obtain the calibrated camera parameters and distortion model.
10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes an optical measurement method program based on camera calibration and distortion model. When the optical measurement method program based on camera calibration and distortion model is executed by a processor, the steps of the optical measurement method based on camera calibration and distortion model as described in any one of claims 1 to 6 are implemented.
Citation Information
Patent Citations
Robust lens distortion correction method
CN108876749A
A single-view linear camera calibration method for small depth-of-field system
CN108986172A
Camera calibration method, calibration system and storage medium
CN119152046A
Three-dimensional geometric reconstruction camera calibration method and system
CN119672129A
Cited By
Optical correction camera lens module and imaging method thereof
CN121095589A
Visibility measurement method and device, terminal and medium
CN121459299A
Three-dimensional model picture matching method and system for disease identification
CN121616862A
