A differential geometry visual recognition method and system for spatial surface features
By constructing Frenet active orthogonal calibration field and basic surface forms, the problems of repeatedly manually changing the calibration plate position and complex surface parameter calculations during camera calibration are solved, thus simplifying camera calibration and surface feature parameter calculation.
Patent Information
- Application Number
- CN202010647015.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-07-07
- Publication Date
- 2024-11-22
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing technologies, especially for spatial curved surfaces, require multiple manual changes of the calibration plate position during camera calibration and distortion correction, which increases the complexity of industrial production lines. At the same time, the calculation process of characteristic parameters based on the surface parametric equation is complex and difficult to apply to free-form and composite surfaces.
Using differential geometry tools, a Frenet active orthogonal calibration field is constructed. By utilizing the relative positional relationship between the camera and the calibration plate, combined with the camera's nonlinear model, the camera calibration process is simplified. Furthermore, the local geometric properties of the surface are calculated using the basic form and formula of the surface, making it applicable to various types of surfaces.
It enables camera calibration without the need for multiple manual changes to the calibration plate position, simplifies the calculation process, and is applicable to general curved surfaces, free-form surfaces, and composite curved surfaces, thereby improving calibration efficiency and ease of calculation.
Smart Images

Figure CN111986264B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of visual recognition, and in particular to a differential geometry visual recognition method and system for spatial surface features. Background Art
[0002] With the development of optical technology and computer technology, binocular stereo vision has become simpler and more common. Camera calibration and distortion correction are the basis of visual recognition systems. Their accuracy and real-time performance determine the accuracy of the measurement results. The geometric characteristics of the target object from three-dimensional data are an important method for describing the target surface and classifying target matching, and are of great significance. For a spatial surface, its curvature direction and degree, and its concave and convex degree are determined by its characteristic parameters such as normal curvature, Gaussian curvature and mean curvature, and these characteristic parameters are uniquely determined by the first and second basic forms of the corresponding surface. Therefore, the use of differential geometry tools for camera calibration and surface geometric property analysis has potential application value in the field of pattern recognition.
[0003] The camera calibration is based on the camera nonlinear model to build the relationship between the camera pixel image coordinates and the real world coordinates, and solve the camera model parameters. Among them, the image coordinate system, the world coordinate system and the camera coordinate system are all Cartesian coordinate systems with fixed origins. The classic Zhang Zhengyouyou calibration method can accurately calibrate the camera, but the position of the calibration plate needs to be changed at least 3 times during the calibration process. This calibration method of manually changing the position of the calibration plate increases the complexity of visual inspection on the industrial assembly line; currently, the local geometric properties of the surface are analyzed through three-dimensional data, and most of them use the method of fitting the surface parameter equation and directly calculating the surface feature parameters to achieve the purpose of quantitatively analyzing the local characteristics of the surface within the allowable error range. However, for free-form surfaces and combined surfaces, the surface parameter equations are often difficult to obtain or the expression form is too complex, resulting in a complex calculation process of the feature parameters. Summary of the invention
[0004] The main purpose of the present invention is to overcome the defects and shortcomings of the existing camera calibration and distortion correction technology based on a fixed Cartesian coordinate system, and to provide a differential geometry visual recognition method for spatial surface features. The method regards the camera as a particle, and relies on the Frenet active orthogonal frame. The calibration result is obtained by solving the camera nonlinear model using three different position points of the camera, thereby avoiding the need to manually change the position of the calibration plate multiple times during the calibration process. On the other hand, the method for calculating feature parameters based on surface parameter equations has a complex calculation process. The method analyzes the local surface characteristics of the surface based on the first basic form and the second basic form of the surface, so that it has the characteristics of being applicable to various types of surfaces and simple calculation.
[0005] Another object of the present invention is to provide a differential geometry visual recognition system for spatial surface features.
[0006] The purpose of the present invention is achieved through the following technical solutions:
[0007] A differential geometry visual recognition method for spatial surface features comprises the following steps in sequence:
[0008] Step (1): By constructing a Frenet active orthogonal frame field and attaching it to the camera and calibration board, the camera intrinsic parameter matrix Q and extrinsic parameter matrix Q are obtained by using the relative position relationship between the frames and the camera nonlinear model.
[0009] Step (2): Use two identical cameras that have been calibrated to form a binocular stereo vision measurement system to collect the surface image of the real object in space;
[0010] Step (3): Construct the Frenet active orthogonal frame field on the space surface and calculate the normal curvature k based on the two basic forms of the surface and the basic formula of the surface n , Gaussian curvature K and mean curvature H are used to analyze the local surface geometric characteristics of the surface.
[0011] Furthermore, the step (1) specifically includes:
[0012] Step (1.1): Based on the initial relative position relationship between the camera and the calibration plate, a Frenet active orthogonal frame field is constructed, and an image of the calibration plate is collected at the initial position, and the image contains n corner points;
[0013] Step (1.2): Solve the initial value of the homography matrix M according to the camera nonlinear model AM=0, and consider the influence of noise to perform nonlinear optimization;
[0014] Step (1.3): Let the camera move along the trajectory of the rotating surface, select more than 3 fixed points (generally ≥3) with different distances on the rotating surface, and construct the relative position relationship between the active frames at each fixed point by the motion formula of the Frenet frame; substitute the relative position parameters of each fixed point into the camera nonlinear model AM=0, and obtain multiple homography matrices M j , where j ≥ 3;
[0015] Step (1.4): Through multiple homography matrices M j Solve the equation vb = 0, and replace the intermediate matrix B = (b ij ) is decomposed to obtain the camera internal parameter matrix Q and external parameter matrix
[0016] Step (1.5): Solve the equation D[k1k2]=d by integrating the 3n corner points of the three camera positions to obtain the camera distortion parameters; finally, optimize the intrinsic parameter matrix Q and the extrinsic parameter matrix by nonlinear least squares method. And distortion parameters k1, k2.
[0017] Furthermore, the parameter equation of the rotation surface in step (1.3) is:
[0018] r=r(u,v)=(f(v)cos u,f(v)sin u,g(v)),0≤u≤2π,a≤v≤b;
[0019] The selection of the camera shooting point should avoid being on the same curvature line, and other virtual shooting points of the camera except the initial position point can be obtained by solving the active frame transformation matrix;
[0020] The transformation matrix of the Frenet moving frame is:
[0021]
[0022] Furthermore, the binocular stereo vision measurement system in step (2) is assembled by using two high-resolution cameras with the same parameters in a parallel arrangement.
[0023] Furthermore, the step (3) specifically includes:
[0024] Step (3.1): Construct the Frenet active orthogonal frame field on the surface and obtain the first basic form and the second basic form of the surface;
[0025] Step (3.2): Combined with the basic formula of the surface, the three curvature characteristics of the surface are calculated: normal curvature k n , Gaussian curvature K and mean curvature H;
[0026] Step (3.3): Define the surface geometric characteristic parameter ξ = k through the three curvature characteristics of the surface n ×K×H, quantitatively analyze the local surface geometric properties of the surface.
[0027] Furthermore, the basic form of the surface in step (3.1) is expressed as:
[0028]
[0029] Among them, g αβ and b αβ are the first and second basic quantities of the surface, respectively, u 1 ,u 2 are the coordinates of the regular parametric surface,
[0030] Furthermore, the basic formula of the surface in step (3.2) is expressed as:
[0031]
[0032] in, Determined by the first fundamental quantity of the surface.
[0033] Furthermore, the curvature characteristic formula of step (3.2) is expressed as:
[0034]
[0035]
[0036]
[0037] Furthermore, the surface geometric characteristic parameter ξ in step (3.3) is defined as:
[0038] When H<0 and K<0: k n >0 indicates a convex peak, k n =0 means no, k n <0 indicates a concave peak;
[0039] When H<0 and K=0: k n >0 indicates a ridge, k n =0 means no, k n <0 indicates a concave ridge;
[0040] When H<0 and K>0: k n >0 indicates a convex saddle ridge, k n =0 means no, k n <0 indicates a concave saddle ridge;
[0041] When H=0 and K<0:k n >0 means no, k n =0 means no, k n <0 means none;
[0042] When H=0 and K=0: k n >0 indicates a plane, k n =0 means no, k n <0 indicates a flat surface;
[0043] When H=0 and K>0:k n >0 indicates the minimum point, k n =0 means no, k n <0 indicates the minimum point;
[0044] When H>0 and K<0: k n >0 indicates a concave bottom, k n =0 means no, k n <0 indicates a concave bottom;
[0045] When H>0 and K=0: k n >0 indicates a convex valley, k n =0 means no, k n<0 indicates a valley;
[0046] When H>0 and K>0: k n >0 means saddle valley, k n =0 means no, k n <0 indicates saddle valley.
[0047] Another object of the present invention is achieved by the following technical solutions:
[0048] A differential geometry visual recognition system for spatial surface features, including
[0049] Frenet camera calibration system: Based on the relative position relationship between the camera and the calibration plate, the Frenet active orthogonal frame field is constructed based on differential geometry theory, the relative position relationship between the camera and the object is established, and the camera nonlinear model is obtained. The internal and external parameters of the camera calibration result are obtained by solving the nonlinear model;
[0050] Surface image acquisition system: a binocular stereo vision measurement system consisting of two identical cameras, used to acquire the surface image of real objects in space;
[0051] Surface characteristic parameter calculation module: Construct the Frenet active orthogonal frame field on the surface, and calculate the normal curvature k of the surface by using the first basic form, the second basic form and the basic formula of the surface n , Guass curvature K and mean curvature H; and analyze the local surface geometric characteristics of the space surface through three curvature characteristics.
[0052] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0053] 1. The present invention uses differential geometry tools to model the camera calibration process, and achieves the purpose of camera calibration by solving the model. During the calibration process, there is no need to manually change the position of the calibration plate multiple times, which simplifies the camera calibration process.
[0054] 2. The present invention utilizes differential geometry tools to analyze the local geometric characteristics of general surfaces, free-form surfaces, and combined surfaces, thereby simplifying the complex calculations required for solving surface parameter equations compared to conventional methods. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] Figure 1 It is a structural schematic diagram of the differential geometry visual recognition system of the spatial surface features described in the present invention.
[0056] Figure 2 It is a schematic diagram of establishing a Frenet frame at a certain point and neighboring points on a surface curve according to the present invention.
[0057] Figure 3It is a camera calibration flow chart based on Frenet moving frame modeling in the present invention.
[0058] Figure 4 It is a flow chart of the present invention using differential geometry tools to analyze local geometric characteristics of a surface. DETAILED DESCRIPTION
[0059] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.
[0060] like Figure 1 As shown, a differential geometry visual recognition system for surface features includes:
[0061] Frenet camera calibration system: Based on the relative position relationship between the camera and the calibration plate, the Frenet active orthogonal frame field is constructed based on differential geometry theory, the relative position relationship between the camera and the object is established and the camera nonlinear model is obtained. The internal and external parameters of the camera calibration result are obtained by solving the nonlinear model.
[0062] Surface image acquisition system: A binocular stereo vision measurement system consisting of two identical cameras, used to acquire the surface images of real objects in space.
[0063] Surface characteristic parameter calculation module: Construct the Frenet active orthogonal frame field on the surface, and calculate the normal curvature k of the surface by using the first basic form, the second basic form and the basic formula of the surface n , Guass curvature K and mean curvature H. The local surface geometric characteristics of the space surface are analyzed through three curvature characteristics.
[0064] The movable orthogonal frame field is a Frenet movable orthogonal frame field.
[0065] A differential geometry visual recognition method based on the spatial surface features of the system comprises the steps of:
[0066] Step (1), such as Figure 2 As shown in the figure, by constructing the Frenet active orthogonal frame field and attaching it to the camera and calibration board, the camera intrinsic parameter matrix Q and extrinsic parameter matrix are solved by using the relative position relationship between the frames and the camera nonlinear model.
[0067] Step (2) uses two identical cameras that have been calibrated to form a binocular stereo vision measurement system to collect spatial real-object surface images.
[0068] Step (3): construct the Frenet active orthogonal frame field on the space surface and calculate the normal curvature k based on the two basic forms of the surface and the basic formula of the surface. n, Gaussian curvature K and mean curvature H are used to analyze the local surface geometric characteristics of the surface.
[0069] Specifically, if Figure 3 As shown, the step (1) specifically includes:
[0070] Step (1.1) constructs a Frenet active orthogonal frame field based on the initial relative position relationship between the camera and the calibration plate, and collects an image of the calibration plate at the initial position, the image containing n corner points.
[0071] Step (1.2), solve the initial value of the homography matrix M according to the camera nonlinear model AM=0, and consider the influence of noise to perform nonlinear optimization.
[0072] Step (1.3), let the camera move along the trajectory of the rotating surface, select three fixed points (generally ≥3) with different distances on the rotating surface, and construct the relative position relationship between the active frames at each fixed point by the motion formula of the Frenet frame. Substitute the relative position parameters of each fixed point into the camera nonlinear model AM=0 to obtain multiple homography matrices M j (j≥3).
[0073] Step (1.4), through multiple homography matrices M j Solve the equation vb = 0, and replace the intermediate matrix B = (b ij ) is decomposed to obtain the camera internal parameter matrix Q and external parameter matrix
[0074] Step (1.5), solve the equation D[k1k2]=d for the 3n corner points of the three camera positions to obtain the camera distortion parameters. Finally, the internal parameter matrix Q and the external parameter matrix are optimized by nonlinear least squares method. And distortion parameters k1, k2.
[0075] Specifically, the rotation surface parameter equation in step (1.3) can be written as:
[0076] r=r(u,v)=(f(v)cos u,f(v)sin u,g(v)),0≤u≤2π,a≤v≤b.
[0077] The selection of the camera shooting point avoids being on the same curvature line, and the other virtual shooting points of the camera except the initial position point are obtained by solving the active frame transformation matrix. The transformation matrix of the Frenet active frame is:
[0078]
[0079] Specifically, in step (2), two high-resolution cameras with the same parameters are used and arranged in parallel to form a stereoscopic vision measurement system.
[0080] Specifically, if Figure 4 As shown, the step (3) specifically includes:
[0081] Step (3.1), construct the Frenet active orthogonal frame field on the surface and obtain the first basic form and the second basic form of the surface.
[0082] Step (3.2), combined with the basic formula of the surface, calculate the three curvature characteristics of the surface (normal curvature k n , Gaussian curvature K and mean curvature H).
[0083] Step (3.3), define the surface geometric characteristic parameter ξ = k through the three curvature characteristics of the surface n ×K×H, can quantitatively analyze the local surface geometric properties of the surface.
[0084] Furthermore, the basic form of the surface in step (3.1) can be expressed as:
[0085]
[0086] where g αβ and b αβ are the first and second basic quantities of the surface, respectively, u 1 ,u 2 are the coordinates of the regular parametric surface,
[0087] Furthermore, the basic formula of the surface in step (3.2) can be expressed as:
[0088]
[0089] in, Determined by the first fundamental quantity of the surface.
[0090] Specifically, the curvature characteristic formula in step (3.2) can be expressed as:
[0091]
[0092]
[0093]
[0094] Specifically, the surface geometric characteristic parameter ξ in step (3.3) is defined as:
[0095]
[0096] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.
Claims
1. A differential geometry visual recognition method for spatial surface features, It is characterized in that The steps involved are as follows: Step (1): By constructing a Frenet active orthogonal frame field and attaching it to the camera and calibration board, the camera intrinsic parameter matrix Q and extrinsic parameter matrix Q are obtained by using the relative position relationship between the frames and the camera nonlinear model. The step (1) specifically comprises: Step (1.1): Based on the initial relative position relationship between the camera and the calibration plate, a Frenet active orthogonal frame field is constructed, and an image of the calibration plate is collected at the initial position, and the image contains n corner points; Step (1.2): Solve the initial value of the homography matrix M according to the camera nonlinear model AM=0, and consider the influence of noise to perform nonlinear optimization; Step (1.3): Let the camera move along the trajectory of the rotating surface, select more than three fixed points with unequal distances on the rotating surface, and construct the relative position relationship between the active frames on each fixed point by the motion formula of the Frenet frame; substitute the relative position parameters of each fixed point into the camera nonlinear model AM=0, and obtain multiple homography matrices M j , where j ≥ 3; Step (1.4): Through multiple homography matrices M j Solve the equation vb = 0, and replace the intermediate matrix B = (b ij ) is decomposed to obtain the camera internal parameter matrix Q and external parameter matrix Step (1.5): Solve the equation D[k 1 k 2 ]=d, and the camera distortion parameters are obtained; finally, the internal parameter matrix Q and the external parameter matrix are optimized by the nonlinear least squares method. and distortion parameter k 1 , k 2 ; Step (2): Use two identical cameras that have been calibrated to form a binocular stereo vision measurement system to collect the surface image of the real object in space; Step (3): Construct the Frenet active orthogonal frame field on the space surface and calculate the normal curvature k based on the two basic forms of the surface and the basic formula of the surface n , Gaussian curvature K and mean curvature H to analyze the local surface geometric characteristics of the surface; the step (3) specifically includes: Step (3.1): Construct the Frenet active orthogonal frame field on the surface and obtain the first basic form and the second basic form of the surface; Step (3.2): Combined with the basic formula of the surface, the three curvature characteristics of the surface are calculated: normal curvature k n , Gaussian curvature K and mean curvature H; Step (3.3): Define the surface geometric characteristic parameter ξ = k through the three curvature characteristics of the surface n ×K×H, quantitatively analyze the local surface geometric properties of the surface.
2. According to the differential geometry visual recognition method of spatial surface features described in claim 1, It is characterized in that Furthermore, the parameter equation of the rotation surface in step (1.3) is: r=r(u,v)=(f(v)cosu,f(v)sin u,g(v)),0≤u≤2π,a≤v≤b; The selection of the camera shooting point should avoid being on the same curvature line, and other virtual shooting points of the camera except the initial position point can be obtained by solving the active frame transformation matrix; The transformation matrix of the Frenet moving frame is:
3. According to the differential geometry visual recognition method of spatial surface features as described in claim 1, It is characterized in that The binocular stereo vision measurement system described in step (2) is assembled in parallel using two high-resolution cameras with the same parameters.
4. According to the differential geometry visual recognition method of spatial surface features as described in claim 1, It is characterized in that The basic form of the surface in step (3.1) is expressed as: Among them, g αβ and b αβ are the first and second basic quantities of the surface, respectively, u 1 ,u 2 are the coordinates of the regular parametric surface, 5. According to the differential geometry visual recognition method of spatial surface features as described in claim 1, It is characterized in that The basic formula of the surface in step (3.2) is expressed as: in, Determined by the first fundamental quantity of the surface.
6. The differential geometry visual recognition method of spatial surface features according to claim 1, It is characterized in that The curvature characteristic formula of step (3.2) is expressed as:
7. The differential geometry visual recognition method of spatial surface features according to claim 1, It is characterized in that The surface geometric characteristic parameter ξ in step (3.3) is defined as: When H<0 and K<0: k n >0 indicates a convex peak, k n =0 means no, k n <0 indicates a concave peak; When H<0 and K=0: k n >0 indicates a ridge, k n =0 means no, k n <0 indicates a concave ridge; When H<0 and K>0: k n >0 indicates a convex saddle ridge, k n =0 means no, k n <0 indicates a concave saddle ridge; When H=0 and K<0:k n >0 means no, k n =0 means no, k n <0 means none; When H=0 and K=0: k n >0 indicates a plane, k n =0 means no, k n <0 indicates a flat surface; When H=0 and K>0:k n >0 indicates the minimum point, k n =0 means no, k n <0 indicates the minimum point; When H>0 and K<0: k n >0 indicates a concave bottom, k n =0 means no, k n <0 indicates a concave bottom; When H>0 and K=0: k n >0 indicates a convex valley, k n =0 means no, k n <0 indicates a valley; When H>0 and K>0: k n >0 means saddle valley, k n =0 means no, k n <0 indicates saddle valley.
8. A differential geometry visual recognition system for spatial surface features. Features: include Frenet camera calibration system: Based on the relative position relationship between the camera and the calibration plate, the Frenet active orthogonal frame field is constructed based on differential geometry theory, the relative position relationship between the camera and the object is established and the camera nonlinear model is obtained. The internal and external parameters of the camera calibration result are obtained by solving the nonlinear model. The Frenet camera calibration system specifically performs: Based on the initial relative position relationship between the camera and the calibration plate, a Frenet active orthogonal frame field is constructed, and an image of the calibration plate is collected at the initial position, and the image contains n corner points; According to the camera nonlinear model AM=0, the initial value of the homography matrix M is solved, and the influence of noise is considered to perform nonlinear optimization; Let the camera move along the trajectory of the rotating surface, select more than three fixed points with unequal distances on the rotating surface, and construct the relative position relationship between the active frames on each fixed point by the motion formula of the Frenet frame; substitute the relative position parameters of each fixed point into the camera nonlinear model AM=0, and obtain multiple homography matrices M j , where j ≥ 3; Through multiple homography matrices M j Solve the equation vb = 0, and replace the intermediate matrix B = (b ij ) is decomposed to obtain the camera internal parameter matrix Q and external parameter matrix The 3n corner points of the three camera positions are combined to solve the equation D[k 1 k 2 ]=d, and the camera distortion parameters are obtained; finally, the internal parameter matrix Q and the external parameter matrix are optimized by the nonlinear least squares method. and distortion parameter k 1 , k 2 ; Surface image acquisition system: a binocular stereo vision measurement system consisting of two identical cameras, used to acquire the surface image of real objects in space; Surface characteristic parameter calculation module: Construct the Frenet active orthogonal frame field on the surface, and calculate the normal curvature k of the surface by using the first basic form, the second basic form and the basic formula of the surface n , Guass curvature K and mean curvature H; and analyze the local surface geometric characteristics of the space surface through three curvature characteristics; the surface feature parameter calculation module specifically performs: Construct the Frenet active orthogonal frame field on the surface and obtain the first basic form and the second basic form of the surface; Combined with the basic formula of the surface, the three curvature characteristics of the surface are calculated: normal curvature k n , Gaussian curvature K and mean curvature H; The surface geometric characteristic parameter ξ=k is defined by the three curvature characteristics of the surface n ×K×H, quantitatively analyze the local surface geometric properties of the surface.
