A method for real-time detection of length and width size distribution of crystal population in crystallization kettle using binocular telecentric camera

Abstract

The invention discloses an in-situ calibration modeling, stereo polar line correction and three-dimensional reconstruction method for a binocular telecentric stereo vision system, and belongs to the field of industrial process control and detection technology. In order to perform in-situ environmental calibration on the binocular telecentric stereo vision system, a calibration rod that can be inserted into the in-situ environment (reactor / glass tube) is designed, a binocular telecentric stereo vision imaging model is established, and a simple calibration method and a calibration plate design scheme suitable for a rotating calibration rod in an in-situ limited space are proposed. In order to improve the binocular matching efficiency, a simplified telecentric stereo polar line correction method is also provided to ensure the accurate matching of on-site snapshot image pairs. Based on the matched image pairs, a three-dimensional reconstruction method based on ray intersection of an analytical solution is provided to measure the three-dimensional posture of particles in a crystallizer. Finally, the three-dimensional length and width of the crystal are quantitatively evaluated by the statistical data of the Euclidean distance of the length and width feature point pairs. The invention has strong operability and can achieve the effect of automatically measuring the three-dimensional size of the crystal.

Description

Technical Field

[0001] The invention belongs to the technical field of industrial process control and detection, and relates to a method for real-time detection of length and width size distribution of a crystal population in a crystallization kettle using a binocular telecentric camera. Background Art

[0002] The process and control technology of crystallization process plays a very important role and significance to the high-quality development of my country's advanced manufacturing field. In the past few decades, imaging systems have been increasingly used to measure the particle size in various crystallization processes in laboratory and industrial-scale crystallizers. There are few literatures and patents at home and abroad on the effects of refraction caused by propagation in cross-medium optical systems, such as a glass crystallizer filled with solution and a glass jacket for heat exchange.

[0003] In order to ensure the accuracy of particle size field measurement based on microscopic image analysis, it is crucial to establish an accurate camera calibration model. However, compared with the existing pinhole imaging system calibration method

[17] , the calibration of telecentric systems has not been fully studied in the past few years. In order to solve the problem of deformation measurement using telecentric lenses

[18] , a simple calibration method based on linear fitting was proposed. This method actually does not consider the lens distortion and can only obtain the internal parameter magnification factor. Li proposed a planar pattern calibration method using a single telecentric lens for imaging in the international optical journal "Optics and Lasers in Engineering", but did not solve the ambiguity problem of external parameters. Furthermore, Chen et al. developed a calibration method that provides Z-axis translation displacement for the calibration pattern through a micro-positioning stage. This method can solve the sign ambiguity problem in the calibration process based on planar patterns. Liu et al. proposed a two-step calibration method based on a three-dimensional calibration plate in the international journal "Optics Express", but due to the very limited accuracy of the three-dimensional calibration mode, a planar calibration mode is still required. In order to adapt to the limited in-situ space of the calibration target motion, Cheng et al. proposed a simplified telecentric camera calibration method that does not require auxiliary equipment in the top journal "IEEE Transactions on Instrumentation and Measurement". The Scheimpflug telecentric model similar to the specific pinhole camera model is used to solve the related sign problem. For telecentric systems that do not have the Scheimpflug property, this strategy may be ineffective, resulting in large reprojection errors. In addition, the above calibration method requires auxiliary equipment for on-site calibration. How to simply and quickly achieve accurate calibration and reconstruction in the limited in-situ calibration motion space remains a challenging problem. Summary of the invention

[0004] The technical problem to be solved by the present invention is to perform in-situ calibration for the refraction effect of the reactor medium on the optical system during the detection process and to measure the three-dimensional length and width of the crystal during the crystallization process. To solve the above problem, a design scheme for an in-situ calibration rod and a binocular telecentric camera imaging calibration modeling method are systematically proposed, as well as a telecentric stereo vision three-dimensional reconstruction technology method based on an analytical solution, so as to realize in-situ online detection of the three-dimensional length and width of the crystal in the solution.

[0005] The technical solution adopted by the present invention is as follows:

[0006] A method for real-time detection of the length and width size distribution of a crystal population in a crystallization reactor by a binocular telecentric camera is implemented based on a non-contact high-resolution backlight calibration rod, which ensures that two telecentric cameras synchronously collect images of the crystal solution in the reactor, thereby measuring the three-dimensional size of the crystal. The backlight calibration rod is composed of a light-emitting diode (LED), a ceramic calibration plate printed with a chessboard pattern, a reflective prism, an optical fiber, and a stainless steel sheath. The light emitted by the LED at the top of the calibration rod is guided to the reflective prism at the bottom of the calibration rod through the optical fiber, and the reflective prism vertically reflects the light to the back of the ceramic calibration plate engraved with a chessboard pattern. The reflected light provides illumination for the shooting of the two telecentric cameras. The device can provide sufficient imaging light intensity within a short exposure time (such as 36-100μs), thereby reducing motion blur caused by solution turbulence and other environmental disturbances in the crystallizer. On-site calibration can be performed by inserting the calibration rod into the glass crystallizer and rotating it clockwise or counterclockwise along its pivot. The backlight calibration rod can be assembled into different sizes to be suitable for on-site calibration of crystallizers of different volumes.

[0007] The present invention places a telecentric camera at two different angles outside the reactor to synchronously collect images in real time. The images are called binocular images, including left and right views. According to the position and posture of the binocular camera, a two-step in-situ stereo imaging calibration model is established, including calculating the closed-loop solution of the camera parameters to obtain the initial values ​​of the model parameters and subsequently using the optimization algorithm to find the optimal camera model parameters. According to the collected binocular images, an analysis method for crystal image matching is proposed, including image preprocessing, interest point detection, feature matching and mismatch removal. After using the crystal contour features to detect its length and width key corner points, the calibrated stereo imaging model is used to perform three-dimensional reconstruction of the length and width corner points of each matched crystal, thereby quantitatively evaluating its three-dimensional size. Specifically, the following steps are included:

[0008] Step 1: Stereo Imaging Calibration

[0009] The telecentric camera imaging model, that is, the projection of a point P (xw, yw, zw) in the world coordinate system to the pixel coordinate system P (u, v) is expressed as

[0010]

[0011] Where m is the effective magnification of the telecentric lens, and R and T are the rotation and translation matrices between the two coordinate systems, respectively.

[0012] (1) Monocular camera parameter calibration:

[0013] The world coordinates of the planar calibration plate can be associated with the captured image coordinates through the homography matrix H, as follows:

[0014]

[0015] For ease of analysis, the Euler angle is introduced to describe the rotation matrix R(α, β, θ), where α, β, θ are the rotation angles around the X, Y, and Z axes of the coordinate system, respectively. The elements of the corresponding rotation matrix are described as:

[0016]

[0017] By making corresponding elements of the 3×3 matrix in (2) equal, the following equation can be easily established:

[0018]

[0019] Where, α, β and θ are the rotation angles in the range (-π / 2, π / 2). In addition, the effective magnification factor is defined as a positive value. The solution of the above equation is:

[0020]

[0021] in c=(h 11 h 22 -h 12 h 21 ) 2 . Solve to get the translation matrix T(t x ,t y )for:

[0022]

[0023] Therefore, the magnification factor m and the Euler angle θ can be uniquely determined. However, the Euler angles α and β have two possible signs, as shown in (5). Therefore, there may be two ambiguous poses with respect to the world coordinate system in the single camera coordinate system, while there may be four ambiguous poses in the binocular camera system. Restoring the true Euler angles α and β is the key to the calibration method of this system.

[0024] Due to the orthographic projection characteristics of the telecentric camera, the camera pose recovered from the 2D model is ambiguous. Existing methods rely on 3D calibration models with limited accuracy

[23] or make full use of external displacement

[20] , but cannot be used for on-site calibration of stereo vision systems due to the inability to establish a physical connection between the imaging target and the stereo vision system. To overcome the above problems, a practical rotation template calibration method is proposed.

[0025] During the calibration process, the calibration pattern is rotated in a specific angular order (clockwise or counterclockwise) around the Y w The camera is rotated around the Y axis so that the camera can obtain orderly calibration images. Therefore, it is observed that the placement angle of the calibration pattern changes in a sequential manner. At the same time, the rotation angle of the camera coordinate system relative to the world coordinate system, the Euler angle β, also increases (or decreases) monotonically. For ease of analysis, it is assumed that the calibration pattern is around the Y axis. w The shaft rotates clockwise.

[0026]

[0027] Considering that the actual rotation angle β is a monotonically increasing sequence, the absolute value of β may present a local minimum point or maintain a monotonic trend. Calculating the absolute value of each function can check whether there is a local minimum point. If there are multiple local minimum points or local maximum points, the calibration condition is not met. However, if the specified conditions are met, it can be inferred that the extreme point will appear near the minimum value of the discrete sequence, resulting in

[0028]

[0029] Through several discretizations, the extreme point can be determined near the minimum, such as the left or right side of the minimum, as shown in Figure 5(a). Since the rotation angle β is monotonically increasing, in addition to the sign of the minimum point, the following equation is used to restore the sign of:

[0030]

[0031] The rotation angle α is recovered by the following formula:

[0032]

[0033] (2) Calibration of stereo vision system parameters:

[0034] The essence of binocular vision calibration is to obtain the relative position relationship between two telecentric cameras. The coordinate position relationship of the left and right cameras is represented by the rotation matrix R C and the translation vector T C Description, that is:

[0035]

[0036] Among them, R L and T L ,R R and T R They are the rotation matrix and transfer vector related to the world coordinate system and the left and right camera coordinate systems respectively.

[0037] For each pair of calibrated images, the external matrices of the left and right cameras change, and the relative rotation matrix R between the left and right cameras is C Remain unchanged. R obtained from the kth pair of calibration images C The Euler angles are expressed as

[0038]

[0039] where α C , β C ,θ C is the rotation angle around the X, Y and Z axes of the left camera coordinate system. The rotation Euler angles and represent the external parameters R in the left and right camera coordinate systems L and R R The superscript k indicates that the data comes from the kth set of calibration images.

[0040] The Euler angle α obtained for N pairs of different calibration images C , β C ,θ C Optimize

[0041]

[0042] Where N k is the number of variables in the calibration image set.

[0043] Convert the calculated rotation matrix Euler angle representation to matrix representation R C , and according to (11) we can obtain T C , complete the stereo calibration work.

[0044] Step 2: Simplified Stereo Correction

[0045] Epipolar correction plays an important role in accurate stereo matching, which simplifies the two-dimensional search for matching points into a one-dimensional search. In telecentric stereo vision, existing methods use a 3×4 matrix for epipolar correction. In fact, from the perspective of geometry and physics, the correction parameters can be further simplified to a 3×3 homography matrix, which is more convenient for computer-aided image analysis. The specific method is as follows: point P is converted from the world coordinate system to the left and right camera pixel coordinate systems p L and p R The projections are expressed as

[0046]

[0047] The projection of point P from the world coordinate system to the corrected left and right camera pixel coordinate systems is determined as:

[0048]

[0049] According to formulas (14) and (15), the transformation from original image coordinates to corrected coordinates is established:

[0050]

[0051] Step 3: Reconstruction and Measurement

[0052] (1) 3D reconstruction:

[0053] For telecentric stereo vision system, an improved ray intersection 3D reconstruction method based on analytical solution is proposed.

[0054] First, construct the matching point pairs on the left and right physical image planes respectively. The position equation of each ray.

[0055]

[0056] Where (x l ,y l ,z l ) represents the coordinates of any point on the left image plane, represents the coordinates of the matching points on the left physical image plane, Represents the direction unit vector of the left camera optical axis, τ L is the light coefficient of the left camera, (x r ,y r ,z r ) represents the coordinates of any point on the right image plane, represents the coordinates of the matching points on the right physical image plane, Represents the direction unit vector of the right camera optical axis, τ R is the ray coefficient of the right camera. If the calculated point on the physical image plane and the position equation of the above ray are accurate, there is only one intersection point between the left and right rays, which is the reconstructed point on the surface of the object. However, due to the presence of noise, the left and right rays may not intersect. In order to find the best intersection point, the two points closest to the left ray and the right ray are calculated respectively. By calculating the analytical solution, the coefficient of the left ray when taking the closest point position is obtained:

[0057]

[0058] in

[0059]

[0060] Correspondingly, the coordinates of the reconstructed points are obtained as follows:

[0061]

[0062] (2) Crystal image matching and measurement:

[0063] First, the image is corrected using the correction method in step 2, and the crystal image is quickly matched through a one-dimensional search. Then, the length feature point pair is obtained through the distance from the contour to the center point of the contour, and the length feature point pair is connected to obtain the crystal main axis, and the projection of the contour on the main axis is calculated to obtain multiple width feature point pairs. Then, the feature point pairs are matched by polar line constraints, and the three-dimensional spatial coordinates of the feature points are calculated. Finally, the distance between the length feature points is used as the length of the crystal, and the distance between multiple width feature points is voted, and the width feature point distance with the most occurrences is selected as the crystal width to complete the measurement of the length and width of the crystal.

[0064] The present invention can realize in-situ calibration and reconstruction of the telecentric stereo vision system, and realize three-dimensional length and width measurement by reconstructing the characteristic points of the crystal, thereby effectively avoiding the measurement error caused by the refraction of light propagating in the cross-medium optical system, and more accurately analyzing the growth state of the crystal in the crystallization process. The method is highly operable, has low requirements for experience and technology, can achieve the effect of rapid system in-situ calibration and automatic measurement of the three-dimensional size of the crystal, and is convenient for practical industrial application. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] FIG1 is a schematic diagram of an application scenario and a calibration rod of the present invention, FIG1(a) is an overall schematic diagram, FIG1(b) is a structural schematic diagram of the calibration rod, and FIG1(c) is a cross-sectional view of the calibration rod;

[0066] Figure 2 It is a schematic diagram of the telecentric imaging model of the present invention;

[0067] Figure 3 It is a schematic diagram of a binocular telecentric stereo imaging model of the present invention;

[0068] Figure 4 is a schematic diagram of the epipolar correction model of the present invention;

[0069] FIG5 is a schematic diagram of polar line correction, FIG5(a) is an original collected crystal image, and FIG5(b) is a corrected crystal image;

[0070] FIG6 is a schematic diagram of obtaining width feature points of the crystal main axis projection of the present invention, wherein FIG6(a) is a schematic diagram of the projection of the crystal contour on the main axis and the selection of width feature points, and FIG6(b) is a schematic diagram of the extracted crystal length feature matching points and width feature matching points;

[0071] Figure 7 It is a schematic diagram of the light intersection reconstruction model of the present invention;

[0072] FIG8 is a schematic diagram of the calculation of the length and width of a crystal according to the present invention, FIG8(a) is a three-dimensionally reconstructed characteristic point of the length and width of the crystal, and FIG8(b) is a measurement result of the width of the crystal. DETAILED DESCRIPTION

[0073] In order to better understand the technical solution of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0074] First, a telecentric camera is placed at two different angles outside the reactor to collect images synchronously in real time. The calibration rod is inserted into the reactor and rotated clockwise to calibrate the binocular system in situ. Secondly, a stereoscopic vision epipolar correction method expressed by a 3*3 matrix is ​​established to simplify the two-dimensional search of matching points into a one-dimensional search. Finally, each pair of crystal images is matched by the calibrated stereo imaging model and the stereoscopic vision epipolar correction method, and the three-dimensional length and width feature points are reconstructed by the ray intersection method based on the analytical solution, and the three-dimensional size is quantitatively evaluated by multiple sets of data.

[0075] The method is implemented based on a non-contact industrial binocular telecentric imaging system, as shown in Figure 1(a). The example uses a 1L crystal glass reactor with a built-in 4-blade stirring paddle connected to a computer, and 500mL of crystal water solution is injected into the reactor. A non-contact binocular imaging device is configured outside the reactor, which includes two high-speed and high-resolution industrial-grade monochrome cameras connected to a computer, and two LED point light sources to provide backlighting, which are located on both sides of the reactor. During the calibration process, the calibration rod is inserted into the imaging target area through the device hole at the top of the reactor, and the system calibration is completed by a rotating calibration rod. The design of the calibration rod is shown in Figure 1(b,c). The light emitted by the LED at the top of the calibration rod is guided to the reflective prism at the bottom of the calibration rod through an optical fiber. The reflective prism reflects the light vertically to the back of the ceramic calibration plate engraved with a chessboard pattern. The reflected light provides illumination for the shooting of the two telecentric cameras, completing the shooting of the calibration image group. During the measurement process, two telecentric cameras synchronously collect binocular images (left view and right view) of the crystallization process. After polar constraint correction and matching, the crystals in any posture in the solution are reconstructed to achieve three-dimensional size measurement of the crystal length and width based on binocular vision.

[0076] The method specifically comprises the following steps:

[0077] The first step is calibration:

[0078] In order to eliminate the influence of refraction caused by the propagation of light in a cross-medium optical system to a certain extent, the present invention proposes a new rotation calibration method. The calibration of the stereo telecentric system can be simply completed by manually rotating the calibration plate around its axis for in-situ imaging. For this purpose, a backlight calibration rod composed of a light-emitting diode (LED), a ceramic calibration plate covered with a checkerboard pattern, a reflective prism and an optical fiber is designed to be used to probe into a target space such as a reactor for in-situ calibration.

[0079] (1) Monocular camera parameter calibration:

[0080] According to Figure 2 In the telecentric camera imaging model shown, the world coordinates of the plane calibration plate can be associated with the captured image coordinates through the homography matrix H, as shown below:

[0081]

[0082] For ease of analysis, the Euler angle is introduced to describe the rotation matrix R(α, β, θ), where α, β, θ are the rotation angles around the X, Y, and Z axes of the coordinate system, respectively. The elements of the corresponding rotation matrix are described as:

[0083]

[0084] By making corresponding elements of the 3×3 matrix in (2) equal, the following equation can be easily established:

[0085]

[0086] Where, α, β and θ are the rotation angles in the range (-π / 2, π / 2). In addition, the effective magnification factor is defined as a positive value. The solution of the above equation is:

[0087]

[0088] in c=(h 11 h 22 -h 12 h 21 ) 2 . Solve to get the translation matrix T(t x ,t y )for:

[0089]

[0090] Therefore, the magnification factor m and the Euler angle θ can be uniquely determined. However, the Euler angles α and β have two possible signs. Therefore, there may be two ambiguous poses with respect to the world coordinate system in the single camera coordinate system, while there may be four ambiguous poses in the binocular camera system. Restoring the true Euler angles α and β is the key to the calibration method of this system.

[0091] Due to the orthographic projection characteristics of the telecentric camera, the camera posture recovered from the two-dimensional mode is ambiguous. Existing methods rely on a three-dimensional calibration mode with limited accuracy, or make full use of external displacement. However, due to the existence of the reactor wall, it is impossible to establish a physical connection between the imaging target and the stereo vision system, and it cannot be used for on-site calibration of the stereo vision system. In order to overcome the above problems, the present invention proposes a practical rotation template calibration method.

[0092] During the calibration process, the calibration pattern is rotated in a specific angular order (clockwise or counterclockwise) around the Y w The camera is rotated around the Y axis so that the camera can obtain orderly calibration images. Therefore, it is observed that the placement angle of the calibration pattern changes in a sequential manner. At the same time, the rotation angle of the camera coordinate system relative to the world coordinate system, the Euler angle β, also increases (or decreases) monotonically. For ease of analysis, it is assumed that the calibration pattern is around the Y axis. w The shaft rotates clockwise.

[0093] For each image captured during the calibration process, the camera’s extrinsic matrix changes. By taking a positive sign for the β solution in (25), we can obtain the rotation matrix R for the kth calibration image: k (α k , β k ,θ k ), translation vector T k (t x k ,t y k ) and the magnification factor m k The calculation formula of the amplification factor m is

[0094]

[0095] Considering that the actual rotation angle β is a monotonically increasing sequence, the absolute value of β may present a local minimum point or maintain a monotonic trend. Calculating the absolute value of each function can check whether there is a local minimum point. If there are multiple local minimum points or local maximum points, the calibration condition is not met. However, if the specified conditions are met, it can be inferred that the extreme point will appear near the minimum value of the discrete sequence, resulting in

[0096]

[0097] Through several discretizations, the extreme point can be determined near the minimum, such as the left or right side of the minimum, as shown in Figure 5(a). Since the rotation angle β is monotonically increasing, in addition to the sign of the minimum point, the following equation is used to restore the sign of:

[0098]

[0099] The rotation angle α is recovered by the following formula:

[0100]

[0101] (2) Calibration of stereo vision system parameters:

[0102] The essence of binocular vision calibration is to obtain the relative position relationship between two telecentric cameras, such as Figure 3 As shown. The coordinate position relationship of the left and right cameras is represented by the rotation matrix R C and the translation vector T C Description, that is:

[0103]

[0104] Among them, R L and T L ,R R and T R They are the rotation matrix and transfer vector related to the world coordinate system and the left and right camera coordinate systems respectively.

[0105] For each pair of calibrated images, the external matrices of the left and right cameras change, and the relative rotation matrix R between the left and right cameras is C Remain unchanged. R obtained from the kth pair of calibration images C The Euler angles are expressed as

[0106]

[0107] where α C , β C ,θ C is the rotation angle around the X, Y and Z axes of the left camera coordinate system. The rotation Euler angles and represent the external parameters R in the left and right camera coordinate systems L and R R The superscript k indicates that the data comes from the kth set of calibration images.

[0108] The Euler angle α obtained for N pairs of different calibration images C , β C ,θ C Optimize

[0109]

[0110] Where Nk is the number of variables in the calibration image set.

[0111] Convert the calculated rotation matrix Euler angle representation to matrix representation R C , and according to (11) we can obtain T C , complete the stereo calibration work.

[0112] Step 2: Calibration:

[0113] In order to reduce the computational complexity of the matching process, the present invention proposes a simplified telecentric stereoscopic epipolar correction method, which completes the reprojection correction process by connecting the images before and after correction through a 3*3 homography matrix. Figure 4 As shown, the coordinate systems of the rectified left and right camera pixels are aligned along the common y-axis.

[0114] The projection of point P from the world coordinate system to the left and right camera pixel coordinate systems are expressed as

[0115]

[0116] Considering that the magnification factors of the left and right cameras should be equal after calibration to ensure the alignment of the y axes of the two cameras in the pixel coordinate system, the magnification factors of the two cameras are determined by finding the largest magnification factor from them:

[0117]

[0118] Therefore, the projections of point P from the world coordinate system to the corrected left and right camera pixel coordinate systems are determined as

[0119]

[0120] like Figure 4 Shown Respectively represent the X-, Y-, and Z-axis direction vectors in the left and right camera coordinate systems before and after correction, namely RL, RR, It should be noted that after correction, the y-axis of the left camera coordinate system is parallel to the y-axis of the right camera, and the z-axis representing the direction of the light remains unchanged. Therefore, the common axis is perpendicular to the optical axes of the left and right cameras. The direction vector of the ideal correction coordinate system of the left and right cameras can be expressed as:

[0121]

[0122] In order to align the y-axis of the corrected left and right camera coordinate systems, the new translation vector is calculated as

[0123]

[0124] Combining equations (34) and (36), the transformation from original image coordinates to corrected coordinates is established:

[0125]

[0126] Step 3: Matching and reconstruction measurements:

[0127] (1) Matching

[0128] The collected original image is shown in Figure 5(a). First, the image is corrected by the correction method in step 2, and the crystal image is quickly matched by one-dimensional search. The corrected image and the matched crystal are shown in Figure 5(b). Then, the length feature point pair is obtained by the distance from the contour to the contour center point, and the length feature point pair is connected to obtain the crystal main axis, as shown by the red dotted line in Figure 6(b).

[0129] It should be noted that during the crystallization process, small crystals will adhere to the edge of large crystals, as shown in Figure 6(b). In order to avoid mis-extraction when measuring the crystal width, the present invention proposes a method for selecting effective width profile points for measurement as follows:

[0130] The measurement of each particle width refers to calculating the Euclidean distance between two contour points, where the line between the contour points is perpendicular to the main axis of the measured particle. To ensure the measurement accuracy, all contour point pairs perpendicular to the main axis line are selected as candidate marking points for measuring the particle width. For this purpose, the particle contour is divided into the upper half and the lower half according to the main axis. The distance from the contour point on each half contour to the main axis is calculated as

[0131]

[0132] Considering that the crystal width distance should reflect the dominant distance of the above-mentioned contour point pairs, rather than their average distance, it is recommended to determine whether the deviation of this distance is less than half of the mean absolute deviation defined respectively:

[0133]

[0134] Where N p is the number of points on each half contour. If yes, the corresponding point is identified as a candidate point, and vice versa, as shown by the cyan and magenta points in Figure 6(a).

[0135] In order to match these candidate points, the contour coordinates in the left and right images are projected onto their main axes respectively by the following formula. The normalized distance from the contour to the main axis is shown in Figure 6(a).

[0136]

[0137] where x u and x d They are pu and p d The x-coordinate value of .

[0138] The candidate points with the same projection coordinates on the upper and lower contours of the left image are regarded as a pair of particle width marking points, as shown by the red dots in Figure 6(a). Similarly, the corresponding matching points in the right image also have the same projection coordinates. In other words, the four candidate points with the same projection coordinates on the upper and lower contours of the left and right images constitute a set of particle width marking points.

[0139] (2) Reconstruction

[0140] For the telecentric stereo vision system, the present invention proposes a new telecentric stereo vision 3D reconstruction method based on ray intersection based on analytical solution. Figure 7 As shown, the details are as follows:

[0141] First, construct the matching point pairs on the left and right physical image planes respectively. The position equation of each ray.

[0142]

[0143] Where (x l ,y l ,z l ) represents the coordinates of any point on the left image plane, represents the coordinates of the matching points on the left physical image plane, Represents the direction unit vector of the left camera optical axis, τ L is the light coefficient of the left camera, (x r ,y r ,z r ) represents the coordinates of any point on the right image plane, represents the coordinates of the matching points on the right physical image plane, Represents the direction unit vector of the right camera optical axis, τ R is the ray coefficient of the right camera. If the calculated points on the physical image plane and the position equations of the above rays are accurate, there is only one intersection point between the left and right rays, which is the reconstructed point on the object surface. However, due to the presence of noise, the left and right rays may not intersect. In order to find the best intersection point, calculate the two points closest to the left and right rays respectively:

[0144] The distance between any two points on the left and right rays is expressed as:

[0145]

[0146] Substituting (45) and (46) into (47), we can obtain L , τR The quadratic equation is:

[0147]

[0148] By order Find f(τ L , τ R ), determine the minimum value of τ L and τ R The values ​​are:

[0149]

[0150] in

[0151]

[0152] Correspondingly, the coordinates of the reconstructed points are obtained as follows:

[0153]

[0154] (3) Measurement

[0155] First, the three-dimensional spatial coordinates of the feature points are calculated for the matching feature point pairs using the above three-dimensional reconstruction method. The reconstructed feature points are shown in Figure 8(a). Then, the Euclidean distance between the length feature points is used as the length of the crystal. The Euclidean distances between multiple width feature points are voted, and the width feature point distance with the most occurrences is selected as the crystal width, that is, the median width, as shown in Figure 8(b), completing the measurement of the length and width of the crystal.

[0156] The length marker point P is calculated using the above three-dimensional reconstruction method u , P d and width marker point set P f , P l Therefore, the length and width of each crystal can be calculated as:

[0157] L=|P u -P d | (52)

[0158] W med =median(P f -P l ) (53)

[0159] W mean =mean(P f -P l ) (54)

[0160] The above-described embodiments merely express the implementation methods of the present invention, but they cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A method for real-time detection of the length and width size distribution of a crystal population in a crystallization kettle using a binocular telecentric camera, characterized in that: The method is based on a non-contact, high-resolution backlight calibration rod, which ensures that two telecentric cameras synchronously collect images of the crystal solution in the reactor, thereby measuring the three-dimensional size of the crystal; the backlight calibration rod is composed of a light-emitting diode (LED), a ceramic calibration plate printed with a chessboard pattern, a reflecting prism, an optical fiber and a stainless steel sheath; the light emitted by the LED at the top of the backlight calibration rod is guided to the reflecting prism at the bottom of the backlight calibration rod through the optical fiber, and the reflecting prism vertically reflects the light to the back of the ceramic calibration plate engraved with a chessboard pattern; the reflected light provides lighting for the shooting of the two telecentric cameras; the backlight calibration rod is inserted into the glass crystallizer and rotated clockwise or counterclockwise along its pivot to perform on-site calibration; the backlight calibration rod can be assembled into different sizes to be suitable for on-site calibration of crystallizers of different volumes; A telecentric camera is placed at two different angles outside the reactor to collect images in real time and synchronously. The images are called binocular images, including left view and right view. According to the position and posture of the binocular camera, a two-step in-situ stereo imaging calibration model is established, including calculating the closed-loop solution of camera parameters to obtain the initial values ​​of model parameters and then using optimization algorithm to find the optimal camera model parameters. According to the collected binocular images, an analysis method for crystal image matching is proposed, including image preprocessing, interest point detection, feature matching and mismatch removal. After using the crystal contour features to detect its length and width key corner points, the calibrated stereo imaging model is used to perform three-dimensional reconstruction of the length and width corner points of each matched crystal, thereby quantitatively evaluating its three-dimensional size.

2. The method for real-time detection of the length and width size distribution of the crystal population in the crystallization kettle by using a binocular telecentric camera according to claim 1, characterized in that: Step 1: Stereo Imaging Calibration The telecentric camera imaging model, that is, the projection of a point P (xw, yw, zw) in the world coordinate system to the pixel coordinate system P (u, v) is expressed as Where m is the effective magnification factor of the telecentric lens, R and T are the rotation and translation matrices between the two coordinate systems respectively; (1) Monocular camera parameter calibration: The world coordinates of the planar calibration plate can be related to the captured image coordinates through the homography matrix H, as follows: For the convenience of analysis, the Euler angle is introduced to describe the rotation matrix R(α, β, θ), where α, β, θ are the rotation angles around the X-axis, Y-axis, and Z-axis of the coordinate system respectively; the elements of the corresponding rotation matrix are described as: By making corresponding elements of the 3×3 matrix in (2) equal, the following equation can be easily established: Where, α, β and θ are the rotation angles in the range of (-π / 2, π / 2); in addition, the effective magnification factor is defined as a positive value; the solution of the above equation is: in c=(h 11 h 22 -h 12 h 21 ) 2 ; Solve to get the translation matrix T(t x, t y )for: Therefore, the effective magnification factor m and the Euler angle θ can be uniquely determined; however, the Euler angles α and β have two possible signs, as shown in (5); therefore, there may be two ambiguous postures related to the world coordinate system in the single camera coordinate system, while there may be four ambiguous postures in the binocular camera system; restoring the true Euler angles α and β is the key to the calibration method of this system; During the calibration process, the calibration patterns are placed in a specific angular order around the Y w The camera is rotated around the Y axis so that the camera can obtain orderly calibration images. Therefore, it is observed that the placement angle of the calibration pattern changes to increase or decrease in sequence. At the same time, the rotation angle of the camera coordinate system relative to the world coordinate system, the Euler angle β, also increases or decreases monotonically. For ease of analysis, it is assumed that the calibration pattern is around the Y axis. w The axis rotates clockwise; Considering that the actual rotation angle β is a monotonically increasing sequence, the absolute value of β may present a local minimum point or maintain a monotonic trend; calculating the absolute value of each function can check whether there is a local minimum point; if there are multiple local minimum points or local maximum points, the calibration condition is not met; however, if the specified conditions are met, it can be inferred that the extreme point will appear near the minimum value of the discrete sequence, resulting in Through several discretizations, the extreme point can be determined near the minimum, such as the left or right side of the minimum; since the rotation angle β is monotonically increasing, in addition to the sign of the minimum point, the following equation is used to restore the sign: The rotation angle α is recovered by the following formula: (2) Calibration of stereo vision system parameters: The essence of binocular vision calibration is to obtain the relative position relationship between the two cameras; the coordinate position relationship of the left and right cameras is represented by the rotation matrix R C and the translation vector T C Description, that is: Among them, R L and T L ,R R and T R They are the rotation matrix and transfer vector related to the world coordinate system and the left and right camera coordinate systems respectively; For each pair of calibrated images, the external matrices of the left and right cameras change, and the relative rotation matrix R between the left and right cameras is C Remain unchanged; R obtained by the kth pair of calibration images C The Euler angles are expressed as where α C , β C ,θ C is the rotation angle around the X-axis, Y-axis and Z-axis of the left camera coordinate system; the rotation Euler angle and represent the external parameters R in the left and right camera coordinate systems L and R R ; The superscript k indicates that the data comes from the kth group of calibration images; The Euler angle α obtained for N pairs of different calibration images C , β C ,θ C Optimize Where N k is the number of variables in the calibration image set; Convert the calculated rotation matrix Euler angle representation to matrix representation R C , and according to (11) we can obtain T C , complete the stereo calibration work; Step 2: Simplified Stereo Correction Epipolar correction plays an important role in accurate stereo matching. It simplifies the two-dimensional search for matching points into a one-dimensional search. In telecentric stereo vision, existing methods use a 3×4 matrix for epipolar correction. In fact, from a geometric and physical point of view, the correction parameters can be further simplified to a 3×3 homography matrix, which is more convenient for computer-aided image analysis. The specific method is as follows: Point P from the world coordinate system to the left and right camera pixel coordinate system p L and p R The projections are expressed as The projection of point P from the world coordinate system to the corrected left and right camera pixel coordinate systems is determined as: According to formulas (14) and (15), the transformation from original image coordinates to corrected coordinates is established: Step 3: Reconstruction and Measurement (1) 3D reconstruction: For telecentric stereo vision system, an improved ray intersection 3D reconstruction method based on analytical solution is proposed. First, construct the matching point pairs on the left and right physical image planes respectively. The position equation of each ray; Where (x l ,y l ,z l ) represents the coordinates of any point on the left image plane, represents the coordinates of the matching points on the left physical image plane, Represents the direction unit vector of the left camera optical axis, τ L is the light coefficient of the left camera, (x r ,y r ,z r ) represents the coordinates of any point on the right image plane, represents the coordinates of the matching points on the right physical image plane, Represents the direction unit vector of the right camera optical axis, τ R is the ray coefficient of the right camera; if the calculated point on the physical image plane and the position equation of the above ray are accurate, there is only one intersection point between the left and right rays, that is, the reconstructed point on the surface of the object; however, due to the presence of noise, the left and right rays may not intersect; in order to find the best intersection point, the two points closest to the left ray and the right ray are calculated respectively. By calculating the analytical solution, the coefficient of the left ray when the closest point position is taken is: in Correspondingly, the coordinates of the reconstructed points are obtained as follows: (2) Crystal image matching and measurement: First, the image is corrected using the correction method in step 2, and the crystal image is quickly matched through one-dimensional search; then, the length feature point pair is obtained through the distance from the contour to the center point of the contour, and the length feature point pair is connected to obtain the crystal main axis, and the projection of the contour on the main axis is calculated to obtain multiple width feature point pairs; then, the feature point pairs are matched through polar line constraints, and the three-dimensional spatial coordinates of the feature points are calculated; finally, the distance between the length feature points is taken as the length of the crystal, and the distance between multiple width feature points is voted, and the width feature point distance with the largest number of occurrences is selected as the crystal width to complete the measurement of the length and width of the crystal.

Images