Pixel equivalent measuring method based on mobile terminal pupil distance intelligent measurement

Abstract

The invention discloses a pixel equivalent measuring method based on mobile terminal pupil distance intelligent measurement, which comprises the following steps: calibrating a camera, and correcting distortion through internal and external parameters obtained through calibration; image preprocessing, including image pre-identification and image enhancement, and highlighting the edge and angular point features of the image; coarse positioning of the calibration object: acquiring rough positions of corner points at four corners of a square frame of the calibration object; carrying out angular point coordinate coarse positioning, and carrying out mask processing on the image by taking an obtained angular point and a two-dimensional neighborhood thereof as a range; refining the coordinates of the angular points, correcting and compensating the angular points, and obtaining the coordinates of the four corrected angular points; performing perspective transformation; determining a pixel equivalent and a transformation matrix; carrying out verification measurement; and verifying and optimizing the determined pixel equivalent. According to the invention, an image processing technology is utilized to solve the calibration problem during the pupil distance measurement of the mobile end, and a small-range straight angle point precision compensation method is utilized to automatically compensate error defects, so that the pupil distance measurement precision is improved, and a new idea is provided for the existing method.

Description

Technical Field

[0001] The present invention relates to the field of machine vision and medical measurement, and in particular to a method for determining pixel equivalent based on intelligent measurement of pupil distance on a mobile terminal. Background Art

[0002] The pupil distance refers to the distance between the centers of the pupils of the two eyes, and is an important physiological parameter. In the field of eyeglass configuration, whether the size is correct or not directly affects future eye health. Currently, the field of medical measurement has begun to realize the measurement of pupil distance through artificial intelligence technology.

[0003] But there are the following problems:

[0004] At present, there are relatively few solutions for pupil distance measurement based on mobile devices. The reason is that the performance of mobile cameras varies greatly, and there is a lack of control over the position and intensity of the light source, making the calibration task difficult.

[0005] In the process of measuring pupil distance, there are errors caused by shooting focus problems or edge distortion caused by image processing, which leads to distortion in the positioning of the straight edge corner points of the calibration object when measuring pixel equivalent, resulting in inaccurate measurement results. Summary of the invention

[0006] In view of this, the purpose of the present invention is to provide a pixel equivalent determination method based on mobile terminal intelligent pupil distance measurement, which adopts image processing technology to reduce the influence of light source, and utilizes a small range straight edge corner point precision compensation method to automatically compensate for error defects and improve the pupil distance measurement accuracy.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] A pixel equivalent determination method based on intelligent measurement of pupil distance on a mobile terminal, the method comprising the following parts:

[0009] Preprocessing part, this part mainly includes two steps: camera calibration and image preprocessing;

[0010] The coordinate refinement part mainly includes three steps: coarse positioning of the calibration object, coarse positioning of the corner point coordinates, and refinement of the corner point coordinates. The area to be processed is windowed through mask operation, which can weaken the error caused by the position and intensity of the light source and reduce the amount of calculation;

[0011] The image correction part is done through perspective transformation, which can determine the value of pixel equivalent and the transformation matrix of the same image;

[0012] Verification measurement part, which includes two steps: determining the position of the verification circle and measuring the verification circle. The algorithm performance is evaluated by the measurement accuracy of the verification circle, and the parameters are adjusted in time to refine the measurement;

[0013] Furthermore, the above scheme is implemented by the following steps:

[0014] Step 1: Calibrate the camera; obtain the internal and external parameters through camera calibration to perform distortion correction and reduce errors;

[0015] Step 2: Image preprocessing: Image processing includes two directions: one is image pre-recognition, which provides prior knowledge for subsequent tasks and improves recognition quality. The other is image enhancement, which highlights the edge and corner features of the image and reduces the impact of noise.

[0016] Step 3: Rough positioning of the calibration object, identifying the solid circle and square frame in the calibration object, and obtaining the rough positions of the four corners of the square frame of the calibration object;

[0017] Step 4: Coarse positioning of corner point coordinates; perform pixel-level corner point detection on the calibration object, compare the corner points obtained in this step with the corner points obtained in the above step 3, and perform mask processing on the image to window the area to be processed with the two types of corner points and their neighborhoods as the range;

[0018] Step 5: Refine the corner point coordinates; refine the area to be processed obtained in the above step 4 by using a small range straight edge corner point accuracy compensation method to obtain four calibrated corner point measurement coordinates;

[0019] Step 6: perspective transformation: the calibration object is corrected according to the coordinates of the four corner points obtained in step 5 above. In this step, the pixel equivalent and the transformation matrix are determined;

[0020] Step 7: Confirmatory measurement: verify the pixel equivalent determined in step 6 above, determine the accuracy of the current algorithm by calculating the measured radius of the black solid circle and the actual size loss, and repeat steps 5 to 7 to continue optimization.

[0021] Furthermore, in step 1, the camera calibration method is Zhang's calibration method;

[0022] Furthermore, the calibration object should be worn on the forehead and kept as parallel to the face as possible to reduce the error;

[0023] Further, in step 2, the image and recognition method is dilb face detection;

[0024] Furthermore, in step three, the solid circle and square frame in the identification calibration object are specifically identified as follows:

[0025] 1) Perform image processing operations such as Gaussian filtering and morphological transformation on the image and obtain edge features;

[0026] 2) Locate the circular coordinates; calculate the area of ​​each contour region and the minimum circumscribed circle area of ​​each contour respectively, and obtain the maximum relative area circular coordinates after comparison;

[0027] 3) Locate the coordinates of the rectangle; based on the geometric relationship of the circle in the rectangle, calculate the minimum circumscribed rectangle area of ​​each contour and compare it with the area of ​​each contour in 2) above to obtain the vertex coordinates of the rectangle with the maximum relative area.

[0028] Furthermore, in step 4, ORB pixel-level corner point detection is performed on the image, and the point with the smallest Euclidean distance to the corner point of the rectangle is obtained among all the corner points;

[0029] Furthermore, in step 4, a two-dimensional neighborhood range of the rough position of the corner point is taken for masking to reduce the light intensity error;

[0030] Furthermore, in step 5, the small-range straight edge corner point accuracy compensation method specifically includes the following steps:

[0031] 1) Perform interpolation on the masked image to enlarge the image to sub-pixel level to improve image quality and clarity

[0032] 2) Multi-scale edge feature detection, fully mining the multi-scale edge information of the image, including image enhancement, texture detection and original image, binary image to highlight edge features, image registration, feature-level fusion and Canny edge detection;

[0033] 3) Edge point clustering is used for primary data processing. The edge points obtained in 2) are obtained through linear interpolation operation. Outlier point cleaning and k-means clustering are performed on the edge points to obtain two types of straight edge point clusters and their centroid points.

[0034] 4) Abnormal data cleaning at the arc is used to reprocess the data based on prior knowledge, and a rectangular area is delineated with the two centroid points obtained in 3) as vertices, and the points contained in the rectangular area are removed;

[0035] 5) Orientation analysis and fitting: used to analyze the direction and position characteristics of data points, fit the analysis results with straight lines, obtain the covariance matrix of the remaining two types of edge points and their eigenvectors and eigenvalues, determine the lattice direction according to the direction of the main direction vector, and obtain the slope and intercept of the fitting line. The intersection of the straight lines is the corner point of the square frame after compensation.

[0036] Furthermore, in step 5, the multi-scale edge detection used is edge-preserving filtering based on wavelet idea, including wavelet decomposition, guided filtering and reconstruction, which is used to smooth the noise and texture in the image while retaining the edge information in the image;

[0037] The wavelet decomposition is used to perform localized processing on the frequency signal and time (space), and decompose it into a plurality of low-frequency and high-frequency sub-bands;

[0038] The guided filtering is used to smooth the low-frequency sub-band and retain the high-frequency sub-band to provide edge information;

[0039] The reconstruction is used to synthesize the filtered sub-bands of each scale into a final enhanced image;

[0040] Furthermore, in step seven, the confirmatory measurement specifically includes the following steps:

[0041] 1) Perform sub-pixel edge detection on the image within the field of view;

[0042] 2) Locate the circular coordinates, calculate the area of ​​each contour region and the minimum circumscribed circle area of ​​each contour, and obtain the maximum relative area circular coordinates after comparison;

[0043] 3) Measure the pixel radius of the obtained circle and convert the pixel size to physical size using pixel equivalent;

[0044] 4) Determine the accuracy of the current algorithm by calculating the loss between the measured radius of the obtained circle and the actual size.

[0045] The pixel equivalent determination method for the mobile terminal pupil distance intelligent measurement provided by the present invention has the following beneficial effects compared with the prior art:

[0046] 1) The present invention improves the problem that there are relatively few solutions for mobile terminal pupil distance measurement, and is subject to the uncertainty of light source, calibration object posture and shooting distance. At the same time, it solves the size relationship between the actual physical size and the pixel size in the visual measurement task, that is, determining the pixel equivalent, and also uses the visual angle of the calibration object as a reference to obtain the transformation matrix for correcting the image to the front view;

[0047] 2) The present invention minimizes the influence of light sources through windowing operation in the visible area, effectively reduces absolute errors, and improves accuracy. It also performs precision compensation for straight edge corner points in a small visual field, with high accuracy and small amount of calculation, and is fully capable of high-precision measurement of calibration objects when measuring pupil distance, thereby obtaining accurate pixel equivalents and transformation matrices;

[0048] 3) The present invention also applies multi-scale edge feature detection to explore the influence of different frequency band images and their textures on edge extraction, fully tap the edge information, and through directional analysis and fitting, it can comprehensively compensate for the errors caused by image edge distortion and improve the accuracy of straight edge corner point detection, providing new ideas for existing methods;

[0049] Other features and advantages of the present invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 Schematic diagram of the calibration sequence of the pupil distance intelligent measurement of the present invention

[0051] Figure 2 Schematic diagram of the calibration object of the present invention

[0052] Figure 3 This is a flow chart of the accuracy compensation method of the present invention.

[0053] Figure 4 Schematic diagram of compensation method of the present invention at a certain angle of the calibration object DETAILED DESCRIPTION

[0054] The preferred implementation of the present invention is described in detail below in conjunction with the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.

[0055] The present invention provides a method for determining pixel equivalent based on intelligent measurement of pupil distance at a mobile terminal, such as Figure 1 As shown, the method includes four parts: image preprocessing, coordinate refinement, image correction and verification measurement, with a total of seven steps.

[0056] Step 1: Calibrate the camera. The camera itself has distortion that affects the image quality. The internal and external parameters of the camera are obtained through Zhang's calibration method, and the camera is calibrated according to the parameters.

[0057] Step 2: Image enhancement. The captured image needs to include Figure 2 The calibration object shown in the figure is used to perform image processing operations such as Gaussian filtering and morphological transformation on the image and obtain edge features; the image is pre-recognized, and the 68 key points of the face are determined using dilb face recognition. The calibration object area is delineated according to the key points, so that it can be brought into the subsequent recognition tasks as prior knowledge to improve recognition efficiency.

[0058] Step 3: Roughly locate the calibration object. Based on the prior knowledge from step 2, narrow the operation range and obtain the area of ​​each contour in the field of view. and its minimum circumscribed circle area According to the relation

[0059]

[0060] Get the rough coordinates of the center of the circle ,in Indicates finding the coordinates of the center of a circle that meets the conditions. Similarly, according to the area of ​​the minimum enclosing rectangle The area of ​​each contour in the field of view get ,, the center of the integrated circle is within the square frame and the pixel radius of the circle Less than or equal to the distance from the center of the circle to the four sides of the square border The minimum value of gets the rough square frame coordinates.

[0061] The formula can be written as:

[0062]

[0063] Note that these two identifications are only for roughly locating the area of ​​the marker, and the specific coordinates obtained are not used as the basis for measurement.

[0064] After the position of the square border is identified, the area to be processed is determined by the coordinates of the four corner points of the minimum circumscribed rectangle and the offset distance within its two-dimensional neighborhood. , further refining the location of the landmarks and gradually narrowing the field of view can reduce the amount of calculation and isolate most of the environmental noise.

[0065] Step 4: Coarsely locate the corner point coordinates and perform ORB corner point detection on the image. Due to the influence of the shape and noise of the marker, it is necessary to use the least squares method to match the four corner points of the minimum circumscribed rectangle of the square frame obtained in step 3 above with the corner points to be screened. The mask range is determined by offsetting the two types of points closest to each corner by a pixel. That is, four areas to be processed can be determined at the four corners of the calibration object. , for further processing.

[0066] Step 5: Refine the corner point coordinates, such as Figure 3 As shown, the specific process of this step includes:

[0067] S1: Reduce the image field of view and improve image quality, including:

[0068] 1) Obtain the coordinates of the neighborhood area to be compensated for the calibration object, and window the area to be processed through mask operation;

[0069] 2) Perform bicubic interpolation on the area obtained in 1) to enlarge the image to its original size. times, so that each pixel represents the original , can improve image quality and clarity, and facilitate further detection of the image;

[0070] In the embodiment of the present invention, the neighborhood of the compensation area to be taken may be 10-15 pixels, distributed at the four corners of the calibration object;

[0071] S2: Multi-scale edge feature fusion is performed on the image to fully explore the possible edge information of the image and provide more data support for subsequent operations, including:

[0072] 1) The image is subjected to edge-preserving filtering and Gabor filtering respectively. The former uses wavelet to decompose, filter and reconstruct the image at multiple scales, while smoothing the noise and texture in the image and retaining the edge information in the image; the latter uses Gabor filter to analyze the texture of the image, which can capture the details missed by edge-preserving filtering and supplement the edge information of the image;

[0073] 2) Binarize the edge features and texture features obtained by the above operation 1) to highlight the edge features, then align and fuse them, and finally use Canny edge detection to obtain the sub-pixel contour map.

[0074] S3: Edge point clustering. After S2, a binary contour map that combines edge features and texture features can be obtained. Linear interpolation is used to obtain contour pixels on the contour map. K-means clustering (K=2) is then applied to these pixels to separate the edge points on both sides and output the cluster center points of the edge points on each side at the same time.

[0075] S4: Abnormal data cleaning at the arc; the rectangular area defined by the two cluster center points contains the local points of the arc part, so the cluster center points are used to clean the abnormal data of the edge points on both sides and enhance the directional trend of the straight edge.

[0076] S5: Orientation analysis and fitting; used to analyze the directional trend and location characteristics of edge points, and fit the edge with a straight line, such as Figure 4 As shown, the specific operations include:

[0077] 1) Calculate the covariance matrix of each edge to obtain the distribution of edge points;

[0078] 2) Calculate the eigenvectors and eigenvalues ​​of the covariance matrix. The eigenvector represents the main direction of the data set, and its corresponding eigenvalue represents the difference or variance in this direction.

[0079] 3) The direction of the eigenvector with a larger eigenvalue is taken as the main direction vector, and the direction of the eigenvector with a smaller eigenvalue is taken as the secondary direction vector. The slope and intercept of the fitting line are determined according to the direction of the main direction vector. The intersection of the straight lines is the corner point of the square frame after compensation.

[0080] Step 6: perspective transformation. The corner point coordinates obtained in step 5 are used as the four points extracted from the whole image for perspective transformation. The image is corrected into the form of positive projection, which is mapped to a larger operating space. The general transformation formula is:

[0081]

[0082] make is the transformation matrix, whose value is

[0083]

[0084] The value obtained in the calibration object detection can be used to calibrate the entire image. After this step is completed, the actual distance Distance to pixel Get pixel equivalent , which is calculated as:

[0085]

[0086] Step 7: Confirmatory measurement. In order to evaluate the accuracy of this algorithm, the pixel equivalent Used for verification measurement of the black solid circle in the center of the calibration object. First, perform sub-pixel edge detection on the calibration object after perspective transformation, and repeat step 3 to obtain the pixel radius of the circle. , if the actual radius of the circle is expressed as , then the algorithm accuracy can be calculated by the following formula:

[0087]

[0088] Absolute error for:

[0089]

[0090] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent substitutions, changes, material substitutions, improvements, etc. according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be within the protection scope of the present invention.

Claims

1. A method for determining pixel equivalent based on intelligent measurement of pupil distance on a mobile terminal, characterized in that Contains the following sections: Preprocessing part, this part mainly includes two steps: camera calibration and image preprocessing; The coordinate refinement part mainly includes three steps: coarse positioning of the calibration object, coarse positioning of the corner point coordinates, and refinement of the corner point coordinates. The area to be processed is windowed through mask operation, which can weaken the error caused by the position and intensity of the light source and reduce the amount of calculation; The image correction part is done through perspective transformation, which can determine the value of pixel equivalent and the transformation matrix of the same image; The verification measurement part includes two steps: determining the position of the verification circle and measuring the verification circle. The algorithm performance is evaluated by the measurement accuracy of the verification circle, and the parameters are adjusted in time to refine the measurement.

2. The pixel equivalent determination method based on mobile terminal pupil distance intelligent measurement according to claim 1 is characterized in that: The method is implemented by the following steps: Step 1: Calibrate the camera; Calibrate the camera; obtain the internal and external parameters through camera calibration to perform distortion correction and reduce errors; Step 2: Image preprocessing: Image processing includes two directions: one is image pre-recognition, which provides prior knowledge for subsequent tasks and improves recognition quality. The other is image enhancement, which highlights the edge and corner features of the image and reduces the impact of noise. Step 3: Rough positioning of the calibration object, identifying the solid circle and square frame in the calibration object, and obtaining the rough positions of the four corners of the square frame of the calibration object; Step 4: Coarse positioning of corner point coordinates; perform pixel-level corner point detection on the calibration object, compare the corner points obtained in this step with the corner points obtained in the above step 3, and perform mask processing on the image to window the area to be processed with the two types of corner points and their neighborhoods as the range; Step 5: Refine the corner point coordinates; refine the area to be processed obtained in the above step 4 by using a small range straight edge corner point accuracy compensation method to obtain four calibrated corner point measurement coordinates; Step 6: perspective transformation: the calibration object is corrected according to the coordinates of the four corner points obtained in step 5 above. In this step, the pixel equivalent and the transformation matrix are determined; Step 7: Confirmatory measurement: verify the pixel equivalent determined in step 6 above, determine the accuracy of the current algorithm by calculating the measured radius of the black solid circle and the actual size loss, and repeat steps 5 to 7 to continue optimization.

3. The pixel equivalent determination method based on mobile terminal pupil distance intelligent measurement according to claim 2 is characterized in that: In step 1, the camera calibration method is Zhang's calibration method.

4. The pixel equivalent determination method based on mobile terminal pupil distance intelligent measurement according to claim 2 is characterized in that: In step 3, the solid circle and square frame in the identification calibration object are specifically identified as follows: 1) Perform image processing operations such as Gaussian filtering and morphological transformation on the image and obtain edge features; 2) Locate the circular coordinates; calculate the area of ​​each contour region and the minimum circumscribed circle area of ​​each contour respectively, and obtain the maximum relative area circular coordinates after comparison; 3) Locate the coordinates of the rectangle; based on the geometric relationship of the circle in the rectangle, calculate the minimum circumscribed rectangle area of ​​each contour and compare it with the area of ​​each contour in 2) above to obtain the vertex coordinates of the rectangle with the maximum relative area.

5. The pixel equivalent determination method based on mobile terminal pupil distance intelligent measurement according to claim 2 is characterized in that: In step 4, ORB pixel-level corner point detection is performed on the image, and the point with the smallest Euclidean distance to the corner point of the rectangle is obtained among all the corner points.

6. The pixel equivalent determination method based on mobile terminal pupil distance intelligent measurement according to claim 5 is characterized in that: In step 4, the rough position of the corner point is masked in its two-dimensional neighborhood to reduce the light intensity error.

7. The pixel equivalent determination method based on mobile terminal pupil distance intelligent measurement according to claim 2 is characterized in that: In step 5, the small-range straight edge corner point accuracy compensation method specifically includes the following steps: 1) Perform interpolation on the masked image to enlarge the image to sub-pixel level to improve image quality and clarity 2) Multi-scale edge feature detection, fully mining the multi-scale edge information of the image, including image enhancement, texture detection and original image, binary image to highlight edge features, image registration, feature-level fusion and Canny edge detection; 3) Edge point clustering is used for primary data processing. The edge points obtained in 2) are obtained through linear interpolation operation. Outlier point cleaning and k-means clustering are performed on the edge points to obtain two types of straight edge point clusters and their centroid points. 4) Cleaning of abnormal data at the arc, which is used to reprocess the data based on prior knowledge, demarcate a rectangular area with the two centroid points obtained in 3) as vertices, and remove the points contained in the rectangular area; 5) Orientation analysis and fitting: used to analyze the direction and position characteristics of data points, fit the analysis results with straight lines, obtain the covariance matrix of the remaining two types of edge points and their eigenvectors and eigenvalues, determine the lattice direction according to the direction of the main direction vector, and obtain the slope and intercept of the fitting line. The intersection of the straight lines is the corner point of the square frame after compensation.

8. The pixel equivalent determination method based on mobile terminal pupil distance intelligent measurement according to claim 7 is characterized in that: In step five, the multi-scale edge detection used is edge-preserving filtering based on wavelet concept, including wavelet decomposition, guided filtering and reconstruction, which is used to smooth the noise and texture in the image while retaining the edge information in the image.

9. The pixel equivalent determination method based on mobile terminal pupil distance intelligent measurement according to claim 2 is characterized in that: In step 7, the confirmatory measurement specifically includes the following steps: 1) Perform sub-pixel edge detection on the image within the field of view; 2) Locate the circular coordinates, calculate the area of ​​each contour region and the minimum circumscribed circle area of ​​each contour, and obtain the maximum relative area circular coordinates after comparison; 3) Measure the pixel radius of the obtained circle and convert its pixel size to physical size using pixel equivalent; 4) Determine the accuracy of the current algorithm by calculating the loss between the measured radius of the obtained circle and the actual size.

Images