A method for identifying coordinates of reading and writing scenes
Through the coordinate identification method of reading and writing scenes, camera calibration and OpenCV function calculations are used to identify reading and writing postures and distances, which solves the posture and distance detection problems in myopia problems in adolescents, and accurately judges and corrects reading and writing postures, reducing the Myopia risk.
Patent Information
- Application Number
- CN201910547731.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-06-24
- Publication Date
- 2024-11-26
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The myopia problem among adolescents is serious, especially due to irregular reading and writing postures and prolonged use of electronic products. It is difficult for the prior art to effectively identify and correct posture and distance problems in reading and writing scenarios.
Using a reading and writing scene coordinate identification method, the parameters inside the camera are obtained through camera calibration, and the three-dimensional coordinates and image pixel coordinates of known objects are used to solve the external parameters and attitude Euro-view angle of the camera, and the three-dimensional coordinates of the object are calculated based on the OpenCV function to judge the Whether the user's reading and writing posture and distance meet the standards.
Achieve accurate detection of reading and writing postures and distances, reminding incorrect postures, reducing the risk of myopia, using a monocular camera to obtain accurate three-dimensional postures, improving detection accuracy, low cost and wide practicality, and avoiding slight head bowing and shaking calculation error.
Smart Images

Figure CN110288718B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image recognition, and in particular to a method for identifying coordinates of a reading and writing scene. Background Art
[0002] The number of myopic people in my country has exceeded 600 million, accounting for almost 50% of the total population of China. The onset of myopia shows a trend of early age, rapid progression and severe degree. According to the survey report, students have a prominent problem of poor eyesight. The detection rates of poor eyesight among fourth-grade and eighth-grade students are 36.5% and 65.3% respectively, among which the proportions of moderately poor and severely poor eyesight among fourth-grade girls are 18.6% and 10.4% respectively, and those among boys are 16.4% and 9% respectively; the proportions of moderately poor and severely poor eyesight among eighth-grade girls are 24.1% and 39.5% respectively, and those among boys are 22.1% and 31.7% respectively.
[0003] In fact, the overall vision of Chinese teenagers is not optimistic. Data shows that the myopia rate among teenagers ranks first in the world. This is closely related to the continuous use of electronic products such as mobile phones and computers. Judging from the development trend in recent years, the risk of myopia among teenagers is gradually increasing due to irregular learning postures and early exposure to electronic products, and they have to wear glasses since childhood. Summary of the invention
[0004] The technical problem to be solved by the present invention is to provide a method for automatically identifying the coordinates of the current reading and writing scene, so as to determine whether the user's reading and writing posture and the reading and writing distance meet the standards.
[0005] To achieve the above object, the present invention adopts the following technical solution:
[0006] A method for identifying coordinates of a reading and writing scene, characterized in that the steps include:
[0007] S1, obtain the intrinsic parameters of the camera through camera calibration;
[0008] S2, using the coordinates of the known object in the three-dimensional space, the one-to-one corresponding image pixel coordinates on the image and the camera intrinsic parameters to solve the external parameters of the camera relative to the known object in the space, that is, the rotation vector and the translation vector;
[0009] S3, perform data analysis and processing on the rotation vector to solve the three-dimensional posture and Euclidean perspective of the camera relative to the spatial coordinates of the known object.
[0010] Furthermore, the intrinsic parameters of the camera include basic parameters and distortion coefficient variables, the basic parameters include the principal point of the image optical axis, focal lengths in the X and Y directions, and the distortion coefficient vector includes a tangential distortion coefficient and a radial distortion coefficient.
[0011] Furthermore, the pinhole imaging model is used to calculate the three-dimensional coordinates of the object, and a projection center is set. The principal point is the image principal point passing through the optical axis, (X, Y, Z) is the object coordinate in the spatial coordinate system, and (x, y, z) is the image pixel coordinate.
[0012] Furthermore, the solvePnP and solvePnPRanssac functions provided by OpenCV are used to solve the rotation and translation vectors of the three-dimensional space coordinate system relative to the known object.
[0013] Furthermore, the camera intrinsic parameters, camera extrinsic parameters and custom 3D space coordinate points (0.0, 0.0, 0.0), (1.5, 0.0, 0.0,), (0.0, 0.0, 1.5), which correspond to the space origin, space X-axis, space Y-axis and space Z-axis respectively, are used to solve the one-to-one corresponding image 2D coordinate points; the projectPoints function provided by OpenCV is used to solve the coordinate points on the image coordinates corresponding to the known space axis coordinate points, and finally the corresponding image coordinate points are connected together to obtain the spatial coordinate system of the object.
[0014] The beneficial effects of the coordinate identification method for a reading and writing scene provided by the present invention are: it can effectively obtain the posture and distance of the user when reading and give a reminder for incorrect ones, and achieve a better detection effect through dual detection and judgment of reading posture and reading distance, and timely correction to reduce the probability of myopia; a monocular camera can be used to obtain accurate three-dimensional posture perspective, and then effective judgment is made through comparison of perspectives, with high accuracy, and only using a monocular camera has low cost and wide practicality; the monocular camera is set with a certain angle, which effectively avoids calculation errors caused by slight bowing and shaking. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 It is the overall flow chart of the present invention;
[0016] Figure 2 Schematic diagram of the calculation of the pinhole imaging model. DETAILED DESCRIPTION
[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary persons in the art without creative work are within the scope of protection of the present invention.
[0018] Embodiment: A method for identifying coordinates of a reading and writing scene.
[0019] A method for identifying coordinates of a reading and writing scene, characterized in that the steps include:
[0020] The monocular camera collects images and sends them to the processor for storage and analysis; the three-dimensional posture view of the monocular camera relative to the plane of the detected object is used, namely the pitch angle, yaw angle, and roll angle;
[0021] First, the camera's intrinsic parameters are obtained through camera calibration. Then, the coordinates of the known object in three-dimensional space are used, and the image pixel coordinates and the camera's intrinsic parameters that correspond one by one on the image are used to solve the camera's extrinsic parameters relative to the known object in space, namely the rotation vector and the translation vector. Finally, the rotation vector is subjected to data analysis and processing to solve the camera's three-dimensional posture and perspective relative to the known object's spatial coordinates.
[0022] Camera calibration: Here, the imaging model of the camera is based on pinhole imaging, but due to problems such as the lens itself and the camera manufacturing process, the imaging model cannot output images 100% according to the pinhole imaging model, and the output image must be distorted. Therefore, the camera needs to be calibrated. The purpose of calibration is to solve the camera's internal parameters, which include the basic camera parameters (the principal point of the image optical axis, the focal length in the X and Y directions) and the distortion coefficient vector (tangential distortion coefficient, radial distortion coefficient).
[0023] The camera is calibrated using the chessboard calibration method. The basic idea of the chessboard calibration method is to take multiple chessboard images in different directions and positions of the same chessboard calibration plate in a three-dimensional scene. Because the corner points of each chessboard image are equidistant, that is, the spatial three-dimensional coordinates of the chessboard corner points are known (the three-dimensional spatial coordinate system is relative to each chessboard object), and then calculate the pixel coordinates of each chessboard image in the image plane. With the one-to-one projection relationship between the three-dimensional spatial coordinates of each chessboard image and the corresponding two-dimensional pixel coordinates of the image pixel plane, the intrinsic parameters of the camera can be calculated.
[0024] OpenCV provides the calibrateCamera() function for calibration, which is used to obtain the intrinsic parameters of the camera, including the camera basic matrix: Distortion coefficient variables: D: (k1, k2, p1, p2[, k3[, k4, k5, k6]]), where k1, k2 are radial distortion, p1, p2 are tangential distortion. For ordinary cameras, we generally only need the first four coefficients, but for cameras with large distortion, such as fisheye cameras, we should use 5-8 coefficient variables.
[0025] The key algorithm for camera pose estimation is to solve the N-point perspective projection problem, also known as the PNP (Persperctive-N-Point) problem. Here, we use the pinhole imaging model as the standard. O is the projection center, and the principal point (u0, v0) is the principal point of the image passing through the optical axis. (X, Y, Z) is the object coordinate in the spatial coordinate system. The reference coordinate system here is the projection center of the camera, (x, y, z) is the image pixel coordinate, and the origin of the image pixel coordinate is the upper left corner. u0
[0026] According to the pinhole projection relationship above, x = f*(X / Z), we can easily derive the following projection relationship, which can be expressed in matrix form: where f x is the focal length expressed in horizontal pixels, f y The focal length expressed in vertical pixels.
[0027] When the reference coordinate system is not located at the projection center of the camera, as shown below:
[0028]
[0029] According to the formula x=M*[R|t]*X, to obtain the external parameters [R|t], it is necessary to know the camera basic matrix M, as well as the coordinate point X of the known object in three-dimensional space and the corresponding image pixel coordinate point x.
[0030] Three-dimensional posture Euclidean perspective solution: Introduce the rotation and translation matrix [R|t], where R is a 3*3 rotation matrix and t is a 3*1 translation vector. The following matrix is obtained:
[0031] Let x represent the matrix of a certain point on the image pixel plane, X represent the matrix of a certain point in the world coordinate system, and M be the basic matrix of the camera, that is, x=M*[R|t]*X.
[0032] A rotation matrix can be expressed as the above three coordinate axis matrices:
[0033]
[0034] To find the rotation angle θ, φ, and the rotation matrix R are expressed as: By calculation we can express it with the following simple code:
[0035]
[0036]
[0037] Among them, the method of atan2(y, x) is: when the absolute value of x is larger than the absolute value of y, use atan(y / x); otherwise, use atan(x / y) to ensure the stability of the numerical value. actan(y / x) is the inverse trigonometric function.
[0038] Rotation matrix R, the rotation matrix of the three coordinate axes:
[0039] A rotation around the X axis The matrix:
[0040] A matrix for rotating the Y axis by θ:
[0041] A matrix for rotating φ around the Z axis:
[0042] The camera's 3D posture is in Euclidean perspective: the yaw angle is the angle φ of rotation around the Z axis, the roll angle is the angle θ of rotation around the Y axis, and the pitch angle is the angle of rotation around the X axis.
[0043] The known camera intrinsic parameters (obtained through camera calibration above) and the known 2D image coordinate points (the corner point coordinates are obtained by extracting the square corner point features inside the object) are used to solve the camera's extrinsic parameters (i.e., rotation vector, translation vector). In this embodiment, the solvePNP function provided by OpenCV is used to solve the extrinsic parameters. Finally, the camera intrinsic parameters, camera extrinsic parameters and the custom 3D space coordinate points (0.0, 0.0, 0.0), (1.5, 0.0, 0.0,), (0.0, 0.0, 1.5), which correspond to the space origin, the space X-axis, the space Y-axis, and the space Z-axis, are used to solve the one-to-one corresponding image 2D coordinate points; the projectPoints function provided by OpenCV is used to solve the coordinate points on the image coordinates corresponding to the known space axis coordinate points, and finally the corresponding image coordinate points are connected together to obtain the object's space coordinate system.
[0044] Through object detection algorithm processing, determine whether there is a human body; substitute the image into the trained object training model according to the specified requirements to obtain the position of the human body's border and the credibility of the human body. According to the model training results, human body credibility greater than or equal to 0.4 is judged to be human. Others are judged to be no one. At the same time, obtain the four vertices of the desktop or book for the subsequent distance calculation.
[0045] Human body detection uses a calibrated data set to compile protobuf for the MobileNet-SSD model. The object detection API uses protobuf to train the model and configure parameters.
[0046] VGG16 is used as the base model. The fully connected layers fc6 and fc7 of VGG16 are converted into 3*3 but dilationrate=6 convolutional layers conv6 and 1*1 convolutional layers conv7. The dropout layer and fc8 layer are removed, and a new convolutional layer is added to obtain more feature maps for predicting offset and confidence.
[0047] The input of the algorithm is 300*300*3, and the outputs of conv4_3 (feature map size 38*38), conv7(19*19), conv8_2(10*10), conv9_2(5*5), conv10_2(3*3) and conv11_2(1*1) are used to extract 6 feature maps to predict location and confidence. A total of 38*38*4+19*19*6+10*10*6+5*5*6+3*3*4+1*1*4=8732 bounding boxes (default box) can be predicted.
[0048] Different aspect ratios of the bounding box: [1, 2, 3, 1 / 2, 1 / 3], the length, width and center of the bounding box are calculated as follows:
[0049] where a r is the aspect ratio, f k is the length or width of the kth feature map.
[0050] The data read into the human body detection model is normalized data, that is, the marked coordinates and the length and width of the object obtained need to be divided by the length and width of the original image.
[0051] The loss function is divided into two parts: positioning loss and classification loss. For the negative example prediction box, the positioning loss is zero.
[0052]
[0053]
[0054]
[0055] Where k is the category of the object, i is the default box number, and j is the ground truth box number. Since each default box corresponds to only one ground truth box, once i is determined, j and k are determined. After j is determined, the object category corresponding to the ground truth box numbered j is determined.
[0056]
[0057] Depthwise separable convolution: Depthwise separable convolution divides the standard convolution into depthwise convolution and a 1*1 convolution, i.e. pointwise convolution. Depthwise convolution uses a single convolution kernel to convolve a single input channel to obtain the depth of the input channel, and then uses a 1*1 convolution to linearly combine the outputs of the depthwise convolution.
[0058] Input feature map Convolution kernel Output feature map
[0059] D F ×D F ×MD K ×D K ×M×ND G ×D G ×N
[0060] Standard convolution calculation amount: D K ×D K ×M×N×D F ×D F
[0061] Depthwise separable convolution computation: D K ×D K ×M×D F ×D F +M×N×D F ×D F .
[0062] Through the face detection algorithm, it is determined whether there is a face; after confirming that there is a person in the picture, it is detected whether there is a face in the picture. Therefore, the face substitutes the picture into the trained model according to the specified requirements to obtain the positions of the five points of the human eye (left and right eyes), nose, mouth (two corners) and the credibility of the face. According to the model training results, the credibility is greater than or equal to 0.4 to determine that there is a face. The others are judged as no face.
[0063] The face detection model is trained through MTCNN, a Multi-task convolutional neural network, which combines face region detection with face key point detection based on the cascade framework. The overall network structure can be divided into three layers: PNet, RNet, and ONet.
[0064] P-Net stands for Proposal Network, and its basic structure is a fully connected network. For the image pyramid built in the previous step, an FCN is used to perform preliminary feature extraction and border calibration, and Bounding-Box Regression is used to adjust the window and NMS is used to filter most of the windows.
[0065] Generally, Pnet only performs detection and face frame regression. Although the input size is 12*12*3 when the network is defined, since Pnet only has convolutional layers, we can directly feed the resized image to the network for forward transmission, but the results are not 1*1*2 and 1*1*4, but m*m*2 and m*m*4. In this way, there is no need to first intercept various 12*12*3 images from the resized image and then feed them into the network. Instead, they are fed in at once, and then the corresponding 12*12 image of each result is deduced from the result. The position of the input image.
[0066] For each image in the pyramid, the network forward calculation obtains the face score and face frame regression results. The face classification score is a three-dimensional matrix of two channels m*m*2, which actually corresponds to m*m 12*12 sliding frames on the network input image. Combined with the scaling scale of the current image in the pyramid image, the specific coordinates of each sliding frame in the original image can be calculated. First, we need to filter according to the score, and exclude the sliding frames with scores below the threshold. Then use nms non-maximum suppression to merge the remaining sliding frames. When all the images in the pyramid are processed, use nms to merge the summarized sliding frames, and then use the Bbox results corresponding to the last remaining sliding frames to convert them into pixel coordinates in the original image, that is, the coordinates of the face frame are obtained.
[0067] R-Net stands for Refine Network. Its basic structure is a convolutional neural network. Compared with the first layer of P-Net, it adds a fully connected layer, so the screening of input data will be more stringent. After the image passes through P-Net, many prediction windows will be left. We send all the prediction windows to R-Net.
[0068] The face frame calculated by Pnet is cut from the original image and resized to 24*24*3 as the input of Rnet. The output is still the score and BBox regression result. Candidate frames with scores below the threshold are discarded, and the remaining candidate frames are merged by nms, and then the BBox regression results are mapped to the pixel coordinates of the original image. Therefore, Rnet finally obtains the face frame selected from the Pnet result.
[0069] O-Net stands for Output Network. Its basic structure is a more complex convolutional neural network with one more convolutional layer than R-Net. The difference between O-Net and R-Net is that this layer of structure will recognize the facial area through more supervision, and will regress the facial feature points of a person, and finally output five facial feature points.
[0070] The face frame calculated by Rnet is cut from the original image and resized to 48*48*3 as the input of Onet. The output is the score, BBox regression result and landmark position data. The Bbox regression data and landmark data corresponding to the candidate frame whose score exceeds the threshold are saved. The Bbox regression data and landmark data are mapped to the original image coordinates. NMS is implemented again to merge the face frames.
[0071] The MTCNN feature descriptor mainly consists of three parts: face / non-face classifier, bounding box regression, and landmark localization.
[0072] The cross entropy loss function for face separation is: where p i is the probability of being a face, is the true label of the background.
[0073] Bounding box regression calculates regression loss by reading and writing scene coordinate identification method distance: in are the background coordinates predicted by the network, where is the actual background coordinate.
[0074] Landmark positioning is calculated by reading and writing scene coordinate recognition method: in are the landmark coordinates predicted by the network, is the actual real landmark coordinates.
[0075] The training of multiple input sources of the face detection module is:
[0076] P-Net R-Net(αdet =1,α box =0.5,α landmark =0.5),
[0077] O-Net(α det =1,α box =0.5,α landmark =0.5), where N is the number of training samples, α j The importance of the task, is the sample label, is the loss function.
[0078] During the training process, The IoU (Intersection-over-Union) ratio of y is as follows:
[0079] 0-0.3: non-human face
[0080] 0.65-1.00: Face
[0081] 0.4-0.65: Partial face
[0082] 0.3-0.4: Landmark
[0083] The ratio of training samples is negative sample: positive sample: part sample: landmark = 3:1:1:2.
[0084] According to the data calibrated by the face detection algorithm, it is judged whether the posture meets the correct reading and writing standards; the camera on the desk lamp is 15cm away from the base plane. As long as the desk lamp is placed within the range of 30° to 150° centered on the person, the distance from the person to the camera is within 80cm, and the person sits upright, the pictures taken by the camera can include the person's complete head and clear face. The pictures taken can meet the requirements of face recognition model calculation. When people read and write normally, their shoulders are flat and their heads are slightly lowered. In order to avoid the calculation error caused by the slight lowering of the head, the camera on this desk lamp has an elevation angle of about 15°.
[0085] The conditions for this method to determine the correct reading and writing posture are (both conditions must be met at the same time):
[0086] a. The person sits upright, without tilting or lowering his head
[0087] b. The distance between eyes and the table is greater than 35 cm
[0088] According to the face model, we can get three angles: Yaw, Roll, and Picth. The following are the situations that satisfy the posture error:
[0089] ①Yaw is in the interval [0,30)
[0090] ②Yaw is in the range [30,45) and Roll is less than -10
[0091] ③Yaw is greater than or equal to 45, and Roll is less than 0
[0092] ④Yaw is in the range [-10,0) and Roll is less than -3
[0093] ⑤Yaw is in the range of [-40,-30) and Roll is less than -5
[0094] ⑥Yaw is in the range of [-30,-10) and Roll is less than -10
[0095] ⑦Yaw is greater than -40, and Roll is less than -30
[0096] Through distance calculation, determine whether the distance between the human eye and the desktop is greater than the correct distance for reading and writing; according to the coordinates of the four vertices of the desktop or book returned by the object training model, calculate the coordinates of the intersection of the diagonal of the quadrilateral based on the four vertices, and then calculate the distance from the intersection to the center of the human eye based on the obtained human eye coordinates (the center point of the left and right eye coordinates). According to the proportional relationship between this distance and the focal length of the camera, the actual distance from the human eye to the desktop or book can be calculated. When this distance is less than 35 cm, the posture is considered wrong.
[0097] Determine whether the total of posture errors and distance errors accumulated within the specified time is greater than the specified standard, and give voice reminders based on the results.
[0098] The basic principles of the present invention are as follows:
[0099] FCN (Full Roll Machine Network)
[0100] The fully convolutional network removes the fully connected layer of the traditional convolutional network, and then deconvolution is performed to upsample the feature map of the last convolutional layer (or other suitable convolutional layer) to restore it to the size of the original image (or other), and a category prediction can be made for each pixel of the deconvolution image, while retaining the spatial information of the original image. At the same time, during the process of deconvolution of the image, the final image can also be predicted by extracting the deconvolution results of other convolutional layers. The appropriate choice will make the result better and more refined.
[0101] IoU
[0102] For a sub-target image of an image and the prediction box calibrated for this sub-target image, the correlation between the final calibrated prediction box and the natural box of the real sub-image (usually requires manual calibration) is called IOU (Intersection over Union). The commonly used standard is the sum of the intersection area and the merged area of the two boxes.
[0103] Bounding-Box regression:
[0104] Problem solved: When the IOU is less than a certain value, one approach is to directly discard the corresponding prediction result. The purpose of Bounding-Box regression is to fine-tune this prediction window to make it close to the true value.
[0105] Specific logic
[0106] In image detection, a sub-window is generally represented by a four-dimensional vector (x, y, w, h), which represents the coordinates of the mother image corresponding to the center of the sub-window and its own width and height. The goal is to make the predicted window closer to the true value through some transformation when the deviation of the predicted window from the real window in the previous step is too large.
[0107] In actual use, the transformation of the input and output of the transformation according to the specific algorithm, the transformed result and the final suitable result can be understood as a linear regression of a loss function.
[0108] NMS (Non-Maximum Suppression)
[0109] As the name implies, non-maximum suppression is to suppress elements that are not maximum values. In the field of target detection, this method can be used to quickly remove prediction boxes with high overlap and relatively inaccurate calibration, but this method is not friendly to overlapping target detection.
[0110] Soft-NMS
[0111] An improved method for optimizing coincident target detection. The core is to reduce the confidence of the suppressed objects instead of directly deleting them during NMS. After processing, the confidence is unified and deleted at the end.
[0112] PRelu
[0113] In MTCNN, the activation function used in the convolutional network is PRelu, which is a Relu with parameters. Compared with Relu's method of filtering out negative values, PRelu adds parameters to negative values instead of filtering them out directly. This method will bring more computational complexity and more possibility of overfitting to the algorithm, but because more information is retained, the training results may also have better fitting performance.
[0114] The above description is only a preferred embodiment of the present invention, but the present invention should not be limited to the contents disclosed in the embodiment and the drawings, so any equivalent or modification completed without departing from the spirit disclosed in the present invention shall fall within the scope of protection of the present invention.
Claims
1. A method for identifying the coordinates of a reading and writing scene, characterized in that the steps include: S1, obtain the intrinsic parameters of the camera through camera calibration; S2, using the coordinates of the known object in three-dimensional space, according to the one-to-one corresponding image pixel coordinates on the image and the camera intrinsic parameters, solve the external parameters of the camera relative to the known object in space, that is, the rotation vector and the translation vector; according to the formula x = M*[R|t]*X, the external parameters [R|t] are required, which requires the camera basic matrix M, and the coordinate point X of the known object in three-dimensional space and the corresponding image pixel plane coordinate point x; S3, perform data analysis and processing on the rotation vector to solve the three-dimensional posture Euclidean perspective of the camera relative to the spatial coordinates of the known object at this time; Three-dimensional posture Euclidean perspective solution: introduce external parameters [R|t], where R is a 3*3 rotation matrix and t is a 3*1 translation vector, and the following matrix is obtained: x is used to represent the matrix of a coordinate point on the image pixel plane, X is used to represent the matrix of a point in the world coordinate system, and M is the basic matrix of the camera; the intrinsic parameters of the camera include basic parameters and distortion coefficient vectors, the basic parameters include the principal point of the image optical axis, the focal length in the X and Y directions, the distortion coefficient vector includes the tangential distortion coefficient and the radial distortion coefficient, the three-dimensional coordinates of the object are calculated using the pinhole imaging model, a projection center is set, the principal point is the image principal point passing through the optical axis, (X, Y, Z) is the object coordinate in the spatial coordinate system, the reference coordinate system here is the projection center of the camera, (x, y, z) is the image pixel coordinate, and the origin of the image pixel coordinate is the upper left corner; the key algorithm for camera posture estimation is to solve the N-point perspective projection problem, also known as the PNP problem, here, we take the pinhole imaging model as the standard; O is the projection center, and the principal point (u0, v0) is the image principal point passing through the optical axis; Use a rotation matrix to represent the following three axis matrices: To find the rotation angle θ, φ, and the rotation matrix R are expressed as: Rotation matrix R, the rotation matrix of the three coordinate axes: A rotation around the X axis The matrix: A matrix for rotating the Y axis by θ: A matrix for rotating φ around the Z axis: The camera's 3D posture is in Euclidean perspective: the yaw angle is the angle φ of rotation around the Z axis, the roll angle is the angle θ of rotation around the Y axis, and the pitch angle is the angle of rotation around the X axis.
2. The method for identifying the coordinates of a reading and writing scene according to claim 1, wherein: Using the camera's intrinsic parameters, camera's extrinsic parameters and custom 3D space coordinate points (0.0, 0.0, 0.0), (1.5, 0.0, 0.0,), (0.0, 0.0, 1.5), which correspond to the space origin, space X-axis, space Y-axis and space Z-axis respectively, we can solve the one-to-one corresponding image 2D coordinate points; use the projectPoints function provided by OpenCV to solve the coordinate points on the image coordinates corresponding to the known space axis coordinate points, and finally connect the corresponding image coordinate points together to obtain the object's space coordinate system.
Citation Information
Patent Citations
Monocular distance-measuring method and system based on human eye positioning
CN103793719A
Intelligent determination method for reading and writing element three-dimensional coordinates in reading and writing scene and application thereof
CN105335699A
Intelligent identification method of three-dimensional coordinates of book in reading and writing scene and application thereof
CN105354828A
Self-calibration method for linear array CCD camera, and measurement method
CN107976146A
Monocular cross-camera multi-target recognition, positioning and tracking method based on spatio-temporal continuity
CN108876823A
