A method for indoor coordinate positioning of monocular camera based on machine vision

Through machine vision technology and ORB feature point matching method, the coordinate system inconsistency caused by camera positioning changes in the indoor coordinate positioning of monocular cameras is solved, the positioning accuracy is improved and the system cost is reduced, and it is suitable for rotating camera scenes.

CN115493568BActive Publication Date: 2024-12-06CHONGQING RUANJIANG TURING ARTIFICIAL INTELLIGENCE TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211038608.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-08-26
Publication Date
2024-12-06
Estimated Expiration
2042-08-26

AI Technical Summary

Technical Problem

When facing camera positioning changes, the existing single-eye camera indoor coordinate positioning method does not have uniformity, and its application is limited by cost and structural simplicity, so it cannot meet the accuracy requirements.

Method used

Using the indoor coordinate positioning method of monocular camera based on machine vision, image distortion correction is performed through Zhang's calibration method, internal parameters and distortion parameters are obtained, reference image coordinate system is established, mapping matrix is ​​calculated using ORB feature point matching method, and arbitrary position poses are selected. The target pixel coordinates are transformed into the image space of the reference position, and the target position coordinates are calculated based on the vertical field angle, the horizontal field angle and the optical axis pitch angle.

Benefits of technology

It improves the accuracy of coordinate positioning of monocular cameras, reduces the cost of measurement systems, avoids dependence on mobile detection carriers, and is suitable for rotating camera scenes, realizing the uniformity of the coordinate system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115493568B_ABST
    Figure CN115493568B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of indoor positioning technology, and specifically relates to a method for indoor coordinate positioning of a monocular camera based on machine vision. The present invention proposes to correct image distortion and improve image quality, so that the coordinate positioning accuracy based on the monocular camera is higher. The method adopted by the present invention only requires one-time position selection and parameter calibration, and does not require a specific mobile detection carrier, so that the measurement system has low cost and simple structure. For scenes where the monocular camera rotates, a feature point matching method is used to calculate the mapping matrix from any pose to a reference pose, and the mapping matrix is ​​used to transform the pixel coordinates of the detected target in any pose to the image space of the reference pose, which solves the problem that the coordinate systems for measuring different camera poses are not unified and the application of existing measurement methods is limited.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of indoor positioning, and specifically relates to a monocular camera indoor coordinate positioning method based on machine vision. Background Art

[0002] The document "Research on Real-time Distance Measurement Method Based on Monocular Vision [J]. Journal of Image and Graphics, 2006(01):74-81" proposed a method for solving the geometric relationship between the distance between the vehicle and the obstacle in front based on monocular vision, which is suitable for automatic driving of automobiles. However, this method is sensitive to the change of the pitch angle of the monocular camera. When the pitch angle changes by 0.4°, the measured distance changes by 5.4445m. This method cannot meet the accuracy requirements of indoor coordinate positioning.

[0003] A Chinese patent application numbered "201611243556.9" proposes a distance measurement method based on a monocular camera. The method uses the distance between the ground boundary point in the monocular camera image and the monocular camera and the pixel coordinates of the measurement point in the image to construct a calculation formula to obtain the physical space distance between the object point and the monocular camera. However, this method does not work well in practical operations, cannot meet the accuracy requirements of coordinate positioning, and cannot cope with the impact of monocular camera rotation.

[0004] The Chinese patent application number "201811579928.4" proposes a photogrammetry method and system based on a monocular camera. It proposes to calibrate the width and height of the monocular camera image, the vertical field of view angle, horizontal field of view angle and optical axis pitch angle of the monocular camera, collect two images of a mobile detection carrier and the moving distance, and obtain the spatial position of any target in the field of view of the monocular camera by solving the formula. This method requires collecting two images of a mobile detection carrier and the moving distance. To obtain the relative moving distance of the mobile detection carrier during the two image collections, additional measurement tools are required or the mobile detection carrier itself has measurement function requirements.

[0005] In indoor measurement scenarios, the monocular cameras used often have an automatic rotation function or are manually adjusted. Due to the change in camera posture, the position coordinate system measured at each camera posture is not uniform. Under the requirements of low cost and simple structure, the application of existing measurement methods based on monocular cameras is limited. Summary of the invention

[0006] In view of the above problems, the present invention proposes a monocular camera indoor coordinate positioning method based on machine vision to solve the problems that the coordinate systems for measuring different camera poses are not unified and the application of existing measurement methods is limited.

[0007] The technical solution of the present invention is:

[0008] A method for indoor coordinate positioning of a monocular camera based on machine vision comprises the following steps:

[0009] S1. Set the monocular camera above the indoor floor, define the projection point of the monocular camera on the indoor floor as point I, and define the vertical central axis of the monocular camera imaging of the indoor scene as EF. Based on the requirement that the extension line of point I must coincide with EF and the field of view requirement of the monocular camera imaging, select the installation posture of the monocular camera and use it as the reference posture;

[0010] S2. Using Zhang's calibration method, perform distortion correction on the monocular camera imaging, obtain the intrinsic parameters and distortion parameters of the monocular camera, and perform distortion correction and cropping on the image obtained by the monocular camera at the reference posture as the reference image;

[0011] S3, collect parameter data: measure the installation height of the monocular camera relative to the indoor floor, define the coordinate point of the monocular camera as point O, then the obtained height is IO, and establish a coordinate system based on the reference image, with the intersection of the horizontal central axis KJ and the vertical central axis EF of the image as the origin G, the vertical central axis as the Y axis, and the horizontal central axis as the X axis; obtain the vertical field of view angle 2α, the horizontal field of view angle 2β and the optical axis pitch angle θ, as well as the pixel width W and pixel height H of the image, where:

[0012]

[0013]

[0014]

[0015] The point F is located on the line segment IG;

[0016] S4, using a monocular camera to acquire a scene image, and obtaining the pixel coordinates of the target in the image based on a target detection method based on machine vision;

[0017] S5, using feature point matching method to calculate the mapping matrix from any pose of the monocular camera to the reference pose, and using the mapping matrix to transform the target pixel coordinates obtained in S4 to the image space of the reference pose, specifically:

[0018] The ORB features of the scene imaging graph of the reference pose and the scene imaging graph of the arbitrary pose are obtained respectively, and the ORB features include feature points and descriptors;

[0019] Use the feature matching method BFMatcher to calculate the distance between two sets of feature points, and obtain multiple pairs of best matching points through sorting and screening;

[0020] Use the mapping matrix solution method FindHomography to calculate the optimal single mapping transformation matrix between multiple pairs of feature points. The mapping matrix represents the mapping relationship from the scene imaging map of any pose to the scene imaging map of the reference pose.

[0021] Use the mapping matrix to transform the target pixel coordinates of any pose to the image space of the reference pose;

[0022] S6, according to the pixel coordinates of the target in the reference posture image space, and the vertical field of view angle, horizontal field of view angle and optical axis pitch angle obtained in S3, the position coordinates of the target on the indoor ground are calculated to complete the target positioning, specifically:

[0023] Define the coordinates of the target in the reference pose image space as (x′, y′), and the Y-axis coordinate of the target P in the reference pose is P y , then the size information of the target in the X-axis and Y-axis directions is GP y and P y P:

[0024]

[0025]

[0026]

[0027] Thereby obtaining the positioning coordinates of the target.

[0028] In the above scheme, the monocular camera used is a non-zoom monocular camera, so the horizontal field of view angle and the vertical field of view angle obtained in S3 are constants and will not change due to changes in the camera's posture.

[0029] The beneficial effect of the present invention is that the present invention proposes to correct image distortion and improve image quality, so that the coordinate positioning accuracy based on the monocular camera is higher. The method adopted by the present invention only needs to perform position selection and parameter calibration once, and does not require a specific mobile detection carrier, so that the measurement system has low cost and simple structure. For scenes where the monocular camera rotates, the feature point matching method is used to calculate the mapping matrix from any posture to the reference posture, and the mapping matrix is ​​used to transform the pixel coordinates of the detected target in any posture to the image space of the reference posture, which solves the problem that the coordinate systems for measuring different camera postures are not unified and the application of existing measurement methods is limited. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flow chart of the indoor coordinate positioning method of a monocular camera based on machine vision of the present invention.

[0031] Figure 2Schematic diagram of the distortion types that exist in scene imaging based on a monocular camera; 2(a) is a normal image, 2(b) is barrel distortion, and 2(c) is pincushion distortion.

[0032] Figure 3 It is a schematic diagram of scene imaging of a monocular camera; points a, b, c, and d are the four corner points of the scene imaging image, the x-axis and y-axis are the horizontal and vertical central axes of the scene imaging image, respectively, points e and f are the intersection points of the vertical central axis and the image boundary, points k and j are the intersection points of the horizontal central axis and the image boundary, g is the intersection point of the horizontal central axis and the vertical central axis, and H and W are the pixel height and pixel width of the image, respectively. The xgy coordinate system is the pixel position coordinate system of scene imaging, point p is the pixel point on the image to be located, and the projection points of point p on the x-axis and y-axis are p x , p y .

[0033] Figure 4 It is a schematic diagram of the monocular camera scene imaging model; point O is the installation position of the monocular camera, points A, B, C, and D are the corner points on the indoor ground corresponding to the monocular camera scene imaging, and the plane composed of ABCD represents the ground plane; points A, B, C, D, E, F, G, K, J, and P on the ground plane correspond to points a, b, c, d, e, f, g, k, j, and p on the scene imaging diagram respectively; the X-axis and Y-axis on the ground plane correspond to the x-axis and y-axis on the scene imaging diagram respectively, and point I is the projection point of the monocular camera installation point on the indoor ground. The XGY coordinate system is the physical position coordinate system of the detected target on the ground, and point P is the target to be located on the ground.

[0034] Figure 5 It is a schematic diagram of the model for solving the Y-axis coordinate of the target point.

[0035] Figure 6 It is a schematic diagram of the model for solving the X-axis coordinate of the target point.

[0036] Figure 7 7(a) is a schematic diagram of the camera pose under the reference pose; 7(b) is a schematic diagram of the camera pose under any pose.

[0037] Figure 8 It is a schematic diagram of the relationship between the scene imaging diagram of a monocular camera in a reference pose and the scene imaging diagram in an arbitrary pose; 8(a) is the scene imaging diagram of the camera in the reference pose, and 8(b) is the scene imaging diagram of the camera in an arbitrary pose; the straight line segments in the figure are the correspondence lines of the key points of the two images based on key point matching; the white broken line in the image of 8(b) is the field of view boundary of the image in an arbitrary pose after the transformation operation to the reference pose. DETAILED DESCRIPTION

[0038] The present invention will be described in detail below in conjunction with the accompanying drawings.

[0039] The method of the present invention is used for the coordinate positioning of indoor targets whose cameras have an automatic rotation function or whose camera posture is adjusted. The monocular camera is installed at a higher position indoors, and the lens of the monocular camera faces the indoor floor and has an angle with the floor, that is, the optical axis of the monocular camera has a certain angle with the floor. In the method of the present invention, the setting of the monocular camera must meet necessary conditions, such as Figure 3 , define the scene imaging image of the monocular camera, a, b, c, d are the four corner points of the scene imaging image, the x-axis and y-axis are the horizontal and vertical axis of the scene imaging image, e and f are the intersections of the vertical axis and the image edges ab and cd, k and j are the intersections of the horizontal axis and the image edges ad and bc, g is the intersection of the horizontal axis and the vertical axis, and g is the origin, H and W are the pixel height and pixel width of the image, respectively. The xgy coordinate system is the pixel position coordinate system of the scene imaging, point p is the pixel point of the target to be located on the image, and the projection points of point p on the x-axis and y-axis are p x , p y . Map the monocular camera scene image to the indoor positioning scene, such as Figure 4 As shown, point O is the installation position of the monocular camera, points A, B, C, and D are the corner points on the indoor ground corresponding to the monocular camera scene imaging, and the plane composed of ABCD represents the ground plane; points A, B, C, D, E, F, G, K, J, and P on the ground plane correspond to points a, b, c, d, e, f, g, k, j, and p on the scene imaging diagram respectively; the X-axis and Y-axis on the ground plane correspond to the x-axis and y-axis on the scene imaging diagram respectively, and point I is the projection point of the monocular camera installation point on the indoor ground. The XGY coordinate system is the physical position coordinate system of the detected target on the ground, and point P is the target to be located on the ground. In the scheme of the present invention, the condition that the monocular camera needs to meet is that the straight line of the vertical median of the monocular camera imaging at the corresponding position on the indoor ground passes through the projection point of the camera installation point on the indoor ground, that is, the extension line of point I needs to coincide with EF, and the camera is a non-zoom camera.

[0040] like Figure 1 As shown, the method of the present invention comprises the following steps:

[0041] Step S1: Figure 4 As shown in Figure 1, the requirement that point I passes through the straight line EF is met, and the camera reference pose with a better field of view is selected.

[0042] To obtain more scene information, the monocular camera installed indoors should first adjust the camera posture to have a better field of view. Figure 5 and Figure 6In order to make the position coordinate calculation model feasible and achieve centimeter-level accuracy in indoor scene position coordinate calculation, it is necessary to ensure that point I passes through the EF line, that is, the line corresponding to the vertical median of the monocular camera's imaging on the indoor ground, and passes through the projection point of the camera installation point on the indoor ground. If point I is far away from the line EF, the depth of the scene imaging will not satisfy the symmetry with respect to the y-axis, making the position coordinate calculation method using similarity principle and symmetry invalid. In general, the camera posture is as follows: Figure 7 The state shown in (a) can basically ensure that point I passes through straight line EF.

[0043] Step S2: perform image distortion correction and edge cropping, and collect a scene image of the reference pose.

[0044] Generally, due to the structural design of the camera or the assembly error of the camera lens, the camera image has certain distortion, such as Figure 2 (b) shows barrel distortion, and (c) shows pincushion distortion. Since the position coordinate calculation model uses operations such as projection from a point to the coordinate axis, the image needs to have good quality and image correction must be performed.

[0045] Let the Cartesian coordinates of a point on the image plane be [u,v] T , its polar coordinate form is [r,Φ] T , r represents the distance from the point to the origin of the coordinate system, and φ represents the angle with the horizontal axis. Image distortion can be divided into radial distortion and tangential distortion, which can be expressed in mathematical form as follows:

[0046] Mathematical model of radial distortion:

[0047]

[0048] Tangential distortion mathematical model:

[0049]

[0050] [u′,v′] T is the coordinate of the distorted point. In the process of distortion correction, five distortion items, k1, k2, k3, p1 and p2, are used. The Zhang calibration method can be used to obtain the distortion vector, and the image can be corrected using the distortion vector. Due to image correction, pixel information will be lost on the edge of the corrected image due to the correction operation, so image cropping is necessary.

[0051] After image distortion correction and cropping, an image of the scene in a reference pose is collected.

[0052] Step S3: Collect the parameter data needed for this positioning method, measure the camera installation height, the size of the vertical field of view boundary point, the horizontal field of view boundary point, and the optical axis position point.

[0053] Since a non-zoom monocular camera is used, the horizontal and vertical field of view angles are constants of the camera's inherent characteristics and will not change due to changes in the camera's posture. The horizontal and vertical field of view angles can be calibrated at any camera posture after distortion correction. For ease of operation, all measurement data are taken at the camera's reference posture.

[0054] Measure the camera installation height, that is, the size of IO; measure the position point size information of the optical axis, that is, the size of IG; measure the vertical field of view boundary information used to calculate the vertical field of view angle, that is, the size of IF; measure the horizontal field of view boundary information used to calculate the horizontal field of view angle, that is, the size of GJ; measure the pixel height H and pixel width W of the image.

[0055] Step S4: For images of any camera pose, use the target detection method of machine vision to obtain the pixel coordinates of the detection target.

[0056] The imaging of a monocular camera only obtains the scene image information under the corresponding field of view. In order to complete indoor coordinate positioning, it is necessary to use the target detection method of machine vision to perform target detection on the scene image to obtain the pixel coordinates of the detected target.

[0057] First, we build a target detection model. Generally, we use a target detection model based on deep learning. After training and testing with a large amount of data, the detection model has a strong generalization detection function. Then, we input the rectified and cropped image into the target detection model. After a series of anchor frame extraction, convolution pooling and post-processing operations inside the model, the model outputs the location information of the detected target. Finally, we transform the location information into Figure 3 In the xgy coordinate system shown.

[0058] Step S5: For any camera posture, the mapping matrix is ​​obtained by using the feature point matching method to transform the detection target coordinates of any posture into the image space of the reference posture.

[0059] After the camera rotates automatically or is manually adjusted, the camera optical axis deviates from the reference position. Since no corresponding position calibration and dimension data measurement are performed for any camera rotation, the coordinate system position of the scene imaged under the new camera position is unclear. Figure 7 As shown in (b), at this time, the projection point I of the camera installation point O on the indoor ground is no longer on the straight line where the Y axis is located, so the application of the target position calculation method based on step S106 is limited. In order to make the measured position coordinate system uniform and the calculation method can still be used in special camera postures, the feature point matching method is used to calculate the mapping matrix from any posture to the reference posture, and the mapping matrix is ​​used to transform the detection target coordinates of any posture to the image space of the reference position.

[0060] The method of solving the mapping matrix based on feature point matching: First, obtain the ORB features of the scene imaging map of the reference pose and the scene imaging map of any pose respectively. The ORB features include feature points and descriptors. Feature points are used to screen relatively "special" points, while descriptors are used to describe the features around a certain point. Secondly, use the feature matching method BFMatcher to calculate the distance between the two sets of feature points, and obtain several pairs of best matching points through sorting and screening. Then, use the mapping matrix solution method FindHomography to calculate the optimal single mapping transformation matrix between multiple pairs of feature points. The mapping matrix represents the mapping relationship from the scene imaging map of any pose to the scene imaging map of the reference pose. Finally, use the mapping matrix to transform the coordinates of the detected target in any pose to the image space of the reference pose. The operation effect of this step is as follows: Figure 8 shown.

[0061] Step S6: For the pixel coordinates of the detection target in the reference pose image space, use a position calculation model based on similar triangles and triangular transformation to calculate its coordinates in the physical position coordinate system XGY.

[0062] Assuming that the xgy coordinate system coordinates of the detection target in the reference posture obtained according to steps S4 and S5 are (x′, y′), then according to the data collected in step S3, the XGY coordinate system coordinates in the reference posture can be obtained according to the process of this step.

[0063] The optical axis pitch angle θ is Figure 5 Dimensions of ∠IOG:

[0064]

[0065] The vertical field of view is 2α, that is Figure 5 Dimensions of ∠EOF:

[0066]

[0067] The horizontal field of view is 2β, that is Figure 6 Size of ∠KOJ:

[0068]

[0069] like Figure 5 As shown, there is a line segment ML passing through point G and parallel to fe, point L is on the extension line of line segment fF, and point M is on line segment OE. ML and OP y Intersect at point Z; P y The dot is the projection of point P on the Y axis; the meanings of the remaining characters are the same as Figure 3 and Figure 4 The size information of point P in the Y-axis direction in the XGY coordinate system is GPy .

[0070] According to the relationship between trigonometric functions and similar triangles, we know that:

[0071]

[0072]

[0073]

[0074] so,

[0075]

[0076]

[0077] and

[0078]

[0079] So

[0080]

[0081] like Figure 6 As shown, PP y The extended line of and the BC line segment intersect at point R, and the size information of point P in the X-axis direction in the XGY coordinate system is P y P.

[0082] In the triangle ΔROP y middle,

[0083]

[0084] According to the similar triangles imaged by point P, we can get:

[0085]

[0086] So

[0087]

[0088]

[0089] Step S106 can obtain the size information P of point P in the X-axis and Y-axis directions in the XGY coordinate system. y P and GP y .

[0090] In summary, the indoor coordinate positioning method of a monocular camera based on machine vision of the present invention can obtain the indoor ground position coordinates with a unified coordinate system for the target in the camera image in any posture.

Claims

1. A method for indoor coordinate positioning of a monocular camera based on machine vision, characterized in that: The following steps are involved: S1. Set the monocular camera above the indoor floor, define the projection point of the monocular camera on the indoor floor as point I, and define the vertical central axis of the monocular camera imaging of the indoor scene as EF. Based on the requirement that the extension line of point I must coincide with EF and the field of view requirement of the monocular camera imaging, select the installation posture of the monocular camera and use it as the reference posture; S2. Using Zhang's calibration method, perform distortion correction on the monocular camera imaging, obtain the intrinsic parameters and distortion parameters of the monocular camera, and perform distortion correction and cropping on the image obtained by the monocular camera at the reference posture as the reference image; S3, collect parameter data: measure the installation height of the monocular camera relative to the indoor floor, define the coordinate point of the monocular camera as point O, and the obtained height is IO; measure the position point size information of the optical axis as IG; measure the vertical field of view boundary information used to calculate the vertical field of view angle as IF; measure the horizontal field of view boundary information used to calculate the horizontal field of view angle as GJ; and establish a coordinate system based on the reference image, with the intersection of the horizontal central axis KJ and the vertical central axis EF of the image as the origin G, the vertical central axis as the Y axis, and the horizontal central axis as the X axis; obtain the vertical field of view angle 2α, the horizontal field of view angle 2β and the optical axis pitch angle θ, as well as the pixel width W and pixel height H of the image, where: The point F is located on the line segment IG; S4, using a monocular camera to acquire a scene image, and obtaining the pixel coordinates of the target in the image based on a target detection method based on machine vision; S5, using feature point matching method to calculate the mapping matrix from any pose of the monocular camera to the reference pose, and using the mapping matrix to transform the target pixel coordinates obtained in S4 to the image space of the reference pose, specifically: The ORB features of the scene imaging graph of the reference pose and the scene imaging graph of the arbitrary pose are obtained respectively, and the ORB features include feature points and descriptors; Use the feature matching method BFMatcher to calculate the distance between two sets of feature points, and obtain multiple pairs of best matching points through sorting and screening; Use the mapping matrix solution method FindHomography to calculate the optimal single mapping transformation matrix between multiple pairs of feature points. The mapping matrix represents the mapping relationship from the scene imaging map of any pose to the scene imaging map of the reference pose. Use the mapping matrix to transform the target pixel coordinates of any pose to the image space of the reference pose; S6, according to the pixel coordinates of the target in the reference posture image space, and the vertical field of view angle, horizontal field of view angle and optical axis pitch angle obtained in S3, the position coordinates of the target on the indoor ground are calculated to complete the target positioning, specifically: Define the coordinates of the target in the reference pose image space as (x′, y′), and the Y-axis coordinate of the target P in the reference pose is P y , then the size information of the target in the X-axis and Y-axis directions is GP y and P y P: Thereby obtaining the positioning coordinates of the target.

2. The method for indoor coordinate positioning of a monocular camera based on machine vision according to claim 1, characterized in that: The monocular camera is a non-zoom monocular camera, so the horizontal field of view angle and the vertical field of view angle obtained in S3 are constants and will not change due to changes in the camera's posture.