Measurement method and apparatus for tunnel interior planar mapping model based on panoramic imagery

Abstract

This invention discloses a measurement method and apparatus for a tunnel planar mapping model based on panoramic images, comprising the following steps: Step S1, acquiring a panoramic image from a camera; Step S2, obtaining a tunnel planar mapping model based on the panoramic image; Step S3, measuring points within the tunnel on the real-time acquired panoramic image based on the tunnel planar mapping model. Using the technical solution of this invention, the measurement of the ground or walls within a tunnel can be achieved, and it facilitates immersive browsing.

Description

Technical Field

[0001] This invention belongs to the field of panoramic observation technology, and in particular relates to a measurement method and device for a tunnel planar mapping model based on panoramic images. Background Technology

[0002] With the advancement and development of panoramic observation technology, people's demands for browsing experience and ease of modeling spatial information are constantly increasing. Spherical panoramic cameras can capture panoramic images, providing users with a superior browsing experience. Simultaneously, each panoramic image can be projected onto a spherical surface, providing the azimuth and pitch angles of each pixel in the panoramic image. This lays the foundation for modeling based on panoramic imagery. Current research has focused on constructing panoramic image pairs based on panoramic images and applying them to street views and digital campuses. However, existing research rarely applies this technology to tunnel browsing and measurement.

[0003] Tunnel visualization and modeling are fundamental requirements for tunnel management. From an experiential perspective, building easy-to-browse models that allow for an immersive experience of the tunnel's interior and realistic observation of its morphology is highly appealing to tunnel managers. From a tunnel safety perspective, constructing sophisticated tunnel models to detect defects is highly attractive to tunnel safety maintenance personnel.

[0004] Traditional methods for modeling 3D terrain involve taking multiple photographs to create image pairs and then calculating point clouds. However, many tunnels in reality are poorly lit, and their floors and sides are mostly smooth concrete walls or decorative panels with relatively simple textures. These characteristics make it difficult to match feature points when using image pair methods for tunnel modeling. This problem often leads to low accuracy in tunnel models. Furthermore, models built using this method have poor tunnel measurement accuracy and are not conducive to immersive viewing experiences. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a measurement method and device for a planar mapping model inside a tunnel based on panoramic images, so as to realize the measurement of the ground or walls inside the tunnel and facilitate immersive browsing.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A measurement method for a tunnel interior planar mapping model based on panoramic imagery includes the following steps:

[0008] Step S1: Acquire panoramic images from the camera;

[0009] Step S2: Based on the panoramic image, obtain the tunnel interior planar mapping model, which includes: the mapping matrix of the tunnel interior facing the ground and the wall.

[0010] Step S3: Based on the tunnel's internal planar mapping model, measure the points within the tunnel on the real-time acquired panoramic image.

[0011] As a preferred option, the plane mapping model H inside the tunnel is:

[0012] H=λFR x R y R z T

[0013] in, R x ,R y ,R z T represents the rotation matrix and position offset matrix around the X-axis, Y-axis, and Z-axis, respectively;

[0014] Wherein, ω, θ, φ are the camera's roll angle, pitch angle, and azimuth angle attitude data, f is the focal length, (u0, v0) are the principal points, λ is the scale factor, and (X) is the focal length. s ,Y s Z s () represents the coordinates of the photography center.

[0015] Preferably, in step S3, the distance Z from the panoramic camera to the ground is determined. h This yields the true coordinates (X, Y) corresponding to any pixel (φ1, θ1), i.e.

[0016]

[0017] Where H' represents the real-time camera pose data ω′, θ′, The result is obtained by substituting H.

[0018] Preferably, in step S3, the true coordinates (X, Y) corresponding to any pixel (φ1, θ1) are obtained based on the distance L from the panoramic camera to the vertical wall of the side tunnel.

[0019]

[0020] Where H' represents the real-time camera pose data ω′, θ′, The result is obtained by substituting H.

[0021] The present invention also provides a measurement device for a tunnel planar mapping model based on panoramic images, comprising:

[0022] The acquisition module is used to acquire panoramic images from the camera;

[0023] The module is used to obtain a planar mapping model inside the tunnel based on the panoramic image. The planar mapping model inside the tunnel includes a mapping matrix for the tunnel facing the ground and walls.

[0024] The measurement module is used to measure points inside the tunnel on real-time panoramic images based on the tunnel's internal planar mapping model.

[0025] As a preferred option, the plane mapping model H inside the tunnel is:

[0026] H=λFR x R y R z T

[0027] in, R x ,R y ,R z T represents the rotation matrix and position offset matrix around the X-axis, Y-axis, and Z-axis, respectively;

[0028] Wherein, ω, θ, φ are the camera's roll angle, pitch angle, and azimuth angle attitude data, f is the focal length, (u0, v0) are the principal points, λ is the scale factor, and (X) is the focal length. s ,Y s Z s () represents the coordinates of the photography center.

[0029] Preferably, the measurement module is based on the distance Z from the panoramic camera to the ground. h This yields the true coordinates (X, Y) corresponding to any pixel (φ1, θ1), i.e.

[0030]

[0031] Where H' represents the real-time camera pose data ω′, θ′, The result is obtained by substituting H.

[0032] Preferably, the measurement module obtains the true coordinates (X, Y) of any pixel (φ1, θ1) based on the distance L from the panoramic camera to the vertical wall of the side tunnel.

[0033]

[0034] Where H' represents the real-time camera pose data ω′, θ′, The result is obtained by substituting H.

[0035] A mapping equation between the tunnel surface and the panoramic image is constructed using image mapping, enabling the measurement of points inside the tunnel from the panoramic image; this allows for tunnel modeling and measurement even without knowing the ground control points. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0037] Figure 1 This is a flowchart illustrating the measurement method of the tunnel planar mapping model based on panoramic images according to an embodiment of the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] Example 1:

[0041] like Figure 1 As shown, this embodiment of the invention provides a measurement method for a planar mapping model within a tunnel based on panoramic images, comprising the following steps:

[0042] Step S1: Acquire panoramic images from the camera;

[0043] Step S2: Based on the panoramic image, obtain the tunnel interior planar mapping model, which includes: the mapping matrix of the tunnel interior facing the ground and the wall.

[0044] Step S3: Based on the tunnel's internal planar mapping model, measure the points within the tunnel on the real-time acquired panoramic image.

[0045] As one embodiment of the present invention, the tunnel internal plane mapping model H is:

[0046] H=λFR x R y R z T

[0047] in, R x ,R y ,R z T represents the rotation matrix and position offset matrix around the X-axis, Y-axis, and Z-axis, respectively;

[0048] Wherein, ω, θ, φ are the camera's roll angle, pitch angle, and azimuth angle attitude data, f is the focal length, (u0, v0) are the principal points, λ is the scale factor, and (X) is the focal length. s ,Y s Z s () represents the coordinates of the photography center.

[0049] In one embodiment of the present invention, in step S3, the distance Z from the panoramic camera to the ground is determined. h This yields the true coordinates (X, Y) corresponding to any pixel (φ1, θ1), i.e.

[0050]

[0051] Where H' represents the real-time camera pose data ω′, θ′, The result is obtained by substituting H.

[0052] In one embodiment of the present invention, in step S3, the true coordinates (X, Y) corresponding to any pixel (φ1, θ1) are obtained based on the distance L from the panoramic camera to the vertical wall of the side tunnel.

[0053]

[0054] Where H' represents the real-time camera pose data ω. ′ ,θ ′ , The result is obtained by substituting H.

[0055] Example 2:

[0056] like Figure 1 As shown, this embodiment of the invention provides a measurement method for a planar mapping model within a tunnel based on panoramic images, comprising the following steps:

[0057] Step S1: Acquire panoramic images from the camera;

[0058] Step S2: Based on the panoramic image, obtain the tunnel interior planar mapping model, which includes: the mapping matrix of the tunnel interior facing the ground and the wall.

[0059] Step S3: Based on the tunnel's internal planar mapping model, measure the points within the tunnel on the real-time acquired panoramic image.

[0060] In one embodiment of the present invention, in step S2, the panoramic image is an image projected onto a sphere. The coordinates of a pixel on the panoramic image are (u, v), and the corresponding image angle coordinates are (φ, θ). The image angle coordinates are converted into a single photograph through projection, and then a mapping relationship between the projected single photograph and the real world is established.

[0061]

[0062] If the mapping surface remains unchanged, the further a pixel is from the projection center, the larger the area represented by each pixel, and the greater the projection error. More importantly, if the mapping model is built facing the ground, when the points on both sides are on the wall, each planar coordinate corresponds to multiple elevation points, leading to multiple solutions. Therefore, it is necessary to consider using projection models with different angles for data at different elevation angles.

[0063] Let the image angular coordinates of the point to be mapped along the axis be (φ0, θ0), and the point within the mapping plane along this direction be (φ1, θ1). Then the image coordinates of any point within the mapping plane can be expressed as:

[0064]

[0065] The mapping point is generally chosen as the point closest to the mapping surface or the center point of the area to be mapped and modeled. When the mapping surface is the ground, θ0 = -90°, φ0 = 0°. In this case, the plane tangent to a sphere with a radius of 1 unit is used as the projection surface. The coordinates of the projected planar image are (x, y).

[0066]

[0067] When the mapped surface is a vertical wall in the direction of travel inside the tunnel, formula (2) becomes formula (4).

[0068]

[0069] Therefore, the mapping plane of the vertical wall on the right side in the direction of travel is θ0 = 0°, φ0 = 90°. The mapping plane of the vertical wall on the left side in the direction of travel is θ0 = 0°, φ0 = 270°.

[0070]

[0071] Since most of the ground inside the tunnel can be approximated as a plane, and most of it is vertical except for curved surfaces, formulas (2) and (4) can satisfy the plane / vertical surface mapping situation in most tunnels. Based on formula (2), when the mapped surface is the ground, θ0 = -90°, φ = 0°.

[0072] Formula (6) can be obtained, where Z h It can be calculated using the height difference between the ground and the camera.

[0073]

[0074] When the mapped surface is the vertical wall in the direction of travel within the tunnel, θ0 = 0°. If the direction of travel φ0 = 0°, then the mapped surface of the vertical wall on the right side of the direction of travel is θ0 = 0°, φ0 = 90°. The mapped surface of the vertical wall on the left side of the direction of travel is θ0 = 0°, φ0 = 270°. The vertical mapping formula is formula (7), where Y h It can be represented by the vertical distance from the camera to the wall.

[0075]

[0076] Formulas (6) and (7) are the mapping matrices for the tunnel facing the ground and walls, respectively. These two mapping matrices can satisfy most cases of planar / vertical plane mapping within tunnels.

[0077] To obtain the azimuth angle φ0 of the panoramic camera relative to the tunnel's forward direction, as well as the other attitude angles, at least six points with known spatial coordinates within the tunnel need to be selected as control points. Substituting the coordinates of these six control points into formula (6) or (7) yields the mapping matrix for the tunnel's orientation towards the ground and walls.

[0078] The mapping matrix H can be further decomposed into expressions for the scale coefficient λ, focal length f, principal point (u0, v0), and attitude angle of the projection plane image.

[0079] H=λFR x R y R z T (8)

[0080] in, R x ,R y ,R z T represents the rotation matrix and position offset matrix around the X-axis, Y-axis, and Z-axis, respectively.

[0081]

[0082] In one embodiment of the present invention, in step S3, the camera's attitude data ω is acquired in real time. ' ,θ ' ,φ'. The X-axis and Y-axis represent the tunnel's forward direction and the horizontal direction perpendicular to the forward direction, respectively. The rotation of the X-axis and Y-axis corresponds to the roll angle ω. ' and pitch angle θ ' In the tunnel, the elevation difference between the two track surfaces creates an inclination angle, while the inclination angle is created by the slope. A gyroscope can sensitively detect changes in these two angles. Therefore, based on these two angles, one can obtain... Rotation along the Z-axis corresponds to the azimuth angle. When the azimuth angle changes, the tunnel's direction also changes. Therefore, R... z It remains unchanged. The mapping matrix at this time can be expressed as formula (9).

[0083] H'=λFR x 'R y 'R z T (9)

[0084] The instrument's height Z is obtained by measuring the distance from the panoramic camera to the ground. h Substituting this into formula (6) along with formula (9), we can obtain the true coordinates (X,Y) corresponding to any pixel (φ1,θ1).

[0085]

[0086] The distance from the camera mounted on the track trolley to the ground remains essentially constant. Therefore, the instrument's height can be adjusted. h Set to a constant value. Then, in formula (10), you only need to input the camera's attitude angle when shooting to obtain the mapping matrix H, and you can calculate the real coordinates of any pixel in the panoramic photo.

[0087] By measuring the distance L from the panoramic camera to the vertical wall of the side tunnel, and substituting it together with formula (8) into formula (7), the true coordinates (X,Y) corresponding to any pixel (φ1,θ1) can be obtained.

[0088]

[0089] The distance from the camera mounted on the track trolley to a wall inside the tunnel remains essentially constant. Therefore, the instrument height L can be set to a constant value. Then, in formula (11), only the camera's attitude angle at the time of shooting needs to be input to obtain the mapping matrix H', which can then be used to calculate the true coordinates of any pixel in the panoramic photo.

[0090] Detailed operation steps:

[0091] To measure the coordinates of any point on the corresponding plane using a panoramic camera, the following steps are required.

[0092] (1) Fix the panoramic camera on the track trolley.

[0093] (2) Use the trolley to collect the camera's attitude parameters ω ' ,θ ' ,φ'. The three angles correspond to the roll angle, pitch angle, and azimuth angle, respectively.

[0094] (3) Substitute the three angles into formula (9) to obtain matrix H'.

[0095] (4) For ground measurements, measure the camera height and obtain the ground elevation difference Z. h .

[0096] (5) For both sides of the wall, measure the vertical distance from the camera to the wall and obtain the parameter L.

[0097] (6) Set control points inside the tunnel and substitute them into formula (6). Obtain the homography matrix H.

[0098] (7) H', Z h Substitute L into formulas (10) and (11) respectively.

[0099] (8) Input the spherical angular coordinates of any point in the panoramic image. If it is a ground point, substitute it into formula (10). If it is a wall point, substitute it into formula (11).

[0100] (9) Calculate the absolute coordinates of the input pixel using formulas (10) and (11).

[0101] Example 3:

[0102] This invention also provides a measurement device for a tunnel planar mapping model based on panoramic images, comprising:

[0103] The acquisition module is used to acquire panoramic images from the camera;

[0104] The module is used to obtain a planar mapping model inside the tunnel based on the panoramic image. The planar mapping model inside the tunnel includes a mapping matrix for the tunnel facing the ground and walls.

[0105] The measurement module is used to measure points inside the tunnel on real-time panoramic images based on the tunnel's internal planar mapping model.

[0106] As one embodiment of the present invention, the tunnel internal plane mapping model H is:

[0107] H=λFR x Ry R z T

[0108] in, R x ,R y ,R z T represents the rotation matrix and position offset matrix around the X-axis, Y-axis, and Z-axis, respectively;

[0109]

[0110] Wherein, ω, θ, φ are the camera's roll angle, pitch angle, and azimuth angle attitude data, f is the focal length, (u0, v0) are the principal points, λ is the scale factor, and (X) is the focal length. s ,Y s Z s () represents the coordinates of the photography center.

[0111] As one embodiment of the present invention, the measurement module measures the distance Z from the panoramic camera to the ground. h This yields the true coordinates (X, Y) corresponding to any pixel (φ1, θ1), i.e.

[0112]

[0113] Where H' represents the real-time camera pose data ω′, θ′, The result is obtained by substituting H.

[0114] In one embodiment of the present invention, the measurement module obtains the true coordinates (X, Y) corresponding to any pixel (φ1, θ1) based on the distance L from the panoramic camera to the vertical wall of the side tunnel.

[0115]

[0116] Where H' represents the real-time camera pose data ω′, θ′, The result is obtained by substituting H.

[0117] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A measurement method for a tunnel interior planar mapping model based on panoramic imagery, characterized in that, Includes the following steps: Step S1: Acquire panoramic images from the camera; Step S2: Based on the panoramic image, obtain the tunnel interior planar mapping model, which includes: the mapping matrix of the tunnel interior facing the ground and the wall. Step S3: Based on the tunnel's internal planar mapping model, measure the points inside the tunnel on the real-time acquired panoramic image. Let the image angle coordinates of the point to be mapped be . Any pixel in the panoramic image is Then the image coordinates of any point within the mapping plane can be expressed as: The mapping point is selected as the point closest to the mapping surface or the center point of the area to be mapped and modeled. When the mapping surface is the ground, θ0 = -90°. Now, let the plane tangent to the sphere with a radius of 1 unit be the projection plane, and the coordinates of the projected plane image be (x, y). In this panoramic image, the coordinates of a pixel are (u, v), and the corresponding angular coordinates are... θ、 These are the elevation angle and the azimuth angle, respectively. When the mapped surface is a vertical wall in the direction of travel within the tunnel, formula (1) becomes as follows: Then the mapping plane of the vertical wall on the right side of the forward direction is θ0 = 0°. The mapping plane of the vertical wall on the left side of the forward direction is θ0 = 0°. The internal plane mapping model H of the tunnel is: H=λFR x R y R z T in, R x ,R y ,R z T represents the rotation matrix and position offset matrix around the X-axis, Y-axis, and Z-axis, respectively; Where ω is the camera roll angle, f is the focal length, (u0,c0) is the principal point, λ is the scale factor, and (X) is the focal length. s ,Y s Z s () represents the coordinates of the photography center; In step S3, any pixel is obtained based on the distance L from the panoramic camera to the vertical wall of the side tunnel. The corresponding true coordinates (X,Y), that is Where H' represents the real-time camera roll angle, pitch angle, and azimuth angle attitude data ω', θ', The result is obtained by substituting H.

2. A measurement method for a tunnel interior planar mapping model based on panoramic imagery, characterized in that, Includes the following steps: Step S1: Acquire panoramic images from the camera; Step S2: Based on the panoramic image, obtain the tunnel interior planar mapping model, which includes: the mapping matrix of the tunnel interior facing the ground and the wall. Step S3: Based on the tunnel's internal planar mapping model, measure the points inside the tunnel on the real-time acquired panoramic image. Let the image angle coordinates of the point to be mapped be . Any pixel in the panoramic image is Then the image coordinates of any point within the mapping plane can be expressed as: The mapping point is selected as the point closest to the mapping surface or the center point of the area to be mapped and modeled. When the mapping surface is the ground, θ0 = -90°. Now, let the plane tangent to the sphere with a radius of 1 unit be the projection plane, and the coordinates of the projected plane image be (x, y). In this panoramic image, the coordinates of a pixel are (u, v), and the corresponding angular coordinates are... θ、 These are the elevation angle and the azimuth angle, respectively. When the mapped surface is a vertical wall in the direction of travel within the tunnel, formula (1) becomes as follows: Then the mapping plane of the vertical wall on the right side of the forward direction is θ0 = 0°. The mapping plane of the vertical wall on the left side of the forward direction is θ0 = 0°. The internal plane mapping model H of the tunnel is: H=λFR x R y R z T in, R x ,R y ,R z T represents the rotation matrix and position offset matrix around the X-axis, Y-axis, and Z-axis, respectively; Where ω is the camera roll angle, f is the focal length, (u0, v0) is the principal point, λ is the scale factor, and (X) is the focal length. s ,Y s Z s () represents the coordinates of the photography center; In step S3, based on the distance Z from the panoramic camera to the ground... h To obtain any pixel The corresponding true coordinates (X,Y), that is Where H' represents the real-time camera roll angle, pitch angle, and azimuth angle attitude data ω', θ', The result is obtained by substituting H.

3. A measurement device for a tunnel planar mapping model based on panoramic images, which implements the measurement method for a tunnel planar mapping model based on panoramic images as described in claim 1 or 2, characterized in that, include: The acquisition module is used to acquire panoramic images from the camera; The module is used to obtain a planar mapping model inside the tunnel based on the panoramic image. The planar mapping model inside the tunnel includes a mapping matrix for the tunnel facing the ground and walls. The measurement module is used to measure points inside the tunnel on real-time panoramic images based on the tunnel's internal planar mapping model.

Images