Method for calculating area of plane structure based on panoramic map

Abstract

This invention belongs to the field of computer vision technology and discloses a method for calculating the area of ​​a planar structure based on a panoramic image. The method includes the following steps: Step 1: Obtain the panoramic image and camera height. The panoramic image is obtained by rotating the camera around a vertical axis. The vertically upward or downward view will be perpendicular to the ceiling and / or floor. The camera height is the distance from the ceiling or ground when capturing the panoramic image. Step 2: Convert the panoramic image into a partial view, which includes an upward view and / or downward view. Step 3: Calculate the length of the planar structure line and the area of ​​the planar structure based on the relative coordinates of key points. Key points include ceiling corner points and / or floor corner points obtained from the panoramic image. This method is based on the Manhattan world assumption, projects and converts the panoramic image, incorporates the camera's height from the ground, and calculates the length of the planar structure line and the area of ​​the planar structure. It has advantages such as low computational load and high accuracy, reducing the workload of surveyors.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a method for calculating the area of ​​a planar structure based on a panoramic image. Background Technology

[0002] In recent years, with the popularization of panoramic cameras, the development of deep learning, and its application in engineering fields, layout reconstruction based on panoramic images has become one of the hot research directions in computer vision. More and more research institutions at home and abroad are conducting research on this problem, such as Princeton University, the University of Illinois at Urbana-Champaign, and the CRS4 laboratory in Italy. These works have actively promoted the development of panoramic room layout reconstruction algorithms. Recently, significant progress has been made in estimating 3D room layouts using single panoramic images. For example, Zou et al. proposed LayoutNet, which for the first time used deep learning to directly reconstruct layouts on panoramic images. Subsequently, Sun et al. proposed HorizonNet, which simplified layout reconstruction from two-dimensional dense prediction to a one-dimensional sequence prediction task. These works construct the internal structure of houses based on panoramic images.

[0003] However, most of the existing methods involve semantic segmentation. Even with high-resolution panoramic images, there are still multiple comparisons during semantic segmentation, resulting in fuzzy boundary lines rather than regular building lines. Moreover, these methods for reconstructing indoor spatial layouts based on panoramic images are only suitable for reconstructing indoor spatial layouts and have drawbacks such as high computational cost, high training requirements, and recognition error. They also suffer from high computational cost and low accuracy in estimating the length of planar structural lines and the area of ​​planar structures. Summary of the Invention

[0004] The purpose of this invention is to provide a method for calculating the area of ​​a planar structure based on a panoramic image, which solves the technical problems of large computational load and low accuracy in the existing technology for calculating the length of planar structure lines and the area of ​​planar structures.

[0005] The method for calculating the area of ​​a planar structure based on a panoramic image includes the following steps:

[0006] Step 1: Obtain the panoramic image and camera height. The panoramic image is obtained by rotating the camera around the vertical axis. The vertical upward or vertical downward view will be perpendicular to the ceiling and / or floor. The height of the camera from the ceiling or ground when taking the panoramic image is the camera height.

[0007] Step 2: Convert the panoramic image into a partial view, which includes an upward view and / or a downward view.

[0008] Step 3: Calculate the length of the planar structure line and the area of ​​the planar structure based on the relative coordinates of the key points. The key points include the ceiling corner points and / or floor corner points obtained from the panoramic view.

[0009] Preferably, the partial view includes a bottom view and / or a top view. The specific steps for obtaining the partial view from the panoramic image in step two are as follows:

[0010] (1) The formula for converting the panoramic image coordinate system to the spherical polar coordinate system is shown below:

[0011]

[0012] (U, V) are the coordinates of A' in the panoramic coordinate system, (W1, H1) are the dimensions of the panoramic image, and the spherical polar coordinates of point A' are (θ, σ, r).

[0013] (2) The formula for converting spherical polar coordinates to spherical point coordinates is shown below:

[0014]

[0015] In the formula, (θ, σ, r) are the spherical polar coordinates of point A', and (x, y, z) are the spherical point coordinates of point A'.

[0016] (3) Place a plane tangent to the south or north pole of the sphere, and the projection onto this plane is the corresponding partial view.

[0017] (4) Set the field of view fov, set the size (w1, w2) of the local planar image to be acquired, and roughly calculate the focal length. The calculation formula is as follows:

[0018] f = (0.5 × W) / tan(fov × 0.5);

[0019] (5) Project the coordinates of the spherical point onto the local plan view. The coordinates (u, v) on the local view are transformed into (x, y, z) coordinates in the spherical coordinate system. The specific formula is as follows:

[0020]

[0021] In the formula, (u, v) are the coordinates of point A on the local planar graph, and (x, y, z) are the coordinates of point A' on the sphere.

[0022] The above steps are used to obtain a partial view and perform coordinate projection transformation based on the panoramic image.

[0023] Preferably, in step three, the ceiling corner points and / or floor corner points are obtained from the panoramic view, and the corresponding local floor plan is obtained by converting the panoramic view to a local view. The corresponding corner points are then connected clockwise or counterclockwise to obtain a floor plan of the house.

[0024] Preferably, in step three, the field of view (fov) is set, and an elevation view and / or a depression view are obtained. Based on the set field of view and the height of the camera from the ceiling and / or floor, the true size of the local plan view can be calculated, as shown in the following formula:

[0025] w1=w2=2×f×tanfov

[0026] In the formula, w1 and w2 are the actual dimensions of the local planar view space, f is the height of the camera from the ceiling and / or floor, and fov is the set field of view angle.

[0027] Preferably, in step three, the length of the planar structure line and the area of ​​the planar structure are calculated based on the relative coordinates of several key points, as shown in the following formula:

[0028]

[0029] In the formula, L represents the length of the planar structure line, S represents the area of ​​the planar structure, (u i v i ) represents the coordinates of the i-th key point of the planar structure, and N represents the total number of corner points in the planar structure.

[0030] Preferably, in step three, the house space layout is constructed along the house boundary direction. If two complete side lines can be constructed from the viewpoint, the average value of the two opposite sides is taken as the house size in that direction. If they cannot be constructed, the length of a single side is taken as the house size. Finally, the house size constructed from the ceiling and floor is averaged, and the corresponding house area is calculated based on the house size.

[0031] This invention has the following advantages: Based on the Manhattan world assumption that all walls are right angles to each other and perpendicular to the floor, the method constructs the house's floor plan layout based on visible floor or ceiling corners. By projecting and converting panoramic images and incorporating the camera's height above the ground, the length of the planar structural lines and the area of ​​the planar structure are calculated. It is applicable both indoors and outdoors, is simple to operate, and has advantages such as low computational load and high accuracy, thus reducing the workload of surveyors.

[0032] The boundary lines of the house's interior layout generated by this method are clear and distinct. Furthermore, the panoramic image used is obtained by rotating the image around the vertical axis. The vertically upward or downward view is perpendicular to the ceiling or floor, and objects on the ceiling and floor are not deformed. Combined with the assumption of Manhattan world for the four corner points of the house, it can better avoid the impact of image distortion and unnaturalness on the estimation of the interior floor plan structure. Attached Figure Description

[0033] Figure 1 This is a basic flowchart of a method for calculating the area of ​​a planar structure based on a panoramic image, according to the present invention.

[0034] Figure 2 This is a schematic diagram of point A' in the coordinate system of the panoramic image in this invention.

[0035] Figure 3 This is a schematic diagram illustrating the transformation of point A' from the panoramic coordinate system to spherical polar coordinates and then to spherical point coordinates in this invention.

[0036] Figure 4 This is a schematic diagram of point A' projected onto a local planar view from the coordinates of a spherical point in this invention.

[0037] Figure 5 This is a schematic diagram of the planar structure on the partial planar view obtained by the present invention.

[0038] Figure 6 This is a panoramic view of room 115 of the geospatial information system, and an upward and downward view obtained by converting the panoramic view into an upward view.

[0039] Figure 7 This is a panoramic view of room 115 in one embodiment of the present invention, showing the effect after selecting each corner point.

[0040] Figure 8 In one embodiment of the present invention, the floor plan of room 115 is estimated from the panoramic view according to the method of the present invention.

[0041] Figure 9 This is a panoramic view of room 119 of the geospatial information system, and an upward and downward view obtained by converting the panoramic view.

[0042] Figure 10 This is a panoramic view of room 119 in another embodiment of the present invention, showing the effect after selecting each corner point.

[0043] Figure 11 In another embodiment of the present invention, the floor plan of room 119 of the geospatial information system is estimated from the panoramic view according to the present method. Detailed Implementation

[0044] The following detailed description of the embodiments, with reference to the accompanying drawings, will further illustrate the specific implementation of the present invention, in order to help those skilled in the art to have a more complete, accurate, and in-depth understanding of the inventive concept and technical solution of the present invention.

[0045] like Figure 1-11As shown, this invention provides a method for calculating the area of ​​a planar structure based on a panoramic image, comprising the following steps:

[0046] Step 1: Obtain the panoramic image and camera height.

[0047] A panoramic view refers to a view from a fixed observation point that provides a free 360-degree horizontal azimuth and a 180-degree vertical perspective (simplified panoramic views only provide a 360-degree horizontal perspective). Currently, panoramic cameras on the market are mainly divided into two types: fisheye panoramic cameras and multi-lens panoramic cameras. A fisheye panoramic camera uses a single sensor paired with a special ultra-wide-angle fisheye lens and relies on image correction technology to reproduce the image. Even after correction, the panoramic image generated by a fisheye panoramic camera still has a certain degree of distortion and unnaturalness. The panoramic image obtained by this solution is captured by rotating around a vertical axis; vertically upward or downward viewing angles will be perpendicular to the ceiling and / or floor.

[0048] When a camera takes a panoramic image, the height above the ceiling or ground is the camera height. The camera height, as the radius of the spherical projection, is an important parameter when calculating the overhead and downward angles.

[0049] Step 2: Convert the panoramic image into an upward-viewing image and / or a downward-viewing image.

[0050] At vertically upward or downward angles, the viewpoint is perpendicular to the ceiling or floor. Objects on the ceiling and floor do not deform at these angles. Therefore, by obtaining a plan view from a panoramic perspective (either upward or downward), the spatial layout of a house can be estimated. For example... Figure 2-4 The specific steps for obtaining a panoramic view from an upward angle are as follows.

[0051] (1) The formula for converting the panoramic image coordinate system to the spherical polar coordinate system is shown below:

[0052]

[0053] (U, V) are the coordinates of A' in the panoramic coordinate system, (W1, H1) are the dimensions of the panoramic image, and the spherical polar coordinates of point A' are (θ, σ, r).

[0054] (2) The formula for converting spherical polar coordinates to spherical point coordinates is shown below:

[0055]

[0056] In the formula, (θ, σ, r) are the spherical polar coordinates of point A', and (x, y, z) are the spherical point coordinates of point A'.

[0057] (3) Place a plane tangent to the south or north pole of the sphere. The projection of this plane is the upward or downward view.

[0058] (4) Set the field of view fov, set the size (w1, w2) of the local planar image to be acquired, and roughly calculate the focal length. The calculation formula is as follows:

[0059] f = (0.5 × W) / tan(fov × 0.5).

[0060] (5) Project the coordinates of the spherical point onto the local plan view. The coordinates (u, v) on the local view (i.e., the upward or downward view) are transformed into (x, y, z) coordinates in the spherical coordinate system. The specific formula is as follows:

[0061]

[0062] In the formula, (u, v) are the coordinates of point A on the local planar graph, and (x, y, z) are the coordinates of point A' on the sphere.

[0063] The above steps are used to obtain an overhead (or downward) view and coordinate projection transformation based on the panoramic image.

[0064] Step 3: Calculate the length of the planar structural line and the area of ​​the planar structure.

[0065] Obtain ceiling and / or floor corner points (i.e., key points) from the panoramic image. Convert the panoramic image to an upward (or downward) view to obtain an upward (or downward) view plan. Connect the house corner points clockwise or counterclockwise to obtain the house floor plan. Based on the formulas for converting the panoramic image to the upward and / or downward view, and given the camera height or distance from the camera to the ceiling, and using the set field of view, the room layout dimensions can be calculated. Because the panoramic image in this method is acquired by rotating the camera along the vertical axis, the upward and downward views do not involve objects attached to the ceiling or floor, allowing for the selection of key points to construct the interior of the house.

[0066] If a viewpoint can construct two complete sidelines, the average of the two opposite sides is taken as the house size in that direction. If it cannot be constructed, the length of a single side is taken as the house size. Finally, the average of the house size constructed from the ceiling and floor is taken, and the house area is calculated based on the obtained house size. Since the house satisfies the Manhattan world assumption, the house spatial layout is finally constructed along the house boundary direction.

[0067] In a specific embodiment, based on the method of converting a panoramic view to an elevation view and / or a top view, the field of view (fov) is set to 160° to obtain the elevation view and / or top view. Based on the set field of view and the height of the camera from the ceiling and / or floor, the true size of the local plan view can be calculated, as shown in the following formula:

[0068] w1=w2=2×f×tanfov

[0069] In the formula, w1 and w2 are the actual dimensions of the local planar view space, f is the height of the camera from the ground (or ceiling), and fov is the set field of view angle. Based on the relative coordinates of several key points, the length of the planar structure line and the area of ​​the planar structure are calculated, as shown in the following formula:

[0070]

[0071] In the formula, L represents the length of the planar structure line, S represents the area of ​​the planar structure, (u i v i ) represents the coordinates of the i-th key point of the planar structure, and N represents the total number of corner points in the planar structure.

[0072] On the panoramic image, select the corner points from left to right, such as... Figure 6 As shown, the corner points of the ceiling include A(u1, v1), B(u2, v2), C(u3, v3), and D(u4, v4), with N being 4. During the selection process, only unobstructed corner points are chosen, as these points are located on the ceiling or floor. Therefore, there is no positional deviation when obtaining the upward or downward view. By connecting the points clockwise or from left to right along the selected points in the panoramic view, the spatial layout of the house can be obtained. The house dimensions can then be calculated from the dimensions of the panoramic image and the distance between the camera and the ceiling or floor.

[0073] In a specific embodiment, panoramic images of two rooms, numbered GIS 115 and GIS 119 respectively, were obtained for the experiment, such as... Figure 6-11 As shown, based on the above method, the estimated length of room 119 is approximately 12.66m, the width is approximately 5.65m, and the area is 71.54m². 2 Room 115 of the Geospatial Information Center is 6.47m long and wide, with a room area of ​​41.80m². 2 The results are shown in Table 1.

[0074] Table 1: Dimensions of Planar Space Layout Obtained from Panoramic Images

[0075]

[0076] The present invention has been described above by way of example with reference to the accompanying drawings. Obviously, the specific implementation of the present invention is not limited to the above-described manner. Any non-substantial improvements made using the inventive concept and technical solution of the present invention, or the direct application of the inventive concept and technical solution of the present invention to other occasions without modification, are all within the protection scope of the present invention.

Claims

1. A method for calculating the area of ​​a planar structure based on panoramic images, characterized in that: Includes the following steps: Step 1: Obtain the panoramic image and camera height. The panoramic image is obtained by rotating the camera around the vertical axis. The vertical upward or vertical downward view will be perpendicular to the ceiling and / or floor. The height of the camera from the ceiling or ground when taking the panoramic image is the camera height. Step 2: Convert the panoramic image into a partial view, which includes an upward view and / or a downward view. Step 3: Calculate the length of the planar structure line and the area of ​​the planar structure based on the relative coordinates of the key points. The key points include the ceiling corner points and / or floor corner points obtained from the panoramic view. The partial view includes a bottom view and / or a top view. The specific steps for obtaining the partial view from the panoramic image in step two are as follows: (1) Transform the panoramic image coordinate system to the spherical polar coordinate system using the following formula: ( U , V )yes A’ The coordinates on the panoramic image coordinate system, ( W 1, H 1) is the size of the panoramic image, points A’ The spherical polar coordinates are ( θ, σ, r ); (2) The formula for converting spherical polar coordinates to spherical point coordinates is as follows: ； In the formula ( θ, σ, r ) is a point A’ spherical polar coordinates, ( x , y , z ) is a point A’ The coordinates of a point on the sphere; (3) Place a plane tangent to the south or north pole of the sphere, and the projection onto the plane is the corresponding partial view. (4) Set the field of view fov Set the size of the local planar view to be acquired. w 1, w 2) Roughly calculate the focal length using the following formula: ； (5) Project the coordinates of the spherical point onto the partial plan view, and the coordinates on the partial view ( u , v ), transformed to spherical coordinates as ( x , y , z The specific formula is as follows: ； In the formula ( u, v ) is a point A The coordinates of the point on the partial plan view, ( x, y, z ) is a point A’ The coordinates of a point on the sphere; The above steps are used to obtain a partial view and perform coordinate projection transformation based on the panoramic image.

2. The method for calculating the area of ​​a planar structure based on a panoramic image according to claim 1, characterized in that: In step three, ceiling corner points and / or floor corner points are obtained from the panoramic view. By converting the panoramic view to a partial view, the corresponding partial floor plan is obtained. The corresponding corner points are connected clockwise or counterclockwise to obtain the house floor plan.

3. The method for calculating the area of ​​a planar structure based on a panoramic image according to claim 2, characterized in that: In step three, the field of view is set. fov By acquiring the elevation view and / or depression view, and based on the set field of view and the camera's height from the ceiling and / or floor, the true dimensions of the partial plan view can be calculated, as shown in the following formula: In the formula w 1 and w 2 represents the actual dimensions of the partial planar view space. f It is the height of the camera from the ceiling and / or floor. fov The set field of view.

4. The method for calculating the area of ​​a planar structure based on a panoramic image according to claim 3, characterized in that: In step three, the length of the planar structure line and the area of ​​the planar structure are calculated based on the relative coordinates of several key points, as shown in the following formula: In the formula, L represents the length of the planar structure line. S Represents the area of ​​a planar structure, ( u i , v i ) represents the first planar structure i The coordinates of the key points N This represents the total number of corner points in a planar structure.

5. The method for calculating the area of ​​a planar structure based on a panoramic image according to claim 4, characterized in that: In step three, the spatial layout of the house is constructed along the boundary of the house. If two complete side lines can be constructed from the viewpoint, the average value of the two opposite sides is taken as the house size in that direction. If they cannot be constructed, the length of a single side is taken as the house size. Finally, the average of the house size constructed from the ceiling and the floor is taken, and the corresponding house area is calculated based on the house size.

Images