Plane detection method

Abstract

Provided in the present invention is a plane detection algorithm, the plane detection algorithm comprising a data collection step, such that an XR device can capture observation points in a plurality of environmental images. The present invention generates a Delaunay triangulation from coordinates of the observation points by means of a triangulation construction step, such that a real-world plane can be converted into a plurality of triangular facets, and if included angles between any plurality of adjacent triangular facets are sufficiently small, there is sufficient confidence to determine that the plurality of triangular facets are in the same plane, that is, the observation points forming the plurality triangular facets are all located on the same plane, so as to achieve plane positioning. Finally, the method uses the coordinates of the plurality of observation points to determine plane characteristic parameters, so as to complete plane detection and positioning.

Description

A plane detection method Technical Field

[0001] The present invention relates to the field of XR technology, and in particular to a plane detection method and data processing equipment. Background Art

[0002] Extended Reality (XR) refers to the use of computers to combine the real and virtual worlds to create a virtual environment where humans and machines can interact. It is a general term for various technologies, including AR, VR, and MR. By integrating the visual interaction technologies of these three, it provides users with an "immersive" experience, with a seamless transition between the virtual and real worlds. XR encompasses a variety of technologies, including virtual reality (VR), augmented reality (AR), and mixed reality (MR), as well as various wearable devices, gesture recognition, voice recognition, and other interaction methods. By simulating and enhancing the experience of multiple senses, XR creates an immersive interactive environment for users.

[0003] In XR devices, visual SLAM (Simultaneous Localization and Mapping) is the primary 6DoF technology they rely on. This involves creating a map based on camera images in a completely unknown environment, even when the device's own position is uncertain, and using the map for autonomous positioning. Furthermore, to provide an immersive mixed reality experience, XR devices often need to identify and stably track planes in real space, then render them and overlay them on top of the real planes, or implicitly utilize them (for example, by placing other virtual objects on the planes). Summary of the Invention

[0004] The present invention provides a plane detection method to solve the technical problem that the existing XR device cannot detect the plane existing in the real space.

[0005] In order to solve the above problems, the present invention provides a plane detection method, which specifically includes the following steps: a data acquisition step, establishing a world coordinate system W, capturing an environmental image through an XR device, obtaining the position TBW of the XR device in the world coordinate system for any frame of the captured environmental image, and establishing a local coordinate system B with any point on the XR device as the origin; obtaining all observation points in the environmental image, and the coordinates of the observation points in the world coordinate system are The observation point is specifically a point on the surface of an object in the real space that can be captured by the environmental image; the coordinate transformation step is to obtain the coordinates of the observation point in the data acquisition step in the world coordinate system as Calculate the coordinates of the observation point in the local coordinate system B and get And calculate the coordinates of the observation point on the spherical coordinate system S, the origin of the spherical coordinate system S is the same as the origin of the local coordinate system B, and get For the coordinates of the observation point on the spherical coordinate system S Among them Let its coordinate r = 1, so that the observation point is projected onto the unit sphere, and we get {(θ0, φ0), (θ1, φ1), ..., (θ n ,φ n )}; triangulation construction step, through the coordinate transformation step obtained {(θ0, φ0), (θ1, φ1), ..., (θ n ,φ n )} Generate a Delaunay triangulation; in the process of generating the Delaunay triangulation, if the distance between the two vertices of a triangle in the local coordinate system B is greater than a preset threshold, it is considered that the triangle does not belong to any plane in the real space, and thus the triangle is deleted; a normal vector calculation step, obtaining the Delaunay triangulation obtained in the triangulation construction step, and calculating the unit normal vector of each triangle in the Delaunay triangulation The calculation formula is as follows:

[0006] in, as well as are the three vertices of a triangle; for each two opposite directions calculated Select the origin of the local coordinate system B Discard the direction away from the origin of the local coordinate system B Normal vector clustering step, for each triangle face unit normal vector And the unit normal vector of any triangle adjacent to this triangle like and If the angle between The corresponding triangles and The corresponding triangular faces are classified into one category; in the plane calculation step, the triangular faces of multiple categories obtained by the normal vector clustering step are obtained, and it is determined whether the number of triangular faces in each category is greater than a preset threshold β; if so, the parameters of the plane corresponding to the category are calculated through the multiple observation points constituting each triangular face in the category; if not, the category is ignored.

[0007] Furthermore, in the coordinate transformation step, the calculation formula for calculating the coordinates of the observation point in the local coordinate system B is as follows:

[0008] in, is the coordinate of the i-th observation point in the local coordinate system B, is the coordinate of the i-th observation point in the world coordinate system W; TBW is a coordinate transformation matrix that can convert the coordinates of any point in the world coordinate system W into the coordinates in the local coordinate system B; in the coordinate transformation step, the calculation formula for the coordinate step of the observation point in the spherical coordinate system S is as follows:

[0009] Among them, x, y and z are the specific values ​​of the coordinates of the observation point in the local coordinate system B, and r, θ and φ are the specific values ​​of the coordinates of the observation point in the spherical coordinate system S.

[0010] Furthermore, if all observation points in the data acquisition step can be projected onto the display surface of the same camera, the coordinate transformation step is not performed, and the two-dimensional pixel coordinates of all observation points are used to generate a Delaunay triangulation in the triangulation construction step.

[0011] Furthermore, in the normal vector calculation step, the origin of the local coordinate system B is selected Discard the direction away from the origin of the local coordinate system B This step specifically includes the following steps: for each two opposite directions obtained In the world coordinate system W, calculate The corresponding point on the triangle As the starting point, with the origin of the local coordinate system B The vector of the end point judge and Is the angle less than 90 degrees? If so, select the If not, discard the

[0012] Furthermore, in the plane calculation step, the step of calculating the parameters of the plane corresponding to a class through multiple observation points constituting each triangular face in a class specifically includes the following steps: a plane equation calculation step, the general calculation formula of the plane equation is as follows: ax+by+cz+d=0(c≠0)

[0013] The formula can be transformed into z=a0x+a1y+a2

[0014] in, For the coordinates of k observation points in this class (x1, y1, z1), (x2, y2, z2), ..., (x k ,yk ,z k ), construct the least squares matrix form Ax = b, that is

[0015] Solving (a0, a1, a2) yields the plane equation parameters of the plane.

[0016] Furthermore, the plane calculation step further includes the following steps: a plane normal vector calculation step, obtaining (a0, a1, a2) in the plane equation calculation step, through the formula:

[0017] Calculate the normal vector of the plane as (a, b, c);

[0018] Furthermore, the plane calculation step also includes the following steps: a plane coordinate system calculation step, establishing a local coordinate system of the plane, making the z-axis of the local coordinate system of the plane and the direction of the plane normal vector the same, and making the x-axis, y-axis and z-axis of the local coordinate system of the plane orthogonal; calculating the projection position of the center point of the k observation points in this class on the plane, and making the projection position the origin of the local coordinate system of the plane; calculating the posture TPW of the local coordinate system of the plane in the world coordinate system W.

[0019] Furthermore, the plane calculation step further includes the following steps: a plane boundary contour calculation step, calculating the projection positions of k observation points in the class on the plane, and obtaining the coordinates of the k projection positions (x′1, y′1), (x′2, y′2), ..., (x′ k ,y′ k ); Calculate the convex hull contours {Q0, Q1…, Qm} of the k projection positions as the boundary contour of the plane.

[0020] Furthermore, after the triangulation network construction step and before the normal vector calculation step, it also includes: a plane expansion step, if there is at least one detected plane in the memory, for each observation point, determine whether the observation point belongs to any plane already in the memory; determine whether there is a plane such that the distance between the observation point and the plane is less than a preset threshold γ, and determine whether there is an observation point in the plane such that the distance between the observation point and the observation point is less than a preset threshold δ; if both are yes, merge the observation point into the plane, and delete the observation point and the triangular face formed by the observation point in the Delaunay triangulation network; if any one of the judgment conditions is no, the observation point is not considered to belong to any plane and is retained.

[0021] Furthermore, after the plane calculation step, it also includes: a plane fusion step, retrieving all observation points, if there is any observation point and the adjacent observation point of the observation point belonging to two planes respectively, then judging whether the two planes are adjacent; judging whether the angle between the normal vector of the first plane and the normal vector of the second plane is less than a preset threshold λ, and judging whether more than half of the observation points in the first plane are less than a preset threshold τ from the second plane, and judging whether more than half of the observation points in the second plane are less than a preset threshold τ from the first plane; if all are yes, then all observation points of the second plane are merged into the first plane, and the second plane is deleted; or all observation points of the first plane are merged into the second plane, and the first plane is deleted; if any one of the judgment conditions is no, then the first plane and the second plane are not considered to be the same plane.

[0022] The present invention further includes a data processing device comprising a memory for storing executable program code. The data processing device further comprises a processor for reading the executable program code to execute a computer program corresponding to the executable program code to perform at least one step of the above-mentioned plane detection method.

[0023] The advantage of the present invention is that the present invention converts the coordinates of the observation points into a spherical coordinate system, and then replaces the pixel coordinates of the plane of the XR camera with the angular coordinates. This allows the observation points recorded by multiple cameras with different orientations to be projected onto the same camera plane. The Delaunay triangulation is then generated using the coordinates of these observation points. The excellent properties of the Delaunay triangulation are the empty circle property and the maximized minimum angle property, which avoid the generation of narrow triangles. This allows the plane in reality to be converted into multiple triangular faces. If the angle between any multiple adjacent triangular faces is small enough, there is enough confidence to believe that these multiple triangular faces are the same plane, that is, it is believed that the observation points that make up these multiple triangular faces are all located on the same plane, thereby achieving the positioning of the plane. Finally, the coordinates of multiple observation points are used to determine the plane feature parameters, and the plane detection and positioning are completed. The present invention can not only detect the newly appeared plane in the current image, but also continuously update the old planes detected in the past using the currently newly added observation points. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] FIG1 is a flow chart of a plane detection method according to an embodiment of the present invention;

[0025] FIG2 is a flow chart of the plane calculation steps in an embodiment of the present invention. Specific embodiments

[0026] The following describes preferred embodiments of the present invention with reference to the accompanying drawings to illustrate that the present invention can be implemented. These embodiments will fully explain the technical content of the present invention to those skilled in the art, making the technical content of the present invention clearer and easier to understand. However, the present invention can be embodied in many different forms of embodiments, and the scope of protection of the present invention is not limited to the embodiments described herein.

[0027] This embodiment provides an XR device, which includes a data processing device, on which a memory and a processor are installed. The memory is used to store executable program code. The processor is used to read the executable program code to run a computer program corresponding to the executable program code to perform at least one step in a plane detection method. The XR device also includes multiple cameras, which are electrically connected to the processor. The cameras face straight ahead, straight down, and / or diagonally downward. The environmental image can be displayed within its field of view.

[0028] Based on the XR device described above, this embodiment also provides a plane detection method, which is implemented by the XR device in the form of software.

[0029] As shown in FIG1 , this embodiment provides a plane detection method, which specifically includes steps S1 to S8 .

[0030] Step S1: Data collection step, establish a world coordinate system W, collect environmental images through an XR device, and obtain the position T of the XR device in the world coordinate system for any frame of the collected environmental image BW And establish a local coordinate system B with any point on the XR device as the origin. Get all the observation points in the environment image, and the coordinates of the observation points in the world coordinate system are The observation point is specifically a point on the surface of an object in the real space that can be captured by the environment image.

[0031] The world coordinate system used in this embodiment is a 6DoF coordinate system based on SLAM technology. The world coordinate system is established so that the direction of gravity in the real world is consistent with the positive direction of the Z axis in the world coordinate system. The X-axis and Y-axis are both located on a horizontal plane, and any two of the X-axis, Y-axis, and Z-axis are perpendicular to each other.

[0032] The present invention converts the position of the observation point into coordinates in the world coordinate system and the local coordinate system of the XR device, so that the observation point can be recorded by the XR system and can be reused.

[0033] Step S2: coordinate transformation step, obtaining the coordinates of the observation point in the data acquisition step in the world coordinate system as Calculate the coordinates of the observation point in the local coordinate system B and get

[0034] The calculation formula for calculating the coordinates of the observation point in the local coordinate system B is as follows:

[0035] in, is the coordinate of the i-th observation point in the local coordinate system B, is the coordinate of the i-th observation point in the world coordinate system W. BW is a coordinate transformation matrix that can transform the coordinates of any point in the world coordinate system W into the coordinates in the local coordinate system B.

[0036] Then calculate the coordinates of the observation point in the spherical coordinate system S, where the origin of the spherical coordinate system S is the same as the origin of the local coordinate system B, and get The calculation formula for calculating the coordinates of the observation point on the spherical coordinate system S is as follows:

[0037] Among them, x, y and z are the specific values ​​of the coordinates of the observation point in the local coordinate system B, and r, θ and φ are the specific values ​​of the coordinates of the observation point in the spherical coordinate system S.

[0038] For the coordinates of the observation point on the spherical coordinate system S Among them Let its coordinate r = 1, so that the observation point is projected onto the unit sphere, and we get {(θ0, φ0), (θ1, φ1), ..., (θ n ,φ n )}.

[0039] Step S3: triangulation network construction step, through the coordinate transformation step obtained {(θ0, φ0), (θ1, φ1), ..., (θ n ,φ n )} Generate a Delaunay triangulation; in the process of generating the Delaunay triangulation, if the distance between two vertices of a triangle in the local coordinate system B is greater than a preset threshold, it is considered that the triangle does not belong to any plane in the real space, and the triangle is deleted;

[0040] In other embodiments, if all observation points in the data acquisition step can be projected onto the display surface of the same camera, the coordinate transformation step is not performed, and the two-dimensional pixel coordinates of all observation points are used to generate a Delaunay triangulation in the triangulation construction step.

[0041] The triangulated mesh calculation results provide two pieces of information: the first is the vertex structure of each triangle; the second is the neighboring points of each vertex. The first point can be directly obtained from the output index position; since the three vertices of a triangle are adjacent to each other, the second point can be obtained by traversing each vertex of each triangle, recording the other two vertices as neighbors, and then traversing all triangles.

[0042] The coordinates of the observation points are converted into a spherical coordinate system, and then the pixel coordinates of the plane of the XR camera are replaced by the angular coordinates. This allows the observation points recorded by multiple cameras in different orientations to be projected onto the same camera plane. The Delaunay triangulation is then generated using the coordinates of these observation points. The Delaunay triangulation can connect a set of discrete point sets into a non-overlapping triangular mesh. Delaunay triangulation is an optimized triangulation method with the property of maximizing the minimum angle or minimizing the maximum side length. This makes the resulting triangular mesh more uniform and regular, avoiding overly sharp or uneven triangles. At the same time, the Delaunay triangulation also performs well when processing large-scale data sets. The outstanding features of the Delaunay triangulation are the empty circle feature and the maximization of the minimum angle feature. These two features avoid the generation of narrow triangles. This allows the plane in reality to be converted into multiple triangular faces. If the angle between any number of adjacent triangular faces is small enough, there is enough confidence to believe that these multiple triangular faces are the same plane, that is, it is believed that the observation points that make up these multiple triangular faces are all located on the same plane, thus achieving the positioning of the plane.

[0043] Step S4: Plane expansion step. If there is at least one detected plane in the memory, for each observation point, determine whether the observation point belongs to any plane already in the memory; determine whether there is a plane such that the distance between the observation point and the plane is less than a preset threshold γ, and determine whether there is an observation point in the plane such that the distance between the observation point and the observation point is less than a preset threshold δ; if both are yes, merge the observation point into the plane, and delete the observation point and the triangle formed by the observation point in the Delaunay triangulation; if any one of the judgment conditions is no, the observation point is not considered to belong to any plane and is retained.

[0044] Step S5: Normal vector calculation step, obtain the Delaunay triangulation obtained in the triangulation construction step, and calculate the unit normal vector of each triangle in the Delaunay triangulation The calculation formula is as follows:

[0045] in, as well as For each of the three vertices of a triangle, Select the origin of the local coordinate system B Discard the direction away from the origin of the local coordinate system B

[0046] Select the origin of the local coordinate system B Discard the direction away from the origin of the local coordinate system B This step specifically includes the following steps: for each two opposite directions obtained In the world coordinate system W, calculate The corresponding point on the triangle As the starting point, with the origin of the local coordinate system B The vector of the end point judge and Is the angle less than 90 degrees? If so, select If not, discard the That is, the plane's normal vector should roughly point in the direction of the device.

[0047] Step S6: Normal vector clustering step, for each triangle face’s unit normal vector And the unit normal vector of any triangle adjacent to this triangle like and If the angle between The corresponding triangles and The corresponding triangles are classified into one category.

[0048] In this embodiment of the present invention, a breadth-first algorithm is specifically used to calculate the angle between the normal vectors of each current triangle and its adjacent triangles. If a condition is met, the triangles are classified into the same category. A breadth-first search starts from a certain point and continues to search through all its adjacent points. Then, starting from a selected adjacent point, the search continues through all adjacent untraversed points, and this process is repeated until all nodes are reached. Specifically, all adjacent triangles of a triangle are first found, and then all adjacent triangles of each adjacent triangle are found in turn, and this process is repeated until all nodes are reached.

[0049] In other embodiments of the present invention, a depth-first algorithm may also be used. The depth-first algorithm first searches for an adjacent triangle of a triangle, then searches for an adjacent triangle of the adjacent triangle, and repeats this process until no adjacent triangles are found.

[0050] Step S7: Plane calculation. The process obtains the multiple categories of triangular facets derived from the normal vector clustering step and determines whether the number of triangular facets in each category is greater than a preset threshold β. If so, the parameters of the plane corresponding to that category are calculated using the multiple observation points that constitute each triangular facet in that category. If not, that category is ignored.

[0051] As shown in FIG2 , in the plane calculation step, the step of calculating the parameters of the plane corresponding to a class through multiple observation points constituting each triangular face in the class specifically includes steps S71 to S74 .

[0052] Step S71: Plane equation calculation step. The general calculation formula of the plane equation is as follows: ax+by+cz+d=0 (c≠0)

[0053] The formula can be transformed into z=a0x+a1y+a2

[0054] in, For the coordinates of k observation points in this class (x1, y1, z1), (x2, y2, z2), ..., (x k ,y k ,z k ), construct the least squares matrix form Ax = b, that is

[0055] Solving (a0, a1, a2) yields the plane equation parameters of the plane.

[0056] In other embodiments, the plane equation can also be calculated using the concept of RANSAC. That is, each sampling process randomly selects three observation points from the k observation points in the class to calculate a plane equation parameter. The sum of the distances from all other points to this plane is then used as the cost function. After multiple sampling processes, the combination with the minimum cost function is selected as the final plane equation parameter.

[0057] Step S72: Plane normal vector calculation step, obtaining (a0, a1, a2) in the plane equation calculation step, by formula:

[0058] The normal vector of the plane is calculated as (a, b, c).

[0059] Step S73: Plane coordinate system calculation step, establish the local coordinate system of the plane, make the z axis of the local coordinate system of the plane and the plane normal vector have the same direction, and make the x axis, y axis and z axis of the local coordinate system of the plane orthogonal. Calculate the projection position of the center point of the k observation points in this class on the plane, and let the projection position be the origin of the local coordinate system of the plane. Calculate the pose T of the local coordinate system of the plane in the world coordinate system W PW .

[0060] Step S74: Plane boundary contour calculation step, calculate the projection positions of k observation points in the class on the plane, and obtain the coordinates of k projection positions (x′1, y′1), (x′2, y′2), ..., (x′ k ,y′ k ). Calculate the convex hull contours {Q0, Q1..., Qm} of the k projection positions as the boundary contour of the plane.

[0061] The convex hull contour algorithm is used to calculate the convex hull of a given set of points, that is, the smallest convex polygon that encloses the point set. There are many algorithms that can achieve convex hull contours, the most commonly used of which are the Graham scan algorithm and the Jarvis case algorithm.

[0062] The Graham scan algorithm follows these steps: Select the lowest point as the starting point, P0, and sort the remaining points by polar angle from smallest to largest. Add P0 and the first sorted point, P1, to a stack. For each subsequent point, Pi (starting with P2), if the direction of the vector formed by Pi and the two top points on the stack is counterclockwise (left-turning), add Pi to the stack. Otherwise, repeatedly pop the top point of the stack until the resulting vector is counterclockwise, then add Pi to the stack. The points in the stack are the calculated convex hull point set. For the Jarvis case algorithm, the steps are as follows: Select the leftmost point as the starting point, P0. For each subsequent point, Pi (starting with P1), initialize the current point to Pi and add it to the convex hull point set. For each remaining point, Pj, if Pj is further from the line segment on the convex hull than the current point, Pi, set Pj as the current point. If the current point returns to the starting point, P0, the algorithm ends. Return the convex hull point set. Both algorithms are based on the idea of ​​iteration and comparison. By selecting a suitable starting point and iterating according to the position relationship of the points, the convex hull point set is finally obtained.

[0063] Step S8: Plane fusion step, retrieve all observation points, if there is any observation point and the adjacent observation point of the observation point belonging to two planes respectively, then determine whether the two planes are adjacent; determine whether the angle between the normal vector of the first plane and the normal vector of the second plane is less than the preset threshold λ, and determine whether more than half of the observation points in the first plane are less than the preset threshold τ from the second plane, and determine whether more than half of the observation points in the second plane are less than the preset threshold τ from the first plane; if all are yes, then merge all observation points of the second plane into the first plane and delete the second plane; or merge all observation points of the first plane into the second plane and delete the first plane; if any one of the judgment conditions is no, then the first plane and the second plane are not considered to be the same plane.

[0064] In actual scenarios, when a user uses an XR device, multiple cameras of the XR device can capture the environment image. For any frame of the captured environment image, obtain the position T of the XR device in the world coordinate system. BW , as well as all observation points in the environment image. Using the coordinates of the observation points in the coordinate system, a Delaunay triangulation can be formed, which allows the plane in reality to be converted into multiple triangular faces. If the angle between any number of adjacent triangular faces is small enough, there is enough confidence to believe that these multiple triangular faces are the same plane, that is, the observation points that make up these multiple triangular faces are all located on the same plane, thus achieving plane detection and positioning. Finally, the calculation of the plane parameters still depends on the coordinates of the observation points to obtain the plane parameters, which are saved in the memory for easy later call or re-verification.

[0065] The advantage of the present invention is that the present invention converts the coordinates of the observation points into a spherical coordinate system, and then replaces the pixel coordinates of the plane of the XR camera with the angular coordinates. This allows the observation points recorded by multiple cameras with different orientations to be projected onto the same camera plane. The Delaunay triangulation is then generated using the coordinates of these observation points. The excellent properties of the Delaunay triangulation are the empty circle property and the maximized minimum angle property, which avoid the generation of narrow triangles. This allows the plane in reality to be converted into multiple triangular faces. If the angle between any multiple adjacent triangular faces is small enough, there is enough confidence to believe that these multiple triangular faces are the same plane, that is, it is believed that the observation points that make up these multiple triangular faces are all located on the same plane, thereby achieving the positioning of the plane. Finally, the coordinates of multiple observation points are used to determine the plane feature parameters, and the plane detection and positioning are completed. The present invention can not only detect the newly appeared plane in the current image, but also continuously update the old planes detected in the past using the currently newly added observation points.

[0066] The above describes in detail the preferred embodiments of the present invention. It should be understood that those skilled in the art can make numerous modifications and variations based on the concepts of the present invention without inventive effort. Therefore, any technical solutions that can be derived by those skilled in the art through logical analysis, reasoning, or limited experimentation based on the concepts of the present invention and the prior art should be within the scope of protection defined by the claims.

Claims

1. A plane detection method, characterized in that: The steps include: Data collection step: establish a world coordinate system W, collect environmental images through an XR device, and obtain the position T of the XR device in the world coordinate system for any frame of environmental image collected. BW And establish a local coordinate system B with any point on the XR device as the origin; obtain all observation points in the environment image, the coordinates of the observation points in the world coordinate system are The observation point is specifically a point on the surface of an object in real space that can be captured by the environment image; The coordinate transformation step is to obtain the coordinates of the observation point in the data acquisition step in the world coordinate system as Calculate the coordinates of the observation point in the local coordinate system B and get And calculate the coordinates of the observation point on the spherical coordinate system S, the origin of the spherical coordinate system S is the same as the origin of the local coordinate system B, and get For the coordinates of the observation point on the spherical coordinate system S Among them Let its coordinate r = 1, so that the observation point is projected onto the unit sphere, and we get {(θ0, φ0), (θ1, φ1), ..., (θ n ,φ n )}; The triangulation network construction step is to obtain {(θ0, φ0), (θ1, φ1), ..., (θ n ,φ n )} Generate a Delaunay triangulation; in the process of generating the Delaunay triangulation, if the distance between two vertices of a triangle in the local coordinate system B is greater than a preset threshold, it is considered that the triangle does not belong to any plane in the real space, and the triangle is deleted; Normal vector calculation step, obtain the Delaunay triangulation obtained in the triangulation construction step, and calculate the unit normal vector of each triangle in the Delaunay triangulation The calculation formula is as follows: in, as well as are the three vertices of a triangle; For each two opposite directions Select the origin of the local coordinate system B Discard the direction away from the origin of the local coordinate system B Normal vector clustering step, for each triangle face unit normal vector And the unit normal vector of any triangle adjacent to this triangle like and If the angle between The corresponding triangles and The corresponding triangles are classified into one category; The plane calculation step obtains multiple categories of triangular faces obtained in the normal vector clustering step, and determines whether the number of triangular faces in each category is greater than a preset threshold β; if so, the parameters of the plane corresponding to the category are calculated through multiple observation points constituting each triangular face in the category; if not, the category is ignored.

2. The plane detection method according to claim 1, wherein: In the coordinate transformation step, the calculation formula for calculating the coordinates of the observation point in the local coordinate system B is as follows: in, is the coordinate of the i-th observation point in the local coordinate system B, is the coordinate of the i-th observation point in the world coordinate system W; T BW is a coordinate transformation matrix that can transform the coordinates of any point in the world coordinate system W into the coordinates of the local coordinate system B; In the coordinate transformation step, the calculation formula for calculating the coordinates of the observation point on the spherical coordinate system S is as follows: Among them, x, y and z are the specific values ​​of the coordinates of the observation point in the local coordinate system B, and r, θ and φ are the specific values ​​of the coordinates of the observation point in the spherical coordinate system S.

3. The plane detection method according to claim 1, wherein: If all observation points in the data acquisition step can be projected onto the display surface of the same camera, the coordinate transformation step is not performed, and the two-dimensional pixel coordinates of all observation points are used to generate a Delaunay triangulation in the triangulation construction step.

4. The plane detection method according to claim 1, wherein: In the normal vector calculation step, select the origin of the local coordinate system B Discard the direction away from the origin of the local coordinate system B This step specifically includes the following steps: For each two opposite directions In the world coordinate system W, calculate The corresponding point on the triangle As the starting point, with the origin of the local coordinate system B The vector of the end point Judgment and Is the angle less than 90 degrees? If so, select the If not, discard the 5. The plane detection method according to claim 1, wherein: In the plane calculation step, the step of calculating the parameters of the plane corresponding to a class using multiple observation points constituting each triangular face in the class specifically includes the following steps: The steps for calculating the plane equation are as follows: ax+by+cz+d=0(c≠0) The formula can be transformed into, z=a0x+a1y+a2 in, For the coordinates of k observation points in this class (x1, y1, z1), (x2, y2, z2), ..., (x k ,y k ,z k ), construct the least squares matrix form Ax = b, that is Solving (a0, a1, a2) yields the plane equation parameters of the plane.

6. The plane detection method according to claim 5, wherein: The plane calculation step further includes the following steps: The plane normal vector calculation step obtains (a0, a1, a2) in the plane equation calculation step, using the formula: Calculate the normal vector of the plane as (a, b, c); 7. The plane detection method according to claim 5, wherein: The plane calculation step also includes the following steps: a plane coordinate system calculation step, establishing a local coordinate system of the plane, making the z axis of the local coordinate system of the plane and the plane normal vector have the same direction, and making the x axis, y axis and z axis of the local coordinate system of the plane orthogonal; calculating the projection position of the center point of the k observation points in the class on the plane, making the projection position the origin of the local coordinate system of the plane; calculating the pose T of the local coordinate system of the plane in the world coordinate system W PW .

8. The plane detection method according to claim 5, wherein: The plane calculation step further includes the following steps: a plane boundary contour calculation step, calculating the projection positions of k observation points in the class on the plane, and obtaining the coordinates of the k projection positions (x′1, y′1), (x′2, y′2), ..., (x′ k ,y′ k ); Calculate the convex hull contours {Q0, Q1..., Q m }, as the boundary contour of the plane.

9. The plane detection method according to claim 1, wherein: After the triangulated network construction step and before the normal vector calculation step, the method further includes: In the plane expansion step, if there is at least one detected plane in the memory, for each observation point, determine whether the observation point belongs to any plane already in the memory; determine whether there is a plane such that the distance between the observation point and the plane is less than a preset threshold γ, and determine whether there is an observation point in the plane and the distance between the observation point and the observation point is less than a preset threshold δ; if both are yes, merge the observation point into the plane, and delete the observation point and the triangle formed by the observation point in the Delaunay triangulation; if any of the judgment conditions is no, the observation point is not considered to belong to any plane and is retained.

10. The plane detection method according to claim 1, wherein: After the plane calculation step, the method further includes: In the plane fusion step, all observation points are retrieved. If any observation point and its adjacent observation point belong to two planes respectively, whether the two planes are adjacent is determined. Whether the angle between the normal vector of the first plane and the normal vector of the second plane is less than a preset threshold λ is determined. Whether more than half of the observation points in the first plane are less than a preset threshold τ from the second plane is determined. Whether more than half of the observation points in the second plane are less than a preset threshold τ from the first plane is determined. If all of the above are true, all observation points in the second plane are merged into the first plane and the second plane is deleted. Alternatively, all observation points in the first plane are merged into the second plane and the first plane is deleted. If any one of the judgment conditions is false, the first plane and the second plane are not considered to be the same plane.

11. A data processing device, characterized in that: include: a memory for storing executable program code; as well as A processor is configured to read the executable program code to run a computer program corresponding to the executable program code to perform at least one step of the plane detection method according to any one of claims 1 to 10.

Images