A method for constructing and measuring indoor structured semantic maps using drones

Abstract

The present invention provides a method for constructing and measuring an indoor structured semantic map of a drone, the method comprising the following steps: S1 a room segmentation algorithm based on a two-dimensional map, step S2 a room semantic map construction based on a voxel plane, step S3 a voxel-based region growing algorithm, and step S4 a room three-dimensional size automatic measurement algorithm. The present invention performs room segmentation on a point cloud map constructed by a laser radar, and divides the indoor point cloud map into a plurality of room sub-maps, so as to clearly display the structure and relative position relationship of each room. Based on the constructed room semantic map, a region growing algorithm based on a voxel plane is used to extract three structural planes in each room: the ground, the ceiling, and the wall, and then the three-dimensional size of the room is automatically calculated, so as to realize stable and accurate automatic measurement of the three-dimensional size of the room, so as to be able to construct a clearly structured indoor structural map and accurately measure the three-dimensional size of the room, and effectively improve the efficiency of drone exploration of unknown indoor environments.

Description

Technical Field

[0001] The invention discloses a method for constructing and measuring an indoor structured semantic map of an unmanned aerial vehicle, and belongs to the technical field of unmanned aerial vehicles. Background Art

[0002] In the existing technology, the exploration of unknown indoor environments by drones is an important research direction in the field of drones. The exploration of unknown indoor environments by drones usually requires the use of lidar to build an indoor environment map, and to measure the dimensions of the indoor environment based on the constructed map. Subsequently, path planning, target tracking and other tasks are carried out based on the constructed map and the dimension measurement results. As the basic technology for exploring unknown indoor environments, indoor map construction and dimension measurement have received widespread attention.

[0003] Traditional point cloud maps of indoor environments are gradually unable to meet the needs of robot intelligence, and the information they provide usually only contains the geometric features of the scene. In comparison, semantic maps contain the semantic information of objects in the scene, which can provide a certain understanding of the environment and thus realize intelligent human-computer interaction. The construction of indoor semantic maps generally includes two parts: semantic segmentation and map construction. The semantic segmentation part usually obtains the semantic labels of objects in the current scene based on target detection, and the map construction part distinguishes different objects and constructs a map based on different semantic labels. At present, deep learning semantic segmentation methods can be divided into point-based semantic segmentation (PointNet algorithm), voxel-based semantic segmentation (PointPillars algorithm), and projection-based semantic segmentation according to the different organizational forms of the input data.

[0004] At present, the construction of semantic maps is usually based on the semantic labels obtained by the above-mentioned semantic segmentation algorithm. Different objects in the scene are distinguished according to the semantic labels and the map is constructed. However, in an indoor environment containing multiple rooms, it is difficult to use semantic labels to distinguish the specific structure of the room and distinguish different rooms, and it is impossible to directly measure the three-dimensional size of the corresponding room. Now there is an urgent need for a drone indoor structured semantic map construction and measurement method to solve the above problems. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention aims to provide a method for constructing and measuring indoor structured semantic maps using drones to solve the problems raised in the above-mentioned background technology. The present invention divides the indoor point cloud map constructed by the laser radar into multiple room sub-maps by performing room segmentation, which can clearly display the structure and relative position relationship of each room. Based on the constructed room semantic map, a region growing algorithm based on voxel planes is used to extract three structural planes in each room: the ground, ceiling, and wall, and then the three-dimensional size of the room is automatically calculated to achieve stable and accurate automatic measurement of the three-dimensional size of the room.

[0006] In order to achieve the above object, the present invention is implemented by the following technical solution: a method for constructing and measuring indoor structured semantic maps of drones, the method comprising the following steps:

[0007] Step S1: A room segmentation algorithm based on a two-dimensional map uses a three-dimensional laser radar sensor to obtain point cloud information of the surrounding environment in real time to construct a three-dimensional point cloud map. After projecting the three-dimensional point cloud map to two dimensions, it is pre-segmented into rooms. Assuming that the largest gap in the wall of each room is the entrance and exit of the room, a circular detection window is designed. The detection window is used to traverse each occupied pixel in the two-dimensional map, and the number of unoccupied pixels around each occupied pixel is counted. When there are no unoccupied pixels around, the current pixel is classified as part of the initial segmentation, otherwise, it is classified as an unsegmented part.

[0008] Compared with the square detection window, the circular detection window avoids mis-segmentation at the corners of the room. The diameter of the circular detection window needs to be larger than the largest gap in the room, otherwise the room gap in the initial segmentation result will not be completely closed, resulting in the inability to segment each room. The gray part is the unoccupied pixel, the white part is the pixel occupied by the room interior, and the black part is the pixel occupied by the room boundary. Based on the pre-segmentation result of the two-dimensional room map, the room gap needs to be closed to obtain the final segmentation result. After the two-dimensional room map pre-segmentation determines the room position, first search for the walls around the room according to the pre-segmentation result. At this time, the search radius of the circular search window is twice the search window during pre-segmentation to ensure that the walls near the room are fully searched and the wall skeleton is extracted. Due to the existence of gaps, there will be multiple partition skeletons. Assuming that there are two gaps in the current room, resulting in the surrounding wall being divided into two wall skeletons, then take one pixel from each of the two skeletons, iteratively calculate the distance between the two pixels, and get the two closest pixels as follows:

[0009]

[0010] Among them, n * , m * is the number of pixels in the two skeletons, P i , P j are the pixel points on the two skeletons respectively. is a set of points on two skeletons that are closest to each other;

[0011] Step S2: Based on the voxel plane, the room semantic map is constructed, and the point cloud map is stored in voxels. Let the voxel be a cube with a side length of ε0, then the corresponding relationship between the voxel coordinate index and the point cloud coordinate is:

[0012]

[0013] Among them, V x , V y , Vz is the voxel three-axis coordinate index, is the point cloud coordinate in the world coordinate system, y=[x] is the Gaussian function, [x] represents the largest integer not exceeding the real number x, that is, x is rounded;

[0014] After extracting the room plan structure, the voxel plane does not contain its specific structural information. The structural plane needs to be classified. The three plane normal vectors of the ceiling, ground and wall are different. They are directly judged by the direction of the plane normal vector. The plane normal vector calculated by the singular value decomposition method does not always point to the direction of the carrier. The plane normal vector direction needs to be uniformly modified to point to one side in the closed room. The judgment is based on the size of the angle between the direction vector from a point on the voxel plane to the carrier position and the voxel plane normal vector. The direction of the plane normal vector that does not meet the standard is reversed. When the angle between the two direction vectors is less than 180°, it means that the direction of this plane normal vector meets the requirements. The plane normal vector direction is reversed. After the plane normal vector direction is corrected, the plane is divided into three categories: ceiling, ground and wall according to the normal vector direction:

[0015]

[0016] Among them, n g , n c , n w are the ground, ceiling, and wall normal vectors respectively, η0 is a unit vector and satisfies η0=[0,0,1], θ t is the angle threshold. The angle between the normal vector and η0 is less than θ t The plane is the ground, which is greater than (π-θ t ) is the ceiling, located at (π / 2-θ t ) and (π / 2+θ t ) is a wall. After classifying the voxel plane into ceiling, ground and wall, the room semantic map consisting of ceiling, ground and wall sub-maps is constructed based on the room segmentation result.

[0017] Step S3: A voxel-based region growing algorithm is used to extract the three planes of the floor, wall, and ceiling of each room from the room semantic map. The specific steps are as follows:

[0018] (1) Establish a dynamic normal histogram H represented by a spherical coordinate system p , use polar coordinates and azimuth to represent the voxel plane normal vector, assuming that the voxel plane set after region growth is R p , set its initial value to the empty set φ;

[0019] (2) Iteratively retrieve the main directions of the voxel plane normals in the histogram and extract the corresponding voxel plane set h p, the voxel plane s with the minimum MSE p Prioritize as seed voxels for region growing;

[0020] (3) Calculate the seed voxel s p With adjacent voxels c p The normal vector angle and the distance between the two voxel planes are determined if they meet the following conditions: p With adjacent voxels c p Coplanar:

[0021] s p and c p The normal vector angle is less than θ M,p ,like:

[0022]

[0023] Among them, n p 、n s The adjacent voxels c p With seed voxels s p The normal vector, θ M is an angle threshold;

[0024] c p Center point to s p The distance is less than the dynamic distance threshold D(c p ,s p ), the value of which is positively correlated with the distance between two voxels, such as:

[0025]

[0026] in, The voxel c p and seed voxels s p The center point coordinates, θ M,p is an angle threshold, D s is the seed voxel s p The intercept of the fitted plane equation, r D is a proportionality coefficient;

[0027] (4) The adjacent voxels c after region growth p As a benchmark, the adjacent voxels are region-grown, and the two voxels are iteratively checked to see if they are coplanar, until there are no adjacent voxels that meet the criteria, and a voxel plane set r is output. p As a result of regional growth;

[0028] (5) The voxel plane set r p From the histogram H p Delete it and find H pThe main directions of the plane normals of the remaining voxels in are used for the next iterative region growing;

[0029] (6) Use the ratio of the second largest eigenvalue to the smallest eigenvalue of the plane covariance matrix r λ Evaluate the planarity of the voxel plane set after region growth, if r λ Greater than the threshold r m , then mark the current set as a plane and add it to the increased plane set R p ;

[0030] Step S4: Automatic measurement algorithm of room 3D dimensions, based on the constructed room semantic map, the specific steps are as follows:

[0031] (1)V w is the wall set obtained by counting in the normal vector histogram. The wall area is positively correlated with the number of occupied voxels. Therefore, V w The included walls are arranged in descending order according to the number of voxels they occupy.

[0032] (2) Extraction of V w The maximum wall P M , according to V w The remaining wall area is traversed from large to small to calculate the current wall area P w With P M If the distance is greater than the distance threshold d m If the distance between the two walls is far, it is considered that the requirements are met, otherwise the next cycle is performed;

[0033] (3) P w Project to P according to the normal vector direction M Get on the plane

[0034] (4) Calculation With P M The overlapping area size S w , if it is greater than the area threshold S m , it means that the two walls are opposite and overlap to a certain extent, which meets the requirements of automatic measurement. This pair of walls is used as the marking wall Q w Output, otherwise proceed to the next loop;

[0035] After obtaining two pairs of marked walls, since the directions of the same pair of marked walls are similar, the distance between the walls is replaced by the distance from the center point of the small wall to the large wall. The distances between the two pairs of marked walls are calculated respectively to obtain the length and width of the target measurement room. The semantic map of the room contains the ground and ceiling maps, and the ground and ceiling of the room may not be completely flat. Directly fitting all voxel planes will result in large measurement errors. Therefore, seed points are randomly selected from the ground and ceiling maps for voxel-based region growth. After traversing all voxel planes, the growth results that occupy the most voxels are extracted as the ground and ceiling planes of the room. Similarly, the size of the extracted ground and ceiling areas is compared, and the distance from the center point of the small plane to the large plane is taken as the distance between the two planes, that is, the height h of the target measurement room. r , the calculation formula is as follows:

[0036]

[0037] Among them, p c , p g are a point on the ceiling and the ground respectively, n c , n g are the normal vectors of the ceiling and the ground plane, S c , S g are the ceiling and floor areas respectively.

[0038] Furthermore, in order to close the display of each room in the room semantic map, the room entrance and exit need to be closed through certain processing, that is, the room is finely segmented. By temporarily constructing a gap larger than the room entrance and exit, the skeleton is divided into two sections. At this time, the nearest pixel in the two skeletons is iteratively detected to be the room entrance and exit. It should be noted that the constructed gap needs to be on the main branch of the skeleton, otherwise, searching for the nearest pixel may detect other nearby branches and cause the closure of the wrong gap. Therefore, it is necessary to first remove other branches in the entire skeleton and leave only the main branch. Use the circular detection window of the same size as mentioned above to traverse the entire skeleton pixels, remove the end pixels and node pixels of other branches, and other skeleton pixels in the detection window. Finally, find the nearest pixels in the skeletons at both ends and connect them to complete the closure of the room entrance and exit.

[0039] Furthermore, in step S2, after the voxel point cloud map is established, a local plane fitting is performed on the point cloud within the same voxel, and the eigenvector corresponding to the minimum eigenvalue of the point cloud covariance matrix is ​​calculated to obtain the local plane normal vector. If the calculated plane root mean square error is small enough, it is classified as a planar voxel and subsequent segmentation operations are performed, otherwise it is classified as a non-planar voxel and removed from the voxel map.

[0040] Beneficial effects of the present invention: The present invention provides a method for constructing and measuring indoor structured semantic maps of drones. The present invention divides the indoor point cloud map into multiple room sub-maps by performing room segmentation on the point cloud map constructed by the laser radar, and can clearly display the structure and relative position relationship of each room. Based on the constructed room semantic map, the three structural planes in each room are extracted using a voxel plane-based region growing algorithm: the ground, ceiling, and wall, and then the three-dimensional dimensions of the room are automatically calculated to achieve stable and accurate automatic measurement of the three-dimensional dimensions of the room. It is possible to construct a clearly structured indoor structure map and accurately measure the three-dimensional dimensions of the room, effectively improving the efficiency of drone exploration of unknown indoor environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0042] Figure 1 This is a flowchart of the room semantic map construction and measurement algorithm based on LiDAR;

[0043] Figure 2 It is a schematic diagram of the room pre-division;

[0044] Figure 3 This is a schematic diagram of the detailed division of the room;

[0045] Figure 4 It is a schematic diagram of room plan classification;

[0046] Figure 5 It is a flow chart of the voxel-based region growing algorithm;

[0047] Figure 6 It is a flow chart of the algorithm for measuring the three-dimensional dimensions of a room. DETAILED DESCRIPTION

[0048] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0049] The present invention provides a technical solution: a method for constructing and measuring indoor structured semantic maps using drones, the flow chart of which is as follows: Figure 1 As shown, the method comprises the following steps:

[0050] Step S1: A room segmentation algorithm based on a two-dimensional map uses a three-dimensional laser radar sensor to obtain point cloud information of the surrounding environment in real time to construct a three-dimensional point cloud map. After projecting the three-dimensional point cloud map to two dimensions, it is pre-segmented into rooms. Assuming that the largest gap in the wall of each room is the entrance and exit of the room, a circular detection window is designed. The detection window is used to traverse each occupied pixel in the two-dimensional map, and the number of unoccupied pixels around each occupied pixel is counted. When there are no unoccupied pixels around, the current pixel is classified as part of the initial segmentation, otherwise, it is classified as an unsegmented part.

[0051] The circular detection window avoids mis-segmentation at the corners of the room compared to the square detection window. The diameter of the circular detection window needs to be larger than the largest gap in the room. Otherwise, the room gaps in the initial segmentation result will not be completely closed, resulting in failure to segment each room. The room pre-segmentation result is as follows: Figure 2 , where the gray part is unoccupied pixels, the white part is the pixels occupied by the room interior, and the black part is the pixels occupied by the room boundary. Based on the pre-segmentation result of the two-dimensional room map, the room gap needs to be closed to obtain the final segmentation result. After the two-dimensional room map pre-segmentation determines the room position, first search for the walls around the room according to the pre-segmentation result. At this time, the search radius of the circular search window is twice the search window during pre-segmentation to ensure that the walls near the room are fully searched and the wall skeleton is extracted. Due to the existence of gaps, there will be multiple partition skeletons. Assuming that there are two gaps in the current room, resulting in the surrounding wall being divided into two wall skeletons, then take one pixel from each of the two skeletons, iteratively calculate the distance between the two pixels, and get the two closest pixels as follows:

[0052]

[0053] Among them, n * , m * is the number of pixels in the two skeletons, P i , P j are the pixel points on the two skeletons respectively. is a set of points on two skeletons that are closest to each other;

[0054] In order to close each room in the room semantic map, the entrance and exit of the room need to be closed through certain processing, that is, the fine segmentation of the room. By temporarily constructing a gap larger than the entrance and exit of the room, the skeleton is divided into two sections. At this time, the closest pixel in the two skeletons is iteratively detected as the entrance and exit of the room. It should be noted that the constructed gap needs to be on the main branch of the skeleton, otherwise, searching for the nearest pixel may detect other nearby branches and cause the closure of the wrong gap. Therefore, it is necessary to first remove other branches in the entire skeleton and leave only the main branch. Use the circular detection window of the same size as mentioned above to traverse the entire skeleton pixels, remove the end pixels and node pixels of other branches, and other skeleton pixels in the detection window. Finally, find the nearest pixels in the skeletons at both ends and connect them to complete the closure of the room entrance and exit. The schematic diagram of the fine segmentation of the room is shown in the figure. Figure 3 ;

[0055] Step S2: Based on the voxel plane, the room semantic map is constructed, and the point cloud map is stored in voxels. Let the voxel be a cube with a side length of ε0, then the corresponding relationship between the voxel coordinate index and the point cloud coordinate is:

[0056]

[0057] Among them, V x , V y , V z is the voxel three-axis coordinate index, is the point cloud coordinate in the world coordinate system, y=[x] is the Gaussian function, [x] represents the largest integer not exceeding the real number x, that is, x is rounded;

[0058] After extracting the room plan structure, the voxel plane does not contain its specific structural information, and the structural plane needs to be classified. The three plane normal vectors of the ceiling, ground and wall are different. They are directly judged by the plane normal vector direction. The plane normal vector calculated by the singular value decomposition method does not always point to the direction of the carrier. The plane normal vector direction needs to be uniformly modified to point to one side in the closed room. The judgment is based on the angle between the direction vector from a point on the voxel plane to the carrier position and the voxel plane normal vector. The direction of the plane normal vector that does not meet the standard is reversed, such as Figure 4 As shown in the figure, when the angle between the two direction vectors is less than 180°, it means that the direction of the plane normal vector meets the requirements. After the direction of the plane normal vector is reversed and corrected, the plane is divided into three categories: ceiling, ground and wall according to the direction of the normal vector:

[0059]

[0060] Among them, n g , n c , n ware the ground, ceiling, and wall normal vectors respectively, η0 is a unit vector and satisfies η0=[0,0,1], θ t is the angle threshold. The angle between the normal vector and η0 is less than θ t The plane is the ground, which is greater than (π-θ t ) is the ceiling, located at (π / 2-θ t ) and (π / 2+θ t ) is a wall. After classifying the voxel plane into ceiling, ground and wall, the room semantic map consisting of ceiling, ground and wall sub-maps is constructed based on the room segmentation result.

[0061] Step S3: A voxel-based region growing algorithm is used to extract the three planes of the floor, wall and ceiling of each room from the room semantic map. The process of the voxel-based region growing algorithm of the present invention is as follows: Figure 5 , the specific steps are as follows:

[0062] (1) Establish a dynamic normal histogram H represented by a spherical coordinate system p , use polar coordinates and azimuth to represent the voxel plane normal vector, assuming that the voxel plane set after region growth is R p , set its initial value to the empty set φ;

[0063] (2) Iteratively retrieve the main directions of the voxel plane normals in the histogram and extract the corresponding voxel plane set h p , the voxel plane s with the minimum MSE p Prioritize as seed voxels for region growing;

[0064] (3) Calculate the seed voxel s p With adjacent voxels c p The normal vector angle and the distance between the two voxel planes are determined if they meet the following conditions: p With adjacent voxels c p Coplanar:

[0065] s p and c p The normal vector angle is less than θ M,p ,like:

[0066]

[0067] Among them, n p 、n s The adjacent voxels c p With seed voxels s p The normal vector, θ M is an angle threshold;

[0068] cp Center point to s p The distance is less than the dynamic distance threshold D(c p ,s p ), the value of which is positively correlated with the distance between two voxels, such as:

[0069]

[0070] in, The voxel c p and seed voxels s p The center point coordinates, θ M,p is an angle threshold, D s is the seed voxel s p The intercept of the fitted plane equation, r D is a proportionality coefficient;

[0071] (4) The adjacent voxels c after region growth p As a benchmark, the adjacent voxels are region-grown, and the two voxels are iteratively checked to see if they are coplanar, until there are no adjacent voxels that meet the criteria, and a voxel plane set r is output. p As a result of regional growth;

[0072] (5) The voxel plane set r p From the histogram H p Delete it and find H p The main directions of the plane normals of the remaining voxels in are used for the next iterative region growing;

[0073] (6) Use the ratio of the second largest eigenvalue to the smallest eigenvalue of the plane covariance matrix r λ Evaluate the planarity of the voxel plane set after region growth, if r λ Greater than the threshold r m , then mark the current set as a plane and add it to the increased plane set R p ;

[0074] Step S4: Automatic measurement algorithm of room 3D dimensions, based on the constructed room semantic map, its process is as follows Figure 6 , the specific steps are as follows:

[0075] (1)V w is the wall set obtained by counting in the normal vector histogram. The wall area is positively correlated with the number of occupied voxels. Therefore, V w The included walls are arranged in descending order according to the number of voxels they occupy.

[0076] (2) Extraction of V w The maximum wall P M , according to V wThe remaining wall area is traversed from large to small to calculate the current wall area P w With P M If the distance is greater than the distance threshold d m If the distance between the two walls is far, it is considered that the requirements are met, otherwise the next cycle is performed;

[0077] (3) P w Project to P according to the normal vector direction M Get on the plane

[0078] (4) Calculation With P M The overlapping area size S w , if it is greater than the area threshold S m , it means that the two walls are opposite and overlap to a certain extent, which meets the requirements of automatic measurement. This pair of walls is used as the marking wall Q w Output, otherwise proceed to the next loop;

[0079] After obtaining two pairs of marked walls, since the directions of the same pair of marked walls are similar, the distance between the walls is replaced by the distance from the center point of the small wall to the large wall. The distances between the two pairs of marked walls are calculated respectively to obtain the length and width of the target measurement room. The semantic map of the room contains the ground and ceiling maps, and the ground and ceiling of the room may not be completely flat. Directly fitting all voxel planes will result in large measurement errors. Therefore, seed points are randomly selected from the ground and ceiling maps for voxel-based region growth. After traversing all voxel planes, the growth results that occupy the most voxels are extracted as the ground and ceiling planes of the room. Similarly, the size of the extracted ground and ceiling areas is compared, and the distance from the center point of the small plane to the large plane is taken as the distance between the two planes, that is, the height h of the target measurement room. r , the calculation formula is as follows:

[0080]

[0081] Among them, p c , p g are a point on the ceiling and the ground respectively, n c , n g are the normal vectors of the ceiling and the ground plane, S c , S g are the ceiling and floor plane areas respectively;

[0082] In step S2, after the voxel point cloud map is established, a local plane fitting is performed on the point cloud within the same voxel, and the eigenvector corresponding to the minimum eigenvalue of the point cloud covariance matrix is ​​calculated to obtain the local plane normal vector. If the calculated plane root mean square error is small enough, it is classified as a planar voxel and subsequent segmentation operations are performed. Otherwise, it is classified as a non-planar voxel and removed from the voxel map.

[0083] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the attached claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any figure mark in the claims should not be regarded as limiting the claims involved.

[0084] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A method for constructing and measuring indoor structured semantic maps using drones, characterized in that: The method comprises the following steps: Step S1: A room segmentation algorithm based on a two-dimensional map uses a three-dimensional laser radar sensor to obtain point cloud information of the surrounding environment in real time to construct a three-dimensional point cloud map. After projecting the three-dimensional point cloud map to two dimensions, it is pre-segmented into rooms. Assuming that the largest gap in the wall of each room is the entrance and exit of the room, a circular detection window is designed. The detection window is used to traverse each occupied pixel in the two-dimensional map, and the number of unoccupied pixels around each occupied pixel is counted. When there are no unoccupied pixels around, the current pixel is classified as part of the initial segmentation, otherwise, it is classified as an unsegmented part. Compared with the square detection window, the circular detection window avoids mis-segmentation at the corners of the room. The diameter of the circular detection window needs to be larger than the largest gap in the room, otherwise the room gap in the initial segmentation result will not be completely closed, resulting in the inability to segment each room. The gray part is the unoccupied pixel, the white part is the pixel occupied by the room interior, and the black part is the pixel occupied by the room boundary. Based on the pre-segmentation result of the two-dimensional room map, the room gap needs to be closed to obtain the final segmentation result. After the two-dimensional room map pre-segmentation determines the room position, first search for the walls around the room according to the pre-segmentation result. At this time, the search radius of the circular search window is twice the search window during pre-segmentation to ensure that the walls near the room are fully searched and the wall skeleton is extracted. Due to the existence of gaps, there will be multiple partition skeletons. Assuming that there are two gaps in the current room, resulting in the surrounding wall being divided into two wall skeletons, then take one pixel from each of the two skeletons, iteratively calculate the distance between the two pixels, and get the two closest pixels as follows: Among them, n * , m * is the number of pixels in the two skeletons, P i , P j are the pixel points on the two skeletons respectively. is a set of points on two skeletons that are closest to each other; Step S2: Based on the voxel plane, the room semantic map is constructed, and the point cloud map is stored in voxels. Let the voxel be a cube with a side length of ε0, then the corresponding relationship between the voxel coordinate index and the point cloud coordinate is: Among them, V x , V y , V z is the voxel three-axis coordinate index, is the point cloud coordinate in the world coordinate system, y=[x] is the Gaussian function, [x] represents the largest integer not exceeding the real number x, that is, x is rounded; After extracting the room plan structure, the voxel plane does not contain its specific structural information. The structural plane needs to be classified. The three plane normal vectors of the ceiling, ground and wall are different. They are directly judged by the direction of the plane normal vector. The plane normal vector calculated by the singular value decomposition method does not always point to the direction of the carrier. The plane normal vector direction needs to be uniformly modified to point to one side in the closed room. The judgment is based on the size of the angle between the direction vector from a point on the voxel plane to the carrier position and the voxel plane normal vector. The direction of the plane normal vector that does not meet the standard is reversed. When the angle between the two direction vectors is less than 180°, it means that the direction of this plane normal vector meets the requirements. The plane normal vector direction is reversed. After the plane normal vector direction is corrected, the plane is divided into three categories: ceiling, ground and wall according to the normal vector direction: Among them, n g , n c , n w are the ground, ceiling, and wall normal vectors respectively, η0 is a unit vector and satisfies η0=[0,0,1], θ t is the angle threshold; the angle between the normal vector and η0 is less than θ t The plane is the ground, which is greater than (π-θ t ) is the ceiling, located at (π / 2-θ t ) and (π / 2+θ t ) is a wall. After classifying the voxel plane into ceiling, ground and wall, the room semantic map consisting of ceiling, ground and wall sub-maps is constructed based on the room segmentation result. Step S3: A voxel-based region growing algorithm is used to extract the three planes of the floor, wall, and ceiling of each room from the room semantic map. The specific steps are as follows: (1) Establish a dynamic normal histogram H represented by a spherical coordinate system p , use polar coordinates and azimuth to represent the voxel plane normal vector, assuming that the voxel plane set after region growth is R p , set its initial value to the empty set φ; (2) Iteratively retrieve the main directions of the voxel plane normals in the histogram and extract the corresponding voxel plane set h p , the voxel plane s with the minimum MSE p Prioritize as seed voxels for region growing; (3) Calculate the seed voxel s p With adjacent voxels c p The normal vector angle and the distance between the two voxel planes are determined if they meet the following conditions: p With adjacent voxels c p Coplanar: s p and c p The normal vector angle is less than θ M ,like: arccos(n p ·n s )<θ M Among them, n p 、n s The adjacent voxels c p With seed voxels s p The normal vector, θ M is an angle threshold; c p Center point to s p The distance is less than the dynamic distance threshold D(c p ,s p ), the value of which is positively correlated with the distance between two voxels, such as: in, The voxel c p and seed voxels s p The center point coordinates, θ M,p is an angle threshold, D s is the seed voxel s p The intercept of the fitted plane equation, r D is a proportionality factor; (4) The adjacent voxels c after region growth p As a benchmark, the adjacent voxels are regionally grown, and the two voxels are iteratively checked to see if they are coplanar, until there are no adjacent voxels that meet the conditions, and a voxel plane set r is output. p As a result of regional growth; (5) The voxel plane set r p From the histogram H p Delete it and find H p The main directions of the plane normals of the remaining voxels in are used for the next iterative region growing; (6) Use the ratio of the second largest eigenvalue to the smallest eigenvalue of the plane covariance matrix r λ Evaluate the planarity of the voxel plane set after region growth, if r λ Greater than the threshold r m , then mark the current set as a plane and add it to the increased plane set R p ; Step S4: Automatic measurement algorithm of room 3D dimensions, based on the constructed room semantic map, the specific steps are as follows: (1)V w is the wall set obtained by counting in the normal vector histogram. The wall area is positively correlated with the number of occupied voxels. Therefore, V w The included walls are arranged in descending order according to the number of voxels they occupy. (2) Extraction of V w The maximum wall P M , according to V w The remaining wall area is traversed from large to small to calculate the current wall area P w With P M If the distance is greater than the distance threshold d m If the distance between the two walls is far, it is considered to meet the requirements, otherwise the next cycle is performed; (3) P w Project to P according to the normal vector direction M Get on the plane (4) Calculation With P M The overlapping area size S w , if it is greater than the area threshold S m , it means that the two walls are opposite and overlap to a certain extent, which meets the requirements of automatic measurement. This pair of walls is used as the marking wall Q w Output, otherwise proceed to the next loop; After obtaining two pairs of marked walls, since the directions of the same pair of marked walls are similar, the distance between the walls is replaced by the distance from the center point of the small wall to the large wall. The distances between the two pairs of marked walls are calculated separately to obtain the length and width of the target measurement room. The semantic map of the room contains the ground and ceiling maps, and the ground and ceiling of the room may not be completely flat. Directly fitting all voxel planes will result in large measurement errors. Therefore, seed points are randomly selected from the ground and ceiling maps for voxel-based region growth. After traversing all voxel planes, the growth results that occupy the most voxels are extracted as the ground and ceiling planes of the room. Similarly, the distance from the center point of the small plane to the large plane is taken as the distance between the two planes, that is, the height h of the target measurement room. r , the calculation formula is as follows: Among them, p c , p g are a point on the ceiling and the ground respectively, n c , n g are the normal vectors of the ceiling and the ground plane, S c , S g are the ceiling and floor areas respectively.

2. The method for constructing and measuring indoor structured semantic maps of unmanned aerial vehicles according to claim 1, characterized in that: In order to close each room in the room semantic map, the room entrance and exit need to be closed through certain processing, that is, the room is finely segmented. By temporarily constructing a gap larger than the room entrance and exit, the skeleton is divided into two sections. At this time, the nearest pixel in the two skeletons is iteratively detected as the room entrance and exit. It should be noted that the constructed gap needs to be on the main branch of the skeleton, otherwise, searching for the nearest pixel may detect other nearby branches and cause the closure of the wrong gap. Therefore, it is necessary to first remove other branches in the entire skeleton and leave only the main branch. Use the circular detection window of the same size as mentioned above to traverse the entire skeleton pixels, remove the end pixels and node pixels of other branches, and other skeleton pixels in the detection window. Finally, find the nearest pixels in the skeletons at both ends and connect them to complete the closure of the room entrance and exit.

3. The method for constructing and measuring indoor structured semantic maps of unmanned aerial vehicles according to claim 1, characterized in that: In step S2, after the voxel point cloud map is established, a local plane fitting is performed on the point cloud within the same voxel, and the eigenvector corresponding to the minimum eigenvalue of the point cloud covariance matrix is ​​calculated to obtain the local plane normal vector. If the calculated plane root mean square error is small enough, it is classified as a planar voxel and subsequent segmentation operations are performed. Otherwise, it is classified as a non-planar voxel and removed from the voxel map.

Images