Large-scale point cloud semantic segmentation method and system based on downscaling projection

Abstract

The invention discloses a large-scale point cloud semantic segmentation method and system based on downscaling projection, and the method comprises the following steps: obtaining a to-be-processed point cloud, setting a downscaling projection viewpoint and sight of a virtual view according to the spatial distribution of the large-scale point cloud, and obtaining the external parameters of a virtual camera; according to the internal reference and the external reference of the virtual camera corresponding to the mth viewpoint, a corresponding virtual view is generated, meanwhile, the mapping relation between the three-dimensional point cloud and the two-dimensional projection point is obtained, and the internal reference of the virtual camera comprises the focal length, the pixel size and the image amplitude size; semantic information in the virtual view is extracted based on an existing two-dimensional image semantic segmentation algorithm, the semantic information is transmitted from two dimensions to three dimensions based on the obtained mapping relation between the three-dimensional point cloud and the two-dimensional projection points, and semantic segmentation of the point cloud is completed. Semantic segmentation of large-scale and large-scale point clouds can be realized, and the semantic segmentation precision is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of semantic segmentation, and in particular relates to a large-scale point cloud semantic segmentation method and system based on downscaling projection. Background Art

[0002] With the popularization and performance improvement of three-dimensional perception technologies such as LiDAR and depth cameras, it has become possible to efficiently and accurately capture and reconstruct point cloud data of complex three-dimensional scenes. However, the geometric information of point cloud data alone is far from meeting the needs of advanced applications. How to automatically understand and parse the semantic information in these point cloud data, that is, to identify the category, location and even the relationship between each object in the scene, has become a hot research topic. Point cloud semantic segmentation technology came into being. By assigning a semantic label (such as vehicles, pedestrians, trees, buildings, etc.) to each point in the point cloud, it has achieved a leap from raw three-dimensional data to advanced semantic understanding, providing a solid foundation for subsequent decision-making, environmental perception and interaction.

[0003] The existing point cloud semantic segmentation methods can be roughly divided into three types: the first is the point-based semantic segmentation method, the second is the voxel-based semantic segmentation method, and the third is the projection-based semantic segmentation method. The first and second methods are greatly affected by the density of the point cloud, and these two methods lack versatility and stability in scenes of different scales. The projection-based semantic segmentation method first converts the three-dimensional data into multiple sets of two-dimensional data, and then uses the currently mature two-dimensional semantic segmentation algorithm to stably obtain semantic information. Therefore, this method is less affected by the point cloud density and scene size, and is more suitable for point clouds in remote sensing scenes than the first two methods. However, due to the low-quality texture two-dimensional image semantic segmentation accuracy is not high enough, and the projection image cannot include all three-dimensional points, resulting in some three-dimensional point clouds lacking semantic information.

[0004] Therefore, it is urgent to solve the above problems. Summary of the invention

[0005] Purpose of the invention: The first purpose of the present invention is to provide a large-scale point cloud semantic segmentation method based on downscaling projection, which can generate projection views containing as many points as possible and as high quality as possible, extract sufficient semantic information, realize semantic segmentation of large-scale and large-scale point clouds, and improve the accuracy of semantic segmentation.

[0006] The second object of the present invention is to provide a large-scale point cloud semantic segmentation system based on downscaling projection.

[0007] A third object of the present invention is to provide an electronic device.

[0008] A fourth object of the present invention is to provide a computer storage medium.

[0009] Technical solution: To achieve the above objectives, the present invention discloses a large-scale point cloud semantic segmentation method based on downscaling projection, comprising the following steps: (1) Obtain the point cloud to be processed, set the downscaled projection viewpoint and line of sight of the virtual view according to the spatial distribution of the large-scale point cloud, and obtain the external parameters of the virtual camera; (2) Generate the corresponding virtual view based on the intrinsic and extrinsic parameters of the virtual camera corresponding to the m-th viewpoint, and obtain the mapping relationship between the 3D point cloud and the 2D projection point. The intrinsic parameters of the virtual camera include focal length, pixel size, and image size. (3) Semantic information in the virtual view is extracted based on the existing 2D image semantic segmentation algorithm. Based on the mapping relationship between the acquired 3D point cloud and the 2D projection points, the semantic information is transferred from 2D to 3D to complete the semantic segmentation of the point cloud.

[0010] Optionally, step (1) specifically includes the following steps: (1.1) Read the 3D model file and obtain the point cloud to be processed , to generate high-quality multi-view projections of point clouds, a multi-layer viewpoint network is set up ; (1.2) Get the point cloud corresponding to the k-th layer of viewpoint network to be processed The minimum directed bounding box of , determine the vertical direction away from the ground and along the ground in two directions and , a total of three directions , the initial value of k is 0; (1.3) Based on point cloud density Determine the image resolution of the current virtual view , , in is a constant, According to the ground height , Image resolution ,focal length and pixel size The geometric relationship between them determines the ground height of the view projection , , According to image resolution Get the ground coverage of the image , , in For square virtual view Figure 1 The number of pixels on the edge; (1.4) Ground height based on the k-th layer virtual view , Ground Coverage , three directions and overlap , set the viewpoint network of the kth layer , the direction of all viewpoints in the viewpoint network is vertical to the ground , the two directions of the view are the two ground directions of the point cloud to be processed, and the rotation matrix is ; The viewpoints in each layer of the viewpoint network are evenly distributed. Line The viewpoint coordinates of the columns are: , The range of row index values ​​is: , The value range of column index is: , The reference viewpoint coordinates are: , in is the ceiling function, are the eight vertices of the minimum directed bounding box, That is, the vertex with index 0. represents the size of the minimum oriented bounding box, is the length of the minimum directed bounding box, is the width of the minimum directional bounding box; thus, the virtual camera extrinsic parameter corresponding to the viewpoint in the i-th row and j-th column of the k-th viewpoint network is obtained. ; (1.5) Based on processing the ground coverage corresponding to each viewpoint , Ground height , three directions , generate the minimum directed bounding box of all point clouds within the field of view of the viewpoint, and extract the point cloud within the box ; If there is a point cloud in the box and the point cloud density Point cloud density of the point cloud corresponding to the previous viewpoint And the point cloud density of the entire point cloud The ratio between meets the threshold condition. , Then the point cloud in the box As the next layer of viewpoint network Repeat steps (1.2) to (1.5) for the point cloud to be targeted, otherwise terminate the iteration, and finally iterate to generate a multi-layer viewpoint network ; (1.6) The virtual camera extrinsic parameters corresponding to each layer of the multi-layer viewpoint network are collected to form a virtual camera extrinsic parameter set, where the virtual camera extrinsic parameter corresponding to the mth viewpoint in the virtual camera extrinsic parameter set is .

[0011] Optionally, step (2) specifically includes the following steps: (2.1) Based on the intrinsic and extrinsic parameters of the virtual camera corresponding to the m-th viewpoint, the entire point cloud The coordinate system of the world coordinate system Rotate the camera coordinate system ,reserve Points with positive Z values ​​in the coordinate system, that is, the point clouds in front of the virtual camera are retained and other point clouds are discarded; (2.2) From the camera coordinate system Convert to pixel coordinate system , retain the projection points within the image frame; (2.3) Corresponding to the camera coordinate system based on the projection points in the image frame The lower Z value, i.e. depth, retains the projection point with the smallest depth in the same pixel block and discards other points; (2.4) Assign the depth or grayscale of the selected 3D point cloud to the corresponding 2D projection point to obtain a one-to-one mapping relationship between the 3D point cloud and the 2D projection point. , to achieve the initial rendering of the virtual view ; (2.5) Rendering based on initialization ,To address the problem of holes and missing textures in the projected image, the minimum axis-aligned bounding box of all projection points in the image is obtained, and the projection view holes are searched within the bounding box; (2.6) Extract the neighborhood of each hole, and calculate the grayscale value of the hole based on the grayscale value of the non-hole in the neighborhood using the distance Gaussian weighted method , thereby filling the holes and enriching the texture, realizing the rendering post-processing of the virtual view, and generating the virtual view corresponding to the mth viewpoint ; , in is the pixel coordinate, yes Neighborhood of Represents a virtual view being initialized middle There is value everywhere, Represents pixel Distance on virtual view.

[0012] Optionally, step (3) specifically includes the following steps: (3.1) Process the point cloud with semantic information used for training according to steps (1) and (2), obtain the intrinsic parameters and extrinsic parameters of the virtual camera, generate a virtual view corresponding to the point cloud with semantic information, assign the semantic coding of the three-dimensional point cloud corresponding to the two-dimensional projection point contained in the virtual view to the corresponding two-dimensional projection point, and complete the first step of generating the semantic label map; in order to solve the problem of holes in the semantic label map, obtain the minimum axis-aligned bounding box where all the projection points in the semantic label map are located, search for image holes in the bounding box, extract the neighborhood of each hole, and count the semantic features in the neighborhood of each hole based on the semantic information of the semantic label map. The semantic feature corresponding to the largest number of points of a certain semantic feature in the neighborhood is the semantic feature of the point, and complete the second step of generating the semantic label map, thereby realizing the generation of the final semantic label map; based on the existing two-dimensional image semantic segmentation algorithm, use the virtual view and the corresponding semantic label map to train and generate a two-dimensional image semantic segmentation model; (3.2) The virtual view of step (2) Input the 2D image semantic segmentation model and extract the semantic label map , get the semantic information in the virtual view; (3.3) Create an initial voting table, initialized to zero, based on the 3D point cloud in each view and the two-dimensional projection point The mapping relationship between And the semantic information extracted from each view , the points that received votes are counted, and the semantic attribute with the highest number of votes at each point is set as the semantic attribute of the point, completing the first vote; (3.4) Create a new voting table. For each point that did not receive a vote in the initial voting table, extract its 3D spatial neighborhood and count the points in the neighborhood that have determined the semantic characteristics in the initial voting results. The semantic characteristic corresponding to the point with the largest number of points of a certain semantic characteristic in the neighborhood is the semantic characteristic of the point, completing the second vote. (3.5) The information contained in the two voting tables is merged to obtain the final voting result, completing the semantic information transmission based on the virtual view semantic label.

[0013] Based on the same inventive concept, the present invention discloses a large-scale point cloud semantic segmentation system based on downscaling projection, comprising: The external parameter acquisition module is used to obtain the point cloud to be processed, set the downscaled projection viewpoint and line of sight of the virtual view according to the spatial distribution of the large-scale point cloud, and obtain the external parameters of the virtual camera; A mapping relationship acquisition module is used to generate a corresponding virtual view for the internal parameters and external parameters of the virtual camera corresponding to the m-th viewpoint, and to obtain the mapping relationship between the three-dimensional point cloud and the two-dimensional projection point, wherein the internal parameters of the virtual camera include focal length, pixel size and image size; The semantic segmentation module is used to extract semantic information in the virtual view based on the existing 2D image semantic segmentation algorithm, and transfer the semantic information from 2D to 3D based on the mapping relationship between the acquired 3D point cloud and the 2D projection points to complete the semantic segmentation of the point cloud.

[0014] Optionally, read the 3D model file in the external parameter acquisition module to obtain the point cloud to be processed ; The 3D model file is obtained by laser radar scanning; in order to generate high-quality multi-view projection of point cloud, a multi-layer viewpoint network is set up ; Get the point cloud corresponding to the k-th layer of viewpoint network currently being processed The minimum directed bounding box of , determine the vertical direction away from the ground and along the ground in two directions and , a total of three directions , the initial value of k is 0; Based on point cloud density Determine the image resolution of the current virtual view , , in is a constant, According to the ground height , Image resolution ,focal length and pixel size The geometric relationship between them determines the ground height of the view projection , , According to image resolution Get the ground coverage of the image , , in For square virtual view Figure 1 The number of pixels on the edge; The ground height based on the k-th layer virtual view , Ground Coverage , three directions and overlap , set the viewpoint network of the kth layer , the direction of all viewpoints in the viewpoint network is vertical to the ground , the two directions of the view are the two ground directions of the point cloud to be processed, and the rotation matrix is ; The viewpoints in each layer of the viewpoint network are evenly distributed. Line The viewpoint coordinates of the columns are: , The range of row index values ​​is: , The value range of column index is: , The reference viewpoint coordinates are: , in is the ceiling function, are the eight vertices of the minimum directed bounding box, That is, the vertex with index 0. represents the size of the minimum oriented bounding box, is the length of the minimum directed bounding box, is the width of the minimum directional bounding box; thus, the virtual camera extrinsic parameter corresponding to the viewpoint in the i-th row and j-th column of the k-th viewpoint network is obtained. ; Based on processing the ground coverage corresponding to each viewpoint , Ground height , three directions , generate the minimum directed bounding box of all point clouds within the field of view of the viewpoint, and extract the point cloud within the box ; If there is a point cloud in the box and the point cloud density Point cloud density of the point cloud corresponding to the previous viewpoint And the point cloud density of the entire point cloud The ratio between meets the threshold condition. , Then the point cloud in the box As the next layer of viewpoint network To target the point cloud, continue to iterate, otherwise terminate the iteration, and finally iterate to generate a multi-layer viewpoint network ; The virtual camera extrinsic parameters corresponding to each layer of the multi-layer viewpoint network are collected to form a virtual camera extrinsic parameter set, where the virtual camera extrinsic parameter corresponding to the mth viewpoint in the virtual camera extrinsic parameter set is .

[0015] Optionally, in the mapping relationship acquisition module, the entire point cloud is obtained based on the intrinsic and extrinsic parameters of the virtual camera corresponding to the m-th viewpoint. The coordinate system of the world coordinate system Rotate the camera coordinate system ,reserve Points with positive Z values ​​in the coordinate system, that is, the point clouds in front of the virtual camera are retained and other point clouds are discarded; From the camera coordinate system Convert to pixel coordinate system , retain the projection points within the image frame; Corresponding to the camera coordinate system based on the projection points in the image frame The lower Z value, i.e. depth, retains the projection point with the smallest depth in the same pixel block and discards other points; Assign the depth or grayscale of the selected 3D point cloud to the corresponding 2D projection point to obtain a one-to-one mapping relationship between the 3D point cloud and the 2D projection point , to achieve the initial rendering of the virtual view ; Initialized rendering ,To address the problem of holes and missing textures in the projected image, the minimum axis-aligned bounding box of all projection points in the image is obtained, and the projection view holes are searched within the bounding box; Extract the neighborhood of each hole, and calculate the grayscale value of the hole based on the grayscale value of the non-hole in the neighborhood using the distance Gaussian weighted method , thereby filling the holes and enriching the texture, realizing the rendering post-processing of the virtual view, and generating the virtual view corresponding to the mth viewpoint ; , in is the pixel coordinate, yes Neighborhood of Represents a virtual view being initialized middle There is value everywhere, Represents pixel Distance on virtual view.

[0016] Optionally, the semantic segmentation module processes the point cloud with semantic information used for training, obtains the intrinsic and extrinsic parameters of the virtual camera, generates a virtual view corresponding to the point cloud with semantic information, and assigns the semantic coding of the three-dimensional point cloud corresponding to the two-dimensional projection point contained in the virtual view to the corresponding two-dimensional projection point, thereby completing the first step of generating a semantic label map; in response to the problem of holes in the semantic label map, obtain the minimum axis-aligned bounding box where all projection points in the semantic label map are located, search for image holes in the bounding box, extract the neighborhood of each hole, and count the semantic features in the neighborhood of each hole based on the semantic information of the semantic label map. The semantic feature corresponding to the largest number of points of a certain semantic feature in the neighborhood is the semantic feature of the point, thereby completing the second step of generating a semantic label map and realizing the generation of the final semantic label map; based on the existing two-dimensional image semantic segmentation algorithm, use the virtual view and the corresponding semantic label map to train and generate a two-dimensional image semantic segmentation model; Virtual View Input the 2D image semantic segmentation model and extract the semantic label map , get the semantic information in the virtual view; Create an initial voting table, initialized to zero, based on the 3D point cloud in each view and the two-dimensional projection point The mapping relationship between And the semantic information extracted from each view , the points that received votes are counted, and the semantic attribute with the highest number of votes at each point is set as the semantic attribute of the point, completing the first vote; A new voting table is created. For each point that did not receive a vote in the initial voting table, its 3D spatial neighborhood is extracted and the points in the neighborhood whose semantic characteristics have been determined in the initial voting results are counted. The semantic characteristic corresponding to the largest number of points of a certain semantic characteristic in the neighborhood is the semantic characteristic of the point, and the second voting is completed. The information contained in the two voting tables is merged to obtain the final voting result, completing the semantic information transmission based on the virtual view semantic label.

[0017] The present invention discloses an electronic device, comprising a processor and a storage medium; the storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the large-scale point cloud semantic segmentation method based on downscale projection as described above.

[0018] The present invention discloses a computer-readable storage medium having a computer program stored thereon, wherein a processor executes the steps of a large-scale point cloud semantic segmentation method based on downscaling projection as described above.

[0019] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: The point cloud semantic segmentation method of the present invention provides a strategy to reduce the quantization loss in the projection process, greatly reduces the information loss in the process of projecting three-dimensional points to views, and improves the fault tolerance of the two-dimensional image semantic segmentation model by generating high-quality projection views and extracting sufficient semantic information. The two-dimensional semantic information is stably transmitted to three dimensions, the semantic segmentation of large-scale and large-scale point clouds is realized, and the accuracy and robustness of point cloud semantic segmentation are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of the process of the present invention; Figure 2 A flowchart of a scheme for setting projection viewpoint and sight direction in the present invention; Figure 3 Schematic diagram of the storage and indexing order of the vertices of the minimum directed bounding box specified in the present invention, wherein the serial numbers 0, 1, 2, 3, 4, 5, 6, 7 represent the index serial numbers of the vertices of the minimum directed bounding box in the array; Figure 4 A flowchart of generating a two-dimensional image semantic segmentation model in the present invention; Figure 5 It is a system schematic diagram of the present invention. DETAILED DESCRIPTION

[0021] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0022] Embodiment 1: This embodiment is recommended in a computer with the following configuration: Intel i5 12400F processor, NVIDIA GeForce RTX 4060 graphics processor, main frequency 3.99GHz, memory 32GB, operating system Windows 11. The implementation of point cloud viewpoint setting is based on the Open3D three-dimensional data processing framework toolkit, the implementation of the semantic segmentation network of point cloud projection image is based on the PyTorch deep learning framework toolkit, and the reasoning of the two-dimensional semantic segmentation of projection image is based on ONNX and ONNX Runtime model reasoning framework toolkit.

[0023] like Figure 1 As shown, this embodiment discloses a large-scale point cloud semantic segmentation method based on downscaling projection, comprising the following steps: (1) If Figure 2 As shown, the point cloud to be processed is obtained, and the downscaled projection viewpoint and line of sight of the virtual view are set according to the spatial distribution of the large-scale point cloud to obtain the external parameters of the virtual camera; Step (1) specifically includes the following steps: (1.1) Read the 3D model file. Take WHU3D data as an example. With the help of pywhu3d, read the h5 data of als type and obtain the point cloud to be processed. ; The 3D model file is obtained by laser radar scanning; in order to generate high-quality multi-view projection of point cloud, a multi-layer viewpoint network is set up ; (1.2) If Figure 3 As shown, get the point cloud corresponding to the k-th layer of viewpoint network currently to be processed The minimum directed bounding box of ,like Figure 1 As shown, determine the vertical direction away from the ground and along the ground in two directions and , a total of three directions , the initial value of k is 0; (1.3) Based on point cloud density Determine the image resolution of the current virtual view , , in is a constant, According to the ground height , Image resolution ,focal length and pixel size The geometric relationship between them determines the ground height of the view projection , , According to image resolution Get the ground coverage of the image , , in For square virtual view Figure 1 The number of pixels on the edge; (1.4) Ground height based on the k-th layer virtual view , Ground Coverage , three directions and overlap , set the viewpoint network of the kth layer , the direction of all viewpoints in the viewpoint network is vertical to the ground , the two directions of the view are the two ground directions of the point cloud to be processed, and the rotation matrix is ; The viewpoints in each layer of the viewpoint network are evenly distributed. Line The viewpoint coordinates of the columns are: , The range of row index values ​​is: , The value range of column index is: , The reference viewpoint coordinates are: , in is the ceiling function, are the eight vertices of the minimum directed bounding box, That is, the vertex with index 0. represents the size of the minimum oriented bounding box, is the length of the minimum directed bounding box, is the width of the minimum directional bounding box; thus, the virtual camera extrinsic parameter corresponding to the viewpoint in the i-th row and j-th column of the k-th viewpoint network is obtained. ; (1.5) Based on processing the ground coverage corresponding to each viewpoint , Ground height , three directions , generate the minimum directed bounding box of all point clouds within the field of view of the viewpoint, and extract the point cloud within the box ; If there is a point cloud in the box and the point cloud density Point cloud density of the point cloud corresponding to the previous viewpoint And the point cloud density of the entire point cloud The ratio between meets the threshold condition. , Then the point cloud in the box As the next layer of viewpoint network Repeat steps (1.2) to (1.5) for the point cloud to be targeted, otherwise terminate the iteration, and finally iterate to generate a multi-layer viewpoint network , the ground height of the viewpoint network in the whole process It gets smaller and smaller, and the distance to the ground point cloud gets closer and closer, and the scale is reduced, that is, downscale projection; (1.6) The virtual camera extrinsic parameters corresponding to each layer of the multi-layer viewpoint network are collected to form a virtual camera extrinsic parameter set, where the virtual camera extrinsic parameter corresponding to the mth viewpoint in the virtual camera extrinsic parameter set is .

[0024] (2) Intrinsic and extrinsic parameters of the virtual camera corresponding to the mth viewpoint , generate the corresponding virtual view , and simultaneously obtain the mapping relationship between the 3D point cloud and the 2D projection points , where the internal parameters of the virtual camera include focal length, pixel size and image size;

[0025] Step (2) specifically includes the following steps: (2.1) Based on the intrinsic and extrinsic parameters of the virtual camera corresponding to the m-th viewpoint, the entire point cloud The coordinate system of the world coordinate system Rotate the camera coordinate system ,reserve Points with positive Z values ​​in the coordinate system, that is, the point clouds in front of the virtual camera are retained and other point clouds are discarded; (2.2) From the camera coordinate system Convert to pixel coordinate system , retain the projection points within the image frame; (2.3) Corresponding to the camera coordinate system based on the projection points in the image frame The lower Z value, i.e. depth, retains the projection point with the smallest depth in the same pixel block and discards other points; (2.4) Assign the depth or grayscale of the selected 3D point cloud to the corresponding 2D projection point to obtain a one-to-one mapping relationship between the 3D point cloud and the 2D projection point. , to achieve the initial rendering of the virtual view ; (2.5) Rendering based on initialization ,To address the problem of holes and missing textures in the projected image, the minimum axis-aligned bounding box of all projection points in the image is obtained, and the projection view holes are searched within the bounding box; (2.6) Extract the neighborhood of each hole, and calculate the grayscale value of the hole based on the grayscale value of the non-hole in the neighborhood using the distance Gaussian weighted method , thereby filling the holes and enriching the texture, realizing the rendering post-processing of the virtual view, and generating the virtual view corresponding to the mth viewpoint ; , in is the pixel coordinate, yes Neighborhood of Represents a virtual view being initialized middle There is value everywhere, Represents pixel Distance on virtual view.

[0026] (3) Extracting semantic information from virtual views based on existing 2D image semantic segmentation algorithms , based on the mapping relationship between the acquired 3D point cloud and the 2D projection points , the semantic information Transfer from two dimensions to three dimensions to complete the semantic segmentation of point cloud.

[0027] Step (3) specifically includes the following steps: (3.1) Process the point cloud with semantic information for training according to steps (1) and (2), obtain the intrinsic parameters and extrinsic parameters of the virtual camera, generate a virtual view corresponding to the point cloud with semantic information, assign the semantic code of the three-dimensional point cloud corresponding to the two-dimensional projection point contained in the virtual view to the corresponding two-dimensional projection point, and complete the first step of generating the semantic label map; in order to solve the problem of holes in the semantic label map, obtain the minimum axis-aligned bounding box where all the projection points in the semantic label map are located, search for image holes in the bounding box, extract the neighborhood of each hole, and count the semantic features in the neighborhood of each hole based on the semantic information of the semantic label map. The semantic feature corresponding to the largest number of points of a certain semantic feature in the neighborhood is the semantic feature of the point, and complete the second step of generating the semantic label map, thereby realizing the generation of the final semantic label map; based on the existing two-dimensional image semantic segmentation algorithm, use the virtual view and the corresponding semantic label map to train and generate a two-dimensional image semantic segmentation model. The two-dimensional image semantic segmentation model can use the onnx format, which is more efficient and occupies less space during model inference; (3.2) The virtual view of step (2) Input the 2D image semantic segmentation model and extract the semantic label map , get the semantic information in the virtual view; (3.3) Create an initial voting table, initialized to zero, based on the 3D point cloud in each view and the two-dimensional projection point The mapping relationship between And the semantic information extracted from each view , the points that received votes are counted, and the semantic attribute with the highest number of votes at each point is set as the semantic attribute of the point, completing the first vote; (3.4) Create a new voting table. For each point that did not receive a vote in the initial voting table, extract its 3D spatial neighborhood and count the points in the neighborhood that have determined the semantic characteristics in the initial voting results. The semantic characteristic corresponding to the point with the largest number of points of a certain semantic characteristic in the neighborhood is the semantic characteristic of the point, completing the second vote. (3.5) The information contained in the two voting tables is merged to obtain the final voting result, completing the semantic information transmission based on the virtual view semantic label and achieving improved accuracy of semantic segmentation of large-scale scene point clouds.

[0028] Example 2: Figure 5 As shown, this embodiment discloses a large-scale point cloud semantic segmentation system based on downscaling projection, including: The external parameter acquisition module is used to obtain the point cloud to be processed, set the downscaled projection viewpoint and line of sight of the virtual view according to the spatial distribution of the large-scale point cloud, and obtain the external parameters of the virtual camera;

[0029] Read the 3D model file in the external parameter acquisition module. Take WHU3D data as an example. With the help of pywhu3d, read the h5 data of als type to obtain the point cloud to be processed. ; The 3D model file is obtained by laser radar scanning; in order to generate high-quality multi-view projection of point cloud, a multi-layer viewpoint network is set up ; like Figure 3 As shown, get the point cloud corresponding to the k-th layer of viewpoint network currently to be processed The minimum directed bounding box of , determine the vertical direction away from the ground and along the ground in two directions and , a total of three directions , the initial value of k is 0; Based on point cloud density Determine the image resolution of the current virtual view , , in is a constant, According to the ground height , Image resolution ,focal length and pixel size The geometric relationship between them determines the ground height of the view projection , , According to image resolution Get the ground coverage of the image , , in For square virtual view Figure 1 The number of pixels on the edge; The ground height based on the k-th layer virtual view , Ground Coverage , three directions and overlap , set the viewpoint network of the kth layer , the direction of all viewpoints in the viewpoint network is vertical to the ground , the two directions of the view are the two ground directions of the point cloud to be processed, and the rotation matrix is ; The viewpoints in each layer of the viewpoint network are evenly distributed. Line The viewpoint coordinates of the columns are: , The range of row index values ​​is: , The value range of column index is: , The reference viewpoint coordinates are: , in is the ceiling function, are the eight vertices of the minimum directed bounding box, That is, the vertex with index 0. represents the size of the minimum oriented bounding box, is the length of the minimum directed bounding box, is the width of the minimum directional bounding box; thus, the virtual camera extrinsic parameter corresponding to the viewpoint in the i-th row and j-th column of the k-th viewpoint network is obtained. ; Based on processing the ground coverage corresponding to each viewpoint , Ground height , three directions , generate the minimum directed bounding box of all point clouds within the field of view of the viewpoint, and extract the point cloud within the box ; If there is a point cloud in the box and the point cloud density Point cloud density of the point cloud corresponding to the previous viewpoint And the point cloud density of the entire point cloud The ratio between meets the threshold condition. , Then the point cloud in the box As the next layer of viewpoint network To target the point cloud, continue to iterate, otherwise terminate the iteration, and finally iterate to generate a multi-layer viewpoint network , the ground height of the viewpoint network in the whole process It gets smaller and smaller, and the distance to the ground point cloud gets closer and closer, and the scale is reduced, that is, downscale projection; The virtual camera extrinsic parameters corresponding to each layer of the multi-layer viewpoint network are collected to form a virtual camera extrinsic parameter set, where the virtual camera extrinsic parameter corresponding to the mth viewpoint in the virtual camera extrinsic parameter set is .

[0030] Mapping relationship acquisition module, used for the intrinsic and extrinsic parameters of the virtual camera corresponding to the mth viewpoint , generate the corresponding virtual view , and simultaneously obtain the mapping relationship between the 3D point cloud and the 2D projection points , where the internal parameters of the virtual camera include focal length, pixel size and image size;

[0031] In the mapping relationship acquisition module, the internal and external parameters of the virtual camera corresponding to the mth viewpoint are used to convert the entire point cloud The coordinate system of the world coordinate system Rotate the camera coordinate system ,reserve Points with positive Z values ​​in the coordinate system, that is, the point clouds in front of the virtual camera are retained and other point clouds are discarded; From the camera coordinate system Convert to pixel coordinate system , retain the projection points within the image frame; Corresponding to the camera coordinate system based on the projection points in the image frame The lower Z value, i.e. depth, retains the projection point with the smallest depth in the same pixel block and discards other points; Assign the depth or grayscale of the selected 3D point cloud to the corresponding 2D projection point to obtain a one-to-one mapping relationship between the 3D point cloud and the 2D projection point , to achieve the initial rendering of the virtual view ; Initialized rendering ,To address the problem of holes and missing textures in the projected image, the minimum axis-aligned bounding box of all projection points in the image is obtained, and the projection view holes are searched within the bounding box; Extract the neighborhood of each hole, and calculate the grayscale value of the hole based on the grayscale value of the non-hole in the neighborhood using the distance Gaussian weighted method , thereby filling the holes and enriching the texture, realizing the rendering post-processing of the virtual view, and generating the virtual view corresponding to the mth viewpoint ; , in is the pixel coordinate, yes Neighborhood of Represents a virtual view being initialized middle There is value everywhere, Represents pixel Distance on virtual view.

[0032] Semantic segmentation module, used to extract semantic information in virtual views based on existing 2D image semantic segmentation algorithms , based on the mapping relationship between the acquired 3D point cloud and the 2D projection points , the semantic information Transfer from two dimensions to three dimensions to complete the semantic segmentation of point cloud.

[0033] The semantic segmentation module processes the point cloud with semantic information used for training, obtains the intrinsic and extrinsic parameters of the virtual camera, generates a virtual view corresponding to the point cloud with semantic information, and assigns the semantic coding of the three-dimensional point cloud corresponding to the two-dimensional projection point contained in the virtual view to the corresponding two-dimensional projection point, completing the first step of generating the semantic label map; in order to solve the problem of holes in the semantic label map, obtain the minimum axis-aligned bounding box where all projection points are located in the semantic label map, search for image holes in the bounding box, extract the neighborhood of each hole, and count the semantic features in the neighborhood of each hole based on the semantic information of the semantic label map. The semantic feature corresponding to the largest number of points of a certain semantic feature in the neighborhood is the semantic feature of the point, completing the second step of generating the semantic label map and realizing the generation of the final semantic label map; based on the existing two-dimensional image semantic segmentation algorithm, use the virtual view and the corresponding semantic label map to train and generate a two-dimensional image semantic segmentation model; Virtual View Input the 2D image semantic segmentation model and extract the semantic label map , get the semantic information in the virtual view; Create an initial voting table, initialized to zero, based on the 3D point cloud in each view and the two-dimensional projection point The mapping relationship between And the semantic information extracted from each view , the points that received votes are counted, and the semantic attribute with the highest number of votes at each point is set as the semantic attribute of the point, completing the first vote; A new voting table is created. For each point that did not receive a vote in the initial voting table, its 3D spatial neighborhood is extracted and the points in the neighborhood whose semantic characteristics have been determined in the initial voting results are counted. The semantic characteristic corresponding to the largest number of points of a certain semantic characteristic in the neighborhood is the semantic characteristic of the point, and the second voting is completed. The information contained in the two voting tables is merged to obtain the final voting result, completing the semantic information transmission based on the virtual view semantic label and achieving improved accuracy of semantic segmentation of large-scale scene point clouds.

[0034] Embodiment 3: Another embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a large-scale point cloud semantic segmentation method based on downscaled projection as described above.

[0035] The electronic device may include: a processor, a memory, a bus and a communication interface, wherein the processor, the communication interface and the memory are connected via a bus; the memory stores a computer program that can be run on the processor, and when the processor runs the computer program, a large-scale point cloud semantic segmentation method based on downscaled projection provided by any of the aforementioned embodiments of the present invention is executed.

[0036] The memory may include high-speed random access memory (RAM), and may also include non-volatile memory, such as at least one disk memory. The communication connection between the device network element and at least one other network element is realized through at least one communication interface (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0037] The bus may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. Among them, the memory is used to store the program, and the processor executes the program after receiving the execution instruction. The large-scale point cloud semantic segmentation method based on downscale projection disclosed in any implementation of the above-mentioned embodiment of the present invention can be applied to the processor, or implemented by the processor.

[0038] The processor may be an integrated circuit chip with signal processing capabilities. Each step of the method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The above-mentioned processor can be a general-purpose processor, which may include a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present invention can be directly embodied as a hardware decoding processor to be executed, or a combination of hardware and software modules in the decoding processor to be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware.

[0039] The electronic device provided in the embodiment of the present application and the method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, operated or implemented by them.

[0040] Embodiment 4: Another embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to implement the method of any of the above-mentioned embodiments. The computer-readable storage medium may be an optical disk on which a computer program (i.e., a program product) is stored, and when the computer program is executed by the processor, the method provided by any of the above-mentioned embodiments will be executed.

[0041] It should be noted that examples of computer-readable storage media may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0042] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0043] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A large-scale point cloud semantic segmentation method based on downscaling projection, characterized in that: The following steps are involved: (1) Obtain the point cloud to be processed, set the downscaled projection viewpoint and line of sight of the virtual view according to the spatial distribution of the large-scale point cloud, and obtain the external parameters of the virtual camera; (2) Generate the corresponding virtual view based on the intrinsic and extrinsic parameters of the virtual camera corresponding to the m-th viewpoint, and obtain the mapping relationship between the 3D point cloud and the 2D projection point. The intrinsic parameters of the virtual camera include focal length, pixel size, and image size. (3) Based on the existing 2D image semantic segmentation algorithm, the semantic information in the virtual view is extracted. Based on the mapping relationship between the acquired 3D point cloud and the 2D projection points, the semantic information is transferred from 2D to 3D to complete the semantic segmentation of the point cloud.

2. The large-scale point cloud semantic segmentation method based on downscaling projection according to claim 1, characterized in that: The step (1) specifically includes the following steps: (1.1) Read the 3D model file and obtain the point cloud to be processed , to generate high-quality multi-view projections of point clouds, a multi-layer viewpoint network is set up ; (1.2) Get the point cloud corresponding to the k-th layer of viewpoint network to be processed The minimum directed bounding box of , determine the vertical direction away from the ground and along the ground in two directions and , a total of three directions , the initial value of k is 0; (1.3) Based on point cloud density Determine the image resolution of the current virtual view , ， in is a constant, According to the ground height , Image resolution ,focal length and pixel size The geometric relationship between them determines the ground height of the view projection , ， According to image resolution Get the ground coverage of the image , ， in is the number of pixels on one side of the square virtual view; (1.4) Ground height based on the k-th layer virtual view , Ground Coverage , three directions and overlap , set the viewpoint network of the kth layer , the direction of all viewpoints in the viewpoint network is vertical to the ground , the two directions of the view are the two ground directions of the point cloud to be processed, and the rotation matrix is ; The viewpoints in each layer of the viewpoint network are evenly distributed. Line The viewpoint coordinates of the columns are: ， The range of row index values ​​is: ， The value range of column index is: ， The reference viewpoint coordinates are: ， in is the ceiling function, are the eight vertices of the minimum directed bounding box, That is, the vertex with index 0. represents the size of the minimum oriented bounding box, is the length of the minimum directed bounding box, is the width of the minimum directional bounding box; thus, the virtual camera extrinsic parameter corresponding to the viewpoint in the i-th row and j-th column of the k-th viewpoint network is obtained. ; (1.5) Based on processing the ground coverage corresponding to each viewpoint , Ground height , three directions , generate the minimum directed bounding box of all point clouds within the field of view of the viewpoint, and extract the point cloud within the box ; If there is a point cloud in the box and the point cloud density Point cloud density of the point cloud corresponding to the previous viewpoint And the point cloud density of the entire point cloud The ratio between meets the threshold condition. ， Then the point cloud in the box As the next layer of viewpoint network Repeat steps (1.2) to (1.5) for the point cloud to be targeted, otherwise terminate the iteration, and finally iterate to generate a multi-layer viewpoint network ; (1.6) The virtual camera extrinsic parameters corresponding to each layer of the multi-layer viewpoint network are collected to form a virtual camera extrinsic parameter set, where the virtual camera extrinsic parameter corresponding to the mth viewpoint in the virtual camera extrinsic parameter set is .

3. The large-scale point cloud semantic segmentation method based on downscaling projection according to claim 1, characterized in that: The step (2) specifically includes the following steps: (2.1) Based on the intrinsic and extrinsic parameters of the virtual camera corresponding to the m-th viewpoint, the entire point cloud The coordinate system of the world coordinate system Rotate the camera coordinate system ,reserve Points with positive Z values ​​in the coordinate system, that is, the point clouds in front of the virtual camera are retained and other point clouds are discarded; (2.2) From the camera coordinate system Convert to pixel coordinate system , retain the projection points within the image frame; (2.3) Corresponding to the camera coordinate system based on the projection points in the image frame The lower Z value, i.e. depth, retains the projection point with the smallest depth in the same pixel block and discards other points; (2.4) Assign the depth or grayscale of the selected 3D point cloud to the corresponding 2D projection point to obtain a one-to-one mapping relationship between the 3D point cloud and the 2D projection point. , to achieve the initial rendering of the virtual view ; (2.5) Rendering based on initialization ,To address the problem of holes and missing textures in the projected image, the minimum axis-aligned bounding box of all projection points in the image is obtained, and the projection view holes are searched within the bounding box; (2.6) Extract the neighborhood of each hole, and calculate the grayscale value of the hole based on the grayscale value of the non-hole in the neighborhood using the distance Gaussian weighted method , thereby filling the holes and enriching the texture, realizing the rendering post-processing of the virtual view, and generating the virtual view corresponding to the mth viewpoint ; ， in is the pixel coordinate, yes Neighborhood of Represents a virtual view being initialized middle There is value everywhere, Represents pixel Distance on virtual view.

4. The large-scale point cloud semantic segmentation method based on downscaling projection according to claim 1, characterized in that: The step (3) specifically includes the following steps: (3.1) Process the point cloud with semantic information used for training according to steps (1) and (2), obtain the intrinsic parameters and extrinsic parameters of the virtual camera, generate a virtual view corresponding to the point cloud with semantic information, assign the semantic coding of the three-dimensional point cloud corresponding to the two-dimensional projection point contained in the virtual view to the corresponding two-dimensional projection point, and complete the first step of generating the semantic label map; in order to solve the problem of holes in the semantic label map, obtain the minimum axis-aligned bounding box where all the projection points in the semantic label map are located, search for image holes in the bounding box, extract the neighborhood of each hole, and count the semantic features in the neighborhood of each hole based on the semantic information of the semantic label map. The semantic feature corresponding to the largest number of points of a certain semantic feature in the neighborhood is the semantic feature of the point, and complete the second step of generating the semantic label map, thereby realizing the generation of the final semantic label map; based on the existing two-dimensional image semantic segmentation algorithm, use the virtual view and the corresponding semantic label map to train and generate a two-dimensional image semantic segmentation model; (3.2) The virtual view of step (2) Input the 2D image semantic segmentation model and extract the semantic label map , get the semantic information in the virtual view; (3.3) Create an initial voting table, initialized to zero, based on the 3D point cloud in each view and the two-dimensional projection point The mapping relationship between And the semantic information extracted from each view , the points that received votes are counted, and the semantic attribute with the highest number of votes at each point is set as the semantic attribute of the point, completing the first vote; (3.4) Create a new voting table. For each point that did not receive a vote in the initial voting table, extract its 3D spatial neighborhood and count the points in the neighborhood that have determined the semantic characteristics in the initial voting results. The semantic characteristic corresponding to the point with the largest number of points of a certain semantic characteristic in the neighborhood is the semantic characteristic of the point, completing the second vote. (3.5) The information contained in the two voting tables is merged to obtain the final voting result, completing the semantic information transmission based on the virtual view semantic label.

5. A large-scale point cloud semantic segmentation system based on downscaling projection, characterized in that: include: The external parameter acquisition module is used to obtain the point cloud to be processed, set the downscaled projection viewpoint and line of sight of the virtual view according to the spatial distribution of the large-scale point cloud, and obtain the external parameters of the virtual camera; A mapping relationship acquisition module is used to generate a corresponding virtual view for the internal parameters and external parameters of the virtual camera corresponding to the m-th viewpoint, and to obtain the mapping relationship between the three-dimensional point cloud and the two-dimensional projection point, wherein the internal parameters of the virtual camera include focal length, pixel size and image size; The semantic segmentation module is used to extract semantic information in the virtual view based on the existing 2D image semantic segmentation algorithm, and transfer the semantic information from 2D to 3D based on the mapping relationship between the acquired 3D point cloud and the 2D projection points to complete the semantic segmentation of the point cloud.

6. The large-scale point cloud semantic segmentation system based on downscaling projection according to claim 5, characterized in that: The external parameter acquisition module reads the three-dimensional model file to obtain the point cloud to be processed ; The 3D model file is obtained by laser radar scanning; in order to generate high-quality multi-view projection of point cloud, a multi-layer viewpoint network is set up ; Get the point cloud corresponding to the k-th layer of viewpoint network currently being processed The minimum directed bounding box of , determine the vertical direction away from the ground and along the ground in two directions and , a total of three directions , the initial value of k is 0; Based on point cloud density Determine the image resolution of the current virtual view , ， in is a constant, According to the ground height , Image resolution ,focal length and pixel size The geometric relationship between them determines the ground height of the view projection , ， According to image resolution Get the ground coverage of the image , ， in is the number of pixels on one side of the square virtual view; The ground height based on the k-th layer virtual view , Ground Coverage , three directions and overlap , set the viewpoint network of the kth layer , the direction of all viewpoints in the viewpoint network is vertical to the ground , the two directions of the view are the two ground directions of the point cloud to be processed, and the rotation matrix is ; The viewpoints in each layer of the viewpoint network are evenly distributed. Line The viewpoint coordinates of the columns are: ， The range of row index values ​​is: ， The value range of column index is: ， The reference viewpoint coordinates are: ， in is the ceiling function, are the eight vertices of the minimum directed bounding box, That is, the vertex with index 0. represents the size of the minimum oriented bounding box, is the length of the minimum directed bounding box, is the width of the minimum directional bounding box; thus, the virtual camera extrinsic parameter corresponding to the viewpoint in the i-th row and j-th column of the k-th viewpoint network is obtained. ; Based on processing the ground coverage corresponding to each viewpoint , Ground height , three directions , generate the minimum directed bounding box of all point clouds within the field of view of the viewpoint, and extract the point cloud within the box ; If there is a point cloud in the box and the point cloud density Point cloud density of the point cloud corresponding to the previous viewpoint And the point cloud density of the entire point cloud The ratio between meets the threshold condition. ， Then the point cloud in the box As the next layer of viewpoint network To target the point cloud, continue to iterate, otherwise terminate the iteration, and finally iterate to generate a multi-layer viewpoint network ; The virtual camera extrinsic parameters corresponding to each layer of the multi-layer viewpoint network are collected to form a virtual camera extrinsic parameter set, where the virtual camera extrinsic parameter corresponding to the mth viewpoint in the virtual camera extrinsic parameter set is .

7. The large-scale point cloud semantic segmentation system based on downscaling projection according to claim 5, characterized in that: The mapping relationship acquisition module converts the entire point cloud into The coordinate system of the world coordinate system Rotate the camera coordinate system ,reserve Points with positive Z values ​​in the coordinate system, that is, the point clouds in front of the virtual camera are retained and other point clouds are discarded; From the camera coordinate system Convert to pixel coordinate system , retain the projection points within the image frame; Corresponding to the camera coordinate system based on the projection points in the image frame The lower Z value, i.e. depth, retains the projection point with the smallest depth in the same pixel block and discards other points; Assign the depth or grayscale of the selected 3D point cloud to the corresponding 2D projection point to obtain a one-to-one mapping relationship between the 3D point cloud and the 2D projection point , to achieve the initial rendering of the virtual view ; Initialized rendering ,To address the problem of holes and missing textures in the projected image, the minimum axis-aligned bounding box of all projection points in the image is obtained, and the projection view holes are searched within the bounding box; Extract the neighborhood of each hole, and calculate the grayscale value of the hole based on the grayscale value of the non-hole in the neighborhood using the distance Gaussian weighted method , thereby filling the holes and enriching the texture, realizing the rendering post-processing of the virtual view, and generating the virtual view corresponding to the mth viewpoint ; ， in is the pixel coordinate, yes Neighborhood of Represents a virtual view being initialized middle There is value everywhere, Represents pixel Distance on virtual view.

8. The large-scale point cloud semantic segmentation system based on downscaling projection according to claim 5, characterized in that: The semantic segmentation module processes the point cloud with semantic information for training, obtains the internal and external parameters of the virtual camera, generates a virtual view corresponding to the point cloud with semantic information, assigns the semantic coding of the two-dimensional projection point corresponding to the three-dimensional point cloud contained in the virtual view to the corresponding two-dimensional projection point, and completes the first step of generating the semantic label map; To solve the problem of holes in the semantic label map, the minimum axis-aligned bounding box of all projection points in the semantic label map is obtained, image holes are searched in the bounding box, the neighborhood of each hole is extracted, and the semantic features in the neighborhood of each hole are counted based on the semantic information of the semantic label map. The semantic feature corresponding to the largest number of points of a certain semantic feature in the neighborhood is the semantic feature of the point, completing the second step of semantic label map generation and realizing the generation of the final semantic label map; based on the existing two-dimensional image semantic segmentation algorithm, the virtual view and the corresponding semantic label map are used to train and generate a two-dimensional image semantic segmentation model; Virtual View Input the 2D image semantic segmentation model and extract the semantic label map , get the semantic information in the virtual view; Create an initial voting table, initialized to zero, based on the 3D point cloud in each view and the two-dimensional projection point The mapping relationship between And the semantic information extracted from each view , the points that received votes are counted, and the semantic attribute with the highest number of votes at each point is set as the semantic attribute of the point, completing the first vote; A new voting table is created. For each point that did not receive a vote in the initial voting table, its 3D spatial neighborhood is extracted and the points in the neighborhood whose semantic characteristics have been determined in the initial voting results are counted. The semantic characteristic corresponding to the largest number of points of a certain semantic characteristic in the neighborhood is the semantic characteristic of the point, and the second voting is completed. The information contained in the two voting tables is merged to obtain the final voting result, completing the semantic information transmission based on the virtual view semantic label.

9. An electronic device, characterized in that: including processor and storage medium; The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 4 are implemented.

Images