A method and system for semantic recognition of hyperspectral point cloud data

Abstract

The present application discloses a method and system for semantic recognition of hyperspectral point cloud data. The technical solution of the present application is based on a public semantic representation method for graph data, converts the acquired hyperspectral point cloud data into graph data, and performs hierarchical iterative segmentation processing on the undirected weighted graph data according to a preset tracking segmentation algorithm and a segmentation layer threshold to obtain multi-scale hyperspectral super-point graph data, which can naturally capture the multilateral relationship between points on the point cloud surface and effectively process complex data structures such as spectral and geometric information, thereby effectively improving the point cloud processing efficiency and reducing the computing resources consumed by the network.

Description

Technical Field

[0001] The present application relates to the technical field of remote sensing information processing, and in particular to a method and device for semantic recognition of hyperspectral point cloud data. Background Art

[0002] Hyperspectral technology is a material composition detection technology developed on the basis of remote sensing technology. It collects the full-band reflectance spectrum information of the target object and extracts the spectrum information of the specific material of the target object, so as to achieve the purpose of analyzing the composition of the object. Hyperspectral technology has a high resolution and can accurately classify and identify objects.

[0003] Hyperspectral LiDAR data not only provides rich spectral information, but also has high-precision detailed geometric information such as surface structure and elevation of objects, which can solve the performance bottleneck of object classification based on single-source data. In fact, semantic segmentation based on hyperspectral point cloud data is an important technical means to achieve effective analysis of multi-dimensional information of observed objects. However, due to the huge amount of hyperspectral point cloud data, the network needs to consume a lot of computing resources to process this data. Summary of the invention

[0004] The present application provides a method and device for semantic recognition of hyperspectral point cloud data, which are used to solve the technical problem of high consumption of computing resources in existing semantic analysis and processing of hyperspectral point cloud data.

[0005] In order to solve the above technical problems, the first aspect of the present application provides a method for semantic recognition of hyperspectral point cloud data, comprising:

[0006] Acquire hyperspectral point cloud data;

[0007] Converting the hyperspectral point cloud data into undirected weighted graph data;

[0008] According to a preset tracking segmentation algorithm and a segmentation layer number threshold, the undirected weighted graph data is subjected to a hierarchical iterative segmentation process to obtain multi-scale hyperspectral super-point graph data, wherein the number of super-points in the hyperspectral super-point graphs of different scales decreases layer by layer;

[0009] The multi-scale hyperspectral super-point map data is input into a semantic segmentation model to obtain a three-dimensional object classification result corresponding to the hyperspectral point cloud data through the semantic segmentation model, wherein the semantic segmentation model is an HSP-GN architecture neural network model obtained by training with preset multi-scale hyperspectral super-point map data samples.

[0010] Preferably, converting the hyperspectral point cloud data into super point map data specifically includes:

[0011] Through a preset K-nearest neighbor algorithm, each node corresponding to the hyperspectral point cloud data is connected to several nearest neighboring nodes to form connected super-point graph data.

[0012] Preferably, according to a preset tracking segmentation algorithm and a segmentation layer number threshold, the undirected weighted graph data is subjected to hierarchical iterative segmentation processing to obtain multi-scale hyperspectral super-point graph data, which specifically includes:

[0013] According to the objective function of the tracking segmentation algorithm, the correlation between each node and the adjacent nodes in the super point graph data is calculated;

[0014] According to the correlation ranking results between different nodes, the edges of several pairs of nodes with the lowest correlation are cut, and then several connected subgraphs are obtained based on the remaining nodes;

[0015] Determine the superpoints in each connected subgraph to obtain a superpoint set, and convert the current superpoint set into an undirected weighted graph data store for the next round of image segmentation, until the number of layers of the superpoint graph data reaches the segmentation layer number threshold, and obtain multi-scale hyperspectral superpoint graph data.

[0016] Preferably, the step of determining superpoints in each connected subgraph specifically includes:

[0017] The geometric center of the connected subgraph is determined, and a superpoint corresponding to the connected subgraph is generated at the position of the geometric center.

[0018] Preferably, the objective function of the tracking segmentation algorithm is specifically:

[0019]

[0020]

[0021] In the formula, is the loss function of the geometric information and spectral information within the node, and It represents the spectral information on nodes u and v. is the spectral distance difference between nodes u and v, R1 is the parameter of the spectral difference perception degree, is a parameter used to balance the importance of spatial geometry in the similarity measure, and are the normal vectors of nodes u and v respectively, E is the Euclidean space distance between the two points, is the weight of the edge connecting the node uv, is the penalty intensity coefficient.

[0022] At the same time, the second aspect of the present application provides a hyperspectral point cloud data semantic recognition system, including:

[0023] A point cloud data acquisition unit, used for acquiring hyperspectral point cloud data;

[0024] A super point graph conversion unit, used for converting the hyperspectral point cloud data into undirected weighted graph data;

[0025] An image segmentation processing unit is used to perform hierarchical iterative segmentation processing on the undirected weighted graph data according to a preset tracking segmentation algorithm and a segmentation layer number threshold to obtain multi-scale hyperspectral super-point graph data, wherein the number of super-points in the hyperspectral super-point graphs of different scales decreases layer by layer;

[0026] The semantic recognition unit is used to input the multi-scale hyperspectral super-point map data into a semantic segmentation model to obtain a three-dimensional object classification result corresponding to the hyperspectral point cloud data through the semantic segmentation model, wherein the semantic segmentation model is an HSP-GN architecture neural network model obtained by training with preset multi-scale hyperspectral super-point map data samples.

[0027] Preferably, the superpoint graph conversion unit is specifically used for:

[0028] Through a preset K-nearest neighbor algorithm, each node corresponding to the hyperspectral point cloud data is connected to several nearest neighboring nodes to form connected super-point graph data.

[0029] Preferably, the image segmentation processing unit is specifically used for:

[0030] According to the objective function of the tracking segmentation algorithm, the correlation between each node and the adjacent nodes in the super point graph data is calculated;

[0031] According to the correlation ranking results between different nodes, the edges of several pairs of nodes with the lowest correlation are cut, and then several connected subgraphs are obtained based on the remaining nodes;

[0032] The superpoints in each connected subgraph are determined to obtain a superpoint set, and then the penalty intensity coefficient in the objective function is updated. The undirected weighted graph data is segmented for the next round based on the updated objective function until the number of layers of the superpoint graph data reaches the segmentation layer number threshold, thereby obtaining multi-scale hyperspectral superpoint graph data.

[0033] Preferably, the step of determining superpoints in each connected subgraph specifically includes:

[0034] The geometric center of the connected subgraph is determined, and a superpoint corresponding to the connected subgraph is generated at the position of the geometric center.

[0035] Preferably, the objective function of the tracking segmentation algorithm is specifically:

[0036]

[0037]

[0038] In the formula, is the loss function of the geometric information and spectral information within the node, and It represents the spectral information on nodes u and v. is the spectral distance difference between nodes u and v, R1 is the parameter of the spectral difference perception degree, is a parameter used to balance the importance of spatial geometry in the similarity measure, and are the normal vectors of nodes u and v respectively, E is the Euclidean space distance between the two points, is the weight of the edge connecting the node uv, is the penalty intensity coefficient.

[0039] It can be seen from the above technical solutions that this application has the following advantages:

[0040] The technical solution of the present application is based on a public semantic representation method for graph data. The acquired hyperspectral point cloud data is converted into graph data. According to a preset tracking segmentation algorithm and a segmentation layer threshold, the undirected weighted graph data is subjected to a layered iterative segmentation process to obtain multi-scale hyperspectral super-point graph data. The data can naturally capture the multilateral relationship between points on the point cloud surface and can effectively process complex data structures such as spectral and geometric information, thereby effectively improving the point cloud processing efficiency and reducing the computing resources consumed by the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0042] Figure 1 A schematic diagram of a flow chart of an embodiment of a method for semantic recognition of hyperspectral point cloud data provided in this application.

[0043] Figure 2 This is a flow chart of step 103 in an embodiment of a method for semantic recognition of hyperspectral point cloud data provided in the present application.

[0044] Figure 3 A schematic diagram of the structure of an embodiment of a hyperspectral point cloud data semantic recognition device provided in the present application. DETAILED DESCRIPTION

[0045] The embodiments of the present application provide a method and system for semantic recognition of hyperspectral point cloud data, which are used to solve the technical problem of high consumption of computing resources in existing semantic analysis and processing of hyperspectral point cloud data.

[0046] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described below are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] First, a detailed description of an embodiment of a method for semantic recognition of hyperspectral point cloud data provided by the present application is as follows:

[0048] See also Figure 1 , this embodiment provides a method for semantic recognition of hyperspectral point cloud data, including:

[0049] Step 101: Acquire hyperspectral point cloud data;

[0050] It should be noted that the acquisition of the hyperspectral point cloud mentioned in this embodiment can be directly taken through mature hardware, or it can be obtained by fusing point clouds and hyperspectral images using mature preprocessing technology. The solution of this example preferably uses the position information of hyperspectral images and point cloud data for registration, and assigns spectral information to corresponding points to obtain hyperspectral point cloud data. Among them, since both hyperspectral images and LiDAR data in remote sensing data contain position information, and this position information is based on the world coordinate system, LiDAR point cloud data and hyperspectral images are respectively determined by the (x, y, z) coordinate values ​​of the points and the coordinates of the upper left corner of the image and the distance between pixels. The x, y position of the point cloud can be used to calculate the corresponding rows and columns (r, c) in the hyperspectral image, expressed as follows:

[0051]

[0052] Where (X, Y) represents the coordinates of the upper left corner of the optical image, and gsd represents the spatial resolution of the optical image.

[0053] After aligning the positions of the point cloud and the hyperspectral image, a hyperspectral point cloud set is generated by embedding the spectral information vector of the corresponding position into the point in the LiDAR point cloud p ,in, , , n is the number of geometric interior points, is the spectral information of the hyperspectral image, and c is the number of bands of the spectral information.

[0054] Step 102, converting the hyperspectral point cloud data into undirected weighted graph data;

[0055] It should be noted that in step 102, based on the hyperspectral point cloud data obtained in step 101, the input hyperspectral point set is converted into graph data. More specifically, this embodiment uses the KNN (k-Nearest Neighbors) algorithm to connect each point to the k nearest neighboring points to form a connected undirected weighted graph. , where V is the node set of the graph, E is the edge set between the nodes, and w is the weight of the edge. In this embodiment, the weight of each edge can be set to 1 by default.

[0056] Step 103: performing layered iterative segmentation processing on the undirected weighted graph data according to a preset tracking segmentation algorithm and a segmentation layer number threshold, to obtain multi-scale hyperspectral super-point graph data, wherein the number of super-points in the hyperspectral super-point graphs of different scales decreases layer by layer;

[0057] It should be noted that, in step 103, based on the graph data obtained in the previous step, the segmentation algorithm and the segmentation layer number threshold are tracked, and the undirected weighted graph data is subjected to hierarchical iterative segmentation processing to obtain a subgraph with a more detailed structure and features;

[0058] More specifically, if Figure 2 As shown, in step 103 of this embodiment, the steps may specifically include:

[0059] Step 1031, calculating the correlation between each node and adjacent nodes in the superpoint graph data according to the objective function of the tracking segmentation algorithm;

[0060] Step 1032: based on the correlation ranking results between different nodes, cut the edges of several pairs of nodes with the lowest correlation, and then obtain several connected subgraphs based on the remaining nodes;

[0061] Step 1033, determine the super points in each connected subgraph to obtain a super point set, then update the penalty intensity coefficient in the objective function, and then perform the next round of segmentation on the undirected weighted graph data based on the updated objective function until the number of layers of the super point graph data reaches the segmentation layer threshold, and obtain multi-scale hyperspectral super point graph data.

[0062] It should be noted that the tracking segmentation algorithm of this embodiment preferably adopts the Parallel Cut Pursuit segmentation algorithm, which evaluates the correlation between different nodes in the graph based on the objective function and selectively cuts the edges with less correlation. Through multiple iterations, the algorithm gradually blocks the nodes with less correlation, so that the nodes with strong correlation form a connected subgraph. . The maximum number of iterations of the algorithm is 15 to ensure that the segmentation is completed within a reasonable time. In addition, if during the iteration process, when the constant component loss value of the divided connected subgraph reaches the expected value, the loop can be exited in advance to ensure that the number of subgraphs meets the demand. In addition, in order to avoid overly aggressive pruning that damages edge features, the algorithm of this embodiment can set a protection radius r. The adjacent nodes of each node and the edges within this radius will not be pruned, thereby retaining local important structures and features. For the tracking segmentation algorithm provided in the described embodiment, its objective function introduces signal strength to quantitatively evaluate the quality of the segmentation. In this way, the optimal graph partitioning problem is transformed into a signal strength minimization problem. The specific expression is as follows:

[0063]

[0064]

[0065] in, It considers both the geometric information and spectral information within the node The loss function represents the signal strength. It is a spectral information difference measure introduced to measure the spectral differences between nodes in the graph, where: and It represents the spectral information on nodes u and v. is the spectral distance difference between nodes u and v, and R1 is the parameter of the degree of perception of spectral difference; similarly, It is a geometric metric introduced to measure the geometric information between nodes in the graph, where: is a parameter used to balance the importance of spatial geometry in the similarity measure, and are the normal vectors of nodes u and v respectively, E is the Euclidean space distance between two points, and is a parameter used to balance the importance of spatial geometry in the similarity metric, and R2 is a parameter used to control the degree of perception of spatial differences during segmentation. The role of is to give dissimilar points or components a larger penalty value to guide the optimization of the algorithm group structure, so as to achieve the purpose of controlling the degree of refinement. Specifically, a set of non-negative regularization strengths is designed. , The higher the value, the higher the penalty strength, resulting in fewer superpoints and coarser components, thereby obtaining superpoint graph data of different scales. is the pseudo-l0 norm, returns 0 if x=0, otherwise returns 1; w is the weight of the edge connecting node uv.

[0066] After obtaining the connected graph, superpoints are generated on the connected subgraphs. The geometric center position of , expressed as ,in is an aggregation function. The aggregation function used in the present invention is a summation function to obtain a superpoint set , then, the generated superpoint set As the input for generating the next layer of superpoint graph, the process of generating graph, segmenting and generating superpoints, and obtaining a superpoint set is repeated. At the same time, the number of clippings in each round of the tracking segmentation algorithm is reduced layer by layer, and finally a superpoint graph with a multi-scale pyramid structure from fine to coarse is generated. Assuming that the target number of layers set in this embodiment is three layers, the superpoint graph obtained through the above processing process is denoted as SPG={G1,G2,G3}.

[0067] Step 104: input the multi-scale hyperspectral super-point map data into the semantic segmentation model to obtain the three-dimensional object classification result corresponding to the hyperspectral point cloud data through the semantic segmentation model; wherein the semantic segmentation model is an HSP-GN architecture graph neural network model obtained by training with preset multi-scale hyperspectral super-point map data samples.

[0068] It should be noted that the multi-scale hyperspectral super-point map data is input into the trained semantic segmentation model to perform semantic segmentation on the obtained hyperspectral super-point map through the semantic segmentation model to obtain the three-dimensional object classification result corresponding to the hyperspectral point cloud data.

[0069] More specifically, the semantic segmentation model of this embodiment is an HSP-GN architecture neural network model obtained by training with preset multi-scale hyperspectral super-point graph data samples. Regarding the construction of the HSP-GN architecture, the HSP-GN that expands the three-dimensional graph neural network uses a super-point graph structure to construct a multi-layer encoder-decoder network to extract and fuse super-point graph features respectively. The encoder network consists of three encoding layers, corresponding to three layers of super-point graph data. Each encoding layer not only accepts the corresponding super-point graph data, but also fuses with the encoding results of the previous encoding layer for learning and feature representation. The decoder network consists of three decoding layers, corresponding to the encoder network of the three encoding layers. The encoder performs encoding three times, so the decoder network performs prediction through three reverse reconstructions. Finally, the decoding and reconstruction results of the third layer are used to obtain the three-dimensional semantic segmentation results through the Softmax classifier.

[0070] The encoder and decoder structures of each layer are similar, consisting of fusion and feature processing modules. The feature processing module is responsible for feature extraction in the encoding stage and reconstruction prediction in the decoding stage. The feature processing module processes the superpoint graph features by MLP-Max and SuperPoint Transformer (Transformer module for point cloud). CatFusion is used as a fusion module to fuse the local superpoint graph with the encoding result of the previous layer. The specific structure is: CatFusion → MLP-Max → Transformer → GraphNorm.

[0071] For the input multi-scale hierarchical structure of the hyperspectral superpoint map and the above-mentioned HSP-GN multi-scale graph segmentation network structure, the HSP-GN model of this embodiment designs the corresponding dual loss function to calculate the loss of the finest level superpoint map and other layers of superpoint maps respectively. In the multi-scale superpoint map data, the G1 layer superpoint has the most complete original information and is closest to the true label; while the superpoints of other layers usually contain multiple objects, and the point distribution inside the superpoint is complex, making it difficult to predict labels. Therefore, we predict the object category of the G1 layer superpoint map, and the distribution of objects in the superpoints of other layers is predicted. The dual loss function designed for the two learning objectives calculates the following losses respectively: (1) The loss of the object classification result predicted by the superpoint of the finest boundary of the G1 layer and the true label. (2) The loss of the label distribution predicted by the coarser level superpoint map of Gi (i>1) and the true label distribution. Capture the diversity inside the superpoint and provide richer and more accurate information.

[0072] This embodiment uses the following cross entropy loss to supervise the two learning objectives respectively. The loss function can be expressed as follows:

[0073]

[0074]

[0075] Specific design Denotes a superpoint p∈G i The true label distribution of the 3D points in the image is obtained by using a dedicated linear layer at each partition level to map the decoder features to the predicted label distribution. . is the one-hot encoding of the most frequent label among its true labels, is the predicted label content of the super point. Where μ is a positive weight, Denotes a superpoint p∈G i is the number of points in the hyperspectral point cloud data, and |C| is the total number of points in the hyperspectral point cloud data, and K is the category set.

[0076] Finally, the codec parameters of each layer of the HSP-GN network are shown in Table 1. is the data geometry at each scale, is the original hyperspectral point cloud data, when hour is the hyperspectral super-point map data. The numbers in [·] are the hidden layer dimensions of each feature extraction module in HSP-GN.

[0077] Table 1. Codec parameters of each layer of HSP-GN network for semantic segmentation model:

[0078]

[0079] The above is a detailed description of an embodiment of a method for semantic recognition of hyperspectral point cloud data provided by the present application. The scheme of the present application first obtains hyperspectral point cloud data through the existing hyperspectral image and point cloud fusion technology (or hyperspectral lidar imaging equipment); then obtains hyperspectral superpoint graph data from the hyperspectral point cloud data through the superpoint graph composition algorithm as the input data of the network; and then uses a three-dimensional graph neural network to classify the superpoint graph containing hyperspectral information to obtain the final three-dimensional object classification result. The present invention uses the data structure of the superpoint graph to greatly reduce the number of points in the data and effectively improve the efficiency of point cloud processing. In addition, the present application uses the public semantic representation method of the graph to naturally capture the multilateral relationship between points on the point cloud surface, obtain better classification effects, and explain the connection between the spectral information and geometric information of the object.

[0080] The following is a detailed description of an embodiment of a hyperspectral point cloud data semantic recognition system provided by the present application.

[0081] See also Figure 3 The second aspect of the present application provides a hyperspectral point cloud data semantic recognition system, comprising:

[0082] A point cloud data acquisition unit 201 is used to acquire hyperspectral point cloud data;

[0083] A super point graph conversion unit 202, used to convert the hyperspectral point cloud data into undirected weighted graph data;

[0084] The image segmentation processing unit 203 is used to perform hierarchical iterative segmentation processing on the undirected weighted graph data according to a preset tracking segmentation algorithm and a segmentation layer number threshold, so as to obtain multi-scale hyperspectral super-point graph data, wherein the number of super-points in the hyperspectral super-point graphs of different scales decreases layer by layer;

[0085] The semantic recognition unit 204 is used to input the multi-scale hyperspectral super-point map data into the semantic segmentation model to obtain the three-dimensional object classification result corresponding to the hyperspectral point cloud data through the semantic segmentation model, wherein the semantic segmentation model is an HSP-GN architecture neural network model obtained by training with preset multi-scale hyperspectral super-point map data samples.

[0086] Furthermore, the superpoint graph conversion unit is specifically used for:

[0087] Through the preset K-nearest neighbor algorithm, each node corresponding to the hyperspectral point cloud data is connected to several nearest adjacent nodes to form a connected super-point graph data.

[0088] Furthermore, the image segmentation processing unit is specifically used for:

[0089] According to the objective function of the pursuit segmentation algorithm, the correlation between each node and the adjacent nodes in the superpoint graph data is calculated;

[0090] According to the correlation ranking results between different nodes, the edges of several pairs of nodes with the lowest correlation are cut, and then several connected subgraphs are obtained based on the remaining nodes;

[0091] The superpoints in each connected subgraph are determined to obtain a superpoint set, and then the penalty intensity coefficient in the objective function is updated. The undirected weighted graph data is segmented for the next round based on the updated objective function until the number of layers of the superpoint graph data reaches a threshold of the number of segmentation layers, thereby obtaining multi-scale hyperspectral superpoint graph data.

[0092] Furthermore, determining the superpoints in each connected subgraph specifically includes:

[0093] Determine the geometric center of the connected subgraph, and generate the superpoint corresponding to the connected subgraph at the position of the geometric center.

[0094] Furthermore, the objective function of the tracking segmentation algorithm is specifically:

[0095]

[0096]

[0097] In the formula, is the loss function of the geometric information and spectral information within the node, and It represents the spectral information on nodes u and v. is the spectral distance difference between nodes u and v, R1 is the parameter of the spectral difference perception degree, is a parameter used to balance the importance of spatial geometry in the similarity measure, and are the normal vectors of nodes u and v respectively, E is the Euclidean space distance between the two points, is the weight of the edge connecting the node uv, is the penalty intensity coefficient.

[0098] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the terminals, systems and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0099] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection of some interfaces, systems or units, which can be electrical, mechanical or other forms.

[0100] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0101] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0102] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0103] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0104] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0105] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for semantic recognition of hyperspectral point cloud data, characterized in that: include: Acquire hyperspectral point cloud data; Converting the hyperspectral point cloud data into undirected weighted graph data; Calculating the correlation between each node and adjacent nodes in the undirected weighted graph data according to the objective function of the tracking segmentation algorithm; According to the correlation ranking results between different nodes, the edges of several pairs of nodes with the lowest correlation are cut, and then several connected subgraphs are obtained based on the remaining nodes; Determine the superpoints in each connected subgraph to obtain a superpoint set, then update the penalty intensity coefficient in the objective function, and then perform the next round of segmentation on the undirected weighted graph data based on the updated objective function until the number of layers of the superpoint graph data is equal to the segmentation layer number threshold, thereby obtaining multi-scale hyperspectral superpoint graph data, wherein the number of superpoints in hyperspectral superpoint graphs of different scales decreases layer by layer; The multi-scale hyperspectral super-point map data is input into a semantic segmentation model to obtain a three-dimensional object classification result corresponding to the hyperspectral point cloud data through the semantic segmentation model, wherein the semantic segmentation model is an HSP-GN architecture neural network model obtained by training with preset multi-scale hyperspectral super-point map data samples.

2. The method for semantic recognition of hyperspectral point cloud data according to claim 1, characterized in that: Converting the hyperspectral point cloud data into undirected weighted graph data specifically includes: Through a preset K-nearest neighbor algorithm, each node corresponding to the hyperspectral point cloud data is connected to several nearest neighboring nodes to form connected undirected weighted graph data.

3. The method for semantic recognition of hyperspectral point cloud data according to claim 1, characterized in that: Determining the super points in each connected subgraph specifically includes: The geometric center of the connected subgraph is determined, and a superpoint corresponding to the connected subgraph is generated at the position of the geometric center.

4. The method for semantic recognition of hyperspectral point cloud data according to claim 1, characterized in that: The objective function of the tracking segmentation algorithm is specifically: ； ； In the formula, is the loss function of the geometric information and spectral information within the node, and It represents the spectral information on nodes u and v. is the spectral distance difference between nodes u and v, R1 is the parameter of the spectral difference perception degree, is a parameter used to balance the importance of spatial geometry in the similarity measure, and are the normal vectors of nodes u and v respectively, E(u,v) is the Euclidean space distance between nodes u and v, is the weight of the edge connecting the node uv, is the penalty strength coefficient, and R2 is a parameter used to control the perceived degree of spatial differences during segmentation.

5. A hyperspectral point cloud data semantic recognition system, characterized in that: include: A point cloud data acquisition unit, used for acquiring hyperspectral point cloud data; A super point graph conversion unit, used for converting the hyperspectral point cloud data into undirected weighted graph data; The image segmentation processing unit is used to: calculate the correlation between each node and the adjacent nodes in the undirected weighted graph data according to the objective function of the tracking segmentation algorithm, cut the edges of several pairs of nodes with the lowest correlation according to the correlation sorting results between different nodes, obtain several connected subgraphs based on the remaining nodes, determine the superpoints in each connected subgraph, and obtain a superpoint set, then update the penalty intensity coefficient in the objective function, and then perform the next round of segmentation on the undirected weighted graph data based on the updated objective function until the number of layers of the superpoint graph data is equal to the segmentation layer number threshold, so as to obtain multi-scale hyperspectral superpoint graph data, wherein the number of superpoints in hyperspectral superpoint graphs of different scales decreases layer by layer; The semantic recognition unit is used to input the multi-scale hyperspectral super-point map data into a semantic segmentation model to obtain a three-dimensional object classification result corresponding to the hyperspectral point cloud data through the semantic segmentation model, wherein the semantic segmentation model is an HSP-GN architecture neural network model obtained by training with preset multi-scale hyperspectral super-point map data samples.

6. A hyperspectral point cloud data semantic recognition system according to claim 5, characterized in that: The superpoint graph conversion unit is specifically used for: Through a preset K-nearest neighbor algorithm, each node corresponding to the hyperspectral point cloud data is connected to several nearest neighboring nodes to form connected undirected weighted graph data.

7. The hyperspectral point cloud data semantic recognition system according to claim 5, characterized in that: Determining the super points in each connected subgraph specifically includes: The geometric center of the connected subgraph is determined, and a superpoint corresponding to the connected subgraph is generated at the position of the geometric center.

8. The hyperspectral point cloud data semantic recognition system according to claim 5, characterized in that: The objective function of the tracking segmentation algorithm is specifically: ； ； In the formula, is the loss function of the geometric information and spectral information within the node, and It represents the spectral information on nodes u and v. is the spectral distance difference between nodes u and v, R1 is the parameter of the spectral difference perception degree, is a parameter used to balance the importance of spatial geometry in the similarity measure, and are the normal vectors of nodes u and v respectively, E(u,v) is the Euclidean space distance between nodes u and v, is the weight of the edge connecting the node uv, is the penalty strength coefficient, and R2 is a parameter used to control the perceived degree of spatial differences during segmentation.

Images