Semantic Segmentation Method, Apparatus, Device and Storage Medium for Multi-Source Heterogeneous Point Clouds

Abstract

The present disclosure provides a semantic segmentation method, apparatus, device, and storage medium for multi-source heterogeneous point clouds. The method includes: acquiring image data of a target area and point cloud data from each source; extracting the attribute features of the point cloud data from each source, and using a pre-constructed attribute alignment model to perform attribute alignment and matching on the point cloud data from each source to obtain the matching results of the point cloud data from each source; fusing the matching results of the point cloud data from each source to obtain fused point cloud data; and performing semantic segmentation on the fused point cloud data based on the image data to obtain the semantic segmentation result of the target area. The method of the present disclosure can improve the semantic segmentation accuracy of multi-source heterogeneous point cloud data.

Description

Technical Field

[0001] The present disclosure relates to the technical field of data processing, and in particular, to a semantic segmentation method, apparatus, device, and storage medium for multi-source heterogeneous point clouds. Background Art

[0002] Point cloud semantic segmentation is to divide point cloud data into different semantic categories, such as buildings, roads, vehicles, vegetation, etc. Its function is to achieve a fine semantic judgment for each point in the point cloud to obtain more detailed semantic structure information, and provide rich semantic features for detection, recognition, modeling, and interaction, etc.

[0003] Due to environmental limitations, in order to obtain more scene information in a region, point cloud data is usually collected by different devices, so that the point cloud data in this region has the characteristics of multi-source heterogeneity. And the current semantic segmentation technology mainly focuses on processing point cloud data from a single source, and the semantic segmentation results of multi-source heterogeneous point cloud data are not accurate. Therefore, how to improve the semantic segmentation accuracy of multi-source heterogeneous point cloud data is a technical problem that those skilled in the art need to solve urgently. Summary of the Invention [[ID=!15]]

[0004] In view of this, the present disclosure provides a semantic segmentation method, apparatus, device, and storage medium for multi-source heterogeneous point clouds, which can improve the semantic segmentation accuracy of multi-source heterogeneous point cloud data.

[0005] According to a first aspect of the present disclosure, there is provided a semantic segmentation method for multi-source heterogeneous point clouds, including:

[0006] Obtain the image data of the target area and the point cloud data of each source;

[0007] Extract the attribute features of the point cloud data of each source, and use a pre-constructed attribute alignment model to align and match the point cloud data of each source to obtain the matching results of the point cloud data of each source;

[0008] Fuse the matching results of the point cloud data of each source to obtain fused point cloud data;

[0009] Based on the image data, perform semantic segmentation on the fused point cloud data to obtain the semantic segmentation result of the target area.

[0010] In a possible implementation manner, when extracting the attribute features of the point cloud data of each source, using a pre-constructed attribute alignment model to align and match the point cloud data of each source to obtain the matching results of the point cloud data of each source, it includes:

[0011] Convert the attribute features of the point cloud data of each source into standard attribute features to obtain the standard attribute features of the point cloud data of each source;

[0012] Match each source point cloud data based on the standard attribute features of each source point cloud data to obtain the matching results of each source point cloud data.

[0013] In a possible implementation manner, when matching each source point cloud data based on the standard attribute features of each source point cloud data to obtain the matching results of each source point cloud data, it includes:

[0014] Determine the target point cloud data from each source point cloud data;

[0015] Traverse each point cloud in the target point cloud data;

[0016] For the currently traversed point cloud, calculate the similarity between the standard attribute features of the currently traversed point cloud and each point cloud in other source point cloud data in sequence, and use the point cloud in other source point cloud data with the similarity greater than the set threshold as the point cloud matching the currently traversed point cloud;

[0017] After the traversal ends, the matching results of each source point cloud data can be obtained.

[0018] In a possible implementation manner, when calculating the similarity between the standard attribute features of the currently traversed point cloud and each point cloud in other source point cloud data, it is implemented based on a preset attribute feature weight.

[0019] In a possible implementation manner, the attribute feature weight is updated according to the similarity between the standard attribute features of the currently traversed point cloud and each point cloud in other source point cloud data.

[0020] In a possible implementation manner, when performing semantic segmentation on the fused point cloud data based on the image data to obtain the semantic segmentation result of the target area, it includes:

[0021] Perform semantic analysis on the image data to obtain a two-dimensional semantic mask of the image data;

[0022] Convert the two-dimensional semantic mask into a three-dimensional semantic mask and align the three-dimensional semantic mask with the fused point cloud data;

[0023] Based on the three-dimensional semantic mask aligned with the fused point cloud data, perform semantic segmentation on the fused point cloud data to obtain the semantic segmentation result of the target area.

[0024] In a possible implementation manner, when performing semantic analysis on the image data to obtain a two-dimensional semantic mask of the image data, it is implemented based on the SAM2 model.

[0025] According to the second aspect of the present disclosure, there is provided a semantic segmentation method device for multi-source heterogeneous point clouds, including:

[0026] A data acquisition module, configured to acquire image data of a target area and point cloud data from each source;

[0027] An attribute alignment module, configured to extract attribute features of the point cloud data from each source, and use a pre-constructed attribute alignment model to perform attribute alignment and matching on the point cloud data from each source, so as to obtain a matching result of the point cloud data from each source;

[0028] A point cloud fusion module, configured to fuse the matching results of the point cloud data from each source to obtain fused point cloud data;

[0029] A semantic segmentation module, configured to perform semantic segmentation on the fused point cloud data based on the image data to obtain a semantic segmentation result of the target area.

[0030] According to a third aspect of the present disclosure, there is provided a device for a semantic segmentation method of multi-source heterogeneous point clouds, including: a processor; a memory for storing processor-executable instructions; wherein, the processor is configured to execute the method described in the first aspect of the present disclosure.

[0031] According to a fourth aspect of the present disclosure, there is provided a non-volatile computer-readable storage medium, on which computer program instructions are stored, wherein, when the computer program instructions are executed by a processor, the method described in the first aspect of the present disclosure is implemented.

[0032] The present disclosure provides a semantic segmentation method, device, equipment and storage medium for multi-source heterogeneous point clouds. The method includes: acquiring image data of a target area and point cloud data from each source; extracting attribute features of the point cloud data from each source, and using a pre-constructed attribute alignment model to perform attribute alignment and matching on the point cloud data from each source, so as to obtain a matching result of the point cloud data from each source; fusing the matching results of the point cloud data from each source to obtain fused point cloud data; performing semantic segmentation on the fused point cloud data based on the image data to obtain a semantic segmentation result of the target area. Through the method of the present disclosure, the accuracy of semantic segmentation of multi-source heterogeneous point cloud data can be improved.

[0033] According to the following detailed description of exemplary embodiments with reference to the accompanying drawings, other features and aspects of the present disclosure will become clear. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings included in the specification and constituting a part of the specification, together with the specification, illustrate exemplary embodiments, features and aspects of the present disclosure, and are used to explain the principles of the present disclosure.

[0035] Figure 1 A flowchart showing a semantic segmentation method of multi-source heterogeneous point clouds according to an embodiment of the present disclosure;

[0036] Figure 2 Shows the reservoir water surface point cloud data according to an embodiment of the present disclosure;

[0037] Figure 3 Shows the reservoir underwater point cloud data according to an embodiment of the present disclosure;

[0038] Figure 4 Shows the semantic segmentation results of the reservoir water surface point cloud data and the underwater point cloud data according to an embodiment of the present disclosure;

[0039] Figure 5 Shows a schematic block diagram of a semantic segmentation method device for multi-source heterogeneous point clouds according to an embodiment of the present disclosure;

[0040] Figure 6 Shows a schematic block diagram of a semantic segmentation method device for multi-source heterogeneous point clouds according to an embodiment of the present disclosure. Detailed implementation manners

[0041] The following will refer to the accompanying drawings to detail various exemplary embodiments, features, and aspects of the present disclosure. The same reference numerals in the drawings denote elements with the same or similar functions. Although various aspects of the embodiments are shown in the drawings, the drawings do not have to be drawn to scale unless otherwise specified.

[0042] The special term "exemplary" here means "serving as an example, embodiment, or illustration". Any embodiment described as "exemplary" here does not have to be construed as superior to or better than other embodiments.

[0043] In addition, for a better illustration of the present disclosure, numerous specific details are given in the following detailed implementation manners. Those skilled in the art should understand that the present disclosure can also be implemented without some specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art are not described in detail so as to highlight the gist of the present disclosure.

[0044] <Method embodiment>

[0045] Figure 1 Shows a flowchart of a semantic segmentation method for multi-source heterogeneous point clouds according to an embodiment of the present disclosure. As Figure 1 shown, the method includes steps S1100 - S1400.

[0046] S1100. Obtain the image data of the target area and the point cloud data from each source. Here, the target area is the geographical area to be analyzed currently. The image data of the target area is the remote sensing image of the target area. The point cloud data of the target area obtained by different detection devices is the point cloud data from each source of the target area. For example, if the target area is the area where a certain reservoir is located, when obtaining the point cloud data of this target area, the water point cloud data of the target area is obtained by an unmanned aerial vehicle, and the water point cloud data is as shown in Figure 2 ; the underwater point cloud data of the target area is obtained by an unmanned ship, and the underwater point cloud data is as shown in Figure 3 ; the water point cloud data and the underwater point cloud data of the target area are the point cloud data from two sources of the target area.

[0047] S1200. Extract the attribute features of the point cloud data from each source, and use the pre-constructed attribute alignment model to align and match the attribute features of the point cloud data from each source to obtain the matching results of the point cloud data from each source.

[0048] Before performing this step, it is necessary to first construct an attribute alignment model. When constructing this attribute alignment model, it includes: constructing a training sample set. Here, the training sample set includes multiple training samples, and each training sample includes the point cloud data from different sources in the same area, as well as the standard attribute features labeled for the point cloud data from different sources and the matching results corresponding to the point cloud data from different sources. Use the training sample set to train a pre-selected neural network model. After the training is completed, the attribute alignment model can be obtained.

[0049] After obtaining the attribute alignment model, the pre-constructed attribute alignment model can be used to align and match the attribute features of the point cloud data from each source. Specifically, extract the attribute features of the point cloud data from each source, and input the attribute features of the point cloud data from each source into the attribute alignment model. The attribute alignment model can automatically align and match the attribute features of the point cloud data from each source and output the matching results of the point cloud data from each source.

[0050] When the attribute alignment model aligns and matches the attribute features of the point cloud data from each source and outputs the matching results of the point cloud data from each source, it may include the following steps:

[0051] First, convert the attribute features of the point cloud data from each source into standard attribute features to obtain the standard attribute features of the point cloud data from each source. Here, the standard attribute features are pre-set and are used to uniformly represent the attribute features of the point cloud data from different sources to avoid the problem of point cloud matching errors caused by inconsistent representation of attribute features between different data sources.

[0052] Second, based on the standard attribute features of each source point cloud data, match each source point cloud data to obtain the matching results of each source point cloud data. The specific matching process may include the following steps:

[0053] First, determine the target point cloud data from each source point cloud data. Specifically, one source of point cloud data can be randomly selected from the point cloud data of multiple sources as the target point cloud data. After selecting the target point cloud data, match the point cloud data of other sources with the target point cloud data respectively, and the matching results of each source point cloud data can be obtained.

[0054] Second, traverse each point cloud in the target point cloud data. For the currently traversed point cloud, calculate the similarity between the current point cloud and the standard attribute features of each point cloud in the point cloud data of other sources in turn, and regard the point cloud in the point cloud data of other sources with a similarity greater than the set threshold as the point cloud matching the current point cloud.

[0055] Finally, after the traversal is completed, the matching results of each source point cloud data can be obtained.

[0056] In a possible implementation, when calculating the similarity between the current point cloud and the standard attribute features of each point cloud in the point cloud data of other sources, it is implemented based on the preset attribute feature weights. Specifically, a point cloud data usually includes multiple standard attribute features, and a corresponding attribute feature weight is preset for each attribute feature. In this way, when calculating the similarity between the current point cloud and the standard attribute features of each point cloud in the point cloud data of other sources, first calculate the similarity between the same standard attribute features respectively, and then perform weighted summation on the calculated similarity through the attribute feature weights corresponding to each standard attribute feature, and the similarity between the current point cloud and the standard attribute features of each point cloud in the point cloud data of other sources can be obtained.

[0057] For example, the standard attribute features are position, color, and normal vector. The attribute feature weight set for the standard attribute feature of position is W1, the attribute feature weight set for the standard attribute feature of color is W2, and the attribute feature weight set for the standard attribute feature of normal vector is W3. P1 is the current point cloud in the target point cloud data, Q1 is a point cloud in the point cloud data of other sources, and the standard attribute features corresponding to P1 and Q1 are shown in Table 1.

[0058] Table 1

[0059]

[0060] In this example, when calculating the similarity between the standard attribute features of P1 and Q1, first calculate the similarity S1 between position a and position b, then calculate the similarity S2 between color b and color c, and then calculate the similarity between normal vector c and normal vector f. Finally, use the attribute feature weight W1 corresponding to the position, the attribute feature weight W2 corresponding to the color, and the attribute feature weight W3 corresponding to the normal vector to perform weighted summation on the similarity S1 between positions, the similarity S2 between colors, and the similarity S3 between normal vectors respectively, so as to obtain the similarity between the standard attribute features of P1 and Q1. That is, the calculation formula for the similarity S between the standard attribute features of P1 and Q1 is as follows:

[0061] S = W1 × S1 + W2 × S2 + W3 × S3

[0062] Referring to the above method, the similarity between the standard attribute features of the current point cloud and each point cloud in the point cloud data from other sources can be calculated. After calculating the similarity between the standard attribute features of the current point cloud and each point cloud in the point cloud data from other sources, the point clouds in the point cloud data from other sources with a similarity greater than the set threshold are regarded as the point clouds matching the current point cloud. In this way, after the traversal ends, the matching results of each source of point cloud data can be obtained.

[0063] In a possible implementation manner, in order to improve the accuracy of the matching between point cloud data, the preset attribute feature weights are updated according to the similarity between the standard attribute features of the current point cloud and each point cloud in the point cloud data from other sources. Continuing the above embodiment, after calculating the similarity S between the standard attribute features of P1 and Q1, the attribute feature weight W1 corresponding to the position will also be updated using the similarity S1 between positions, the attribute feature weight W2 corresponding to the color will be updated using the similarity S2 between colors, and the attribute feature weight W3 corresponding to the normal vector will be updated using the similarity S3 between normal vectors, so that the similarity between the standard attribute features of subsequent point clouds can be calculated based on the updated weights. Among them, the update formulas for each attribute feature weight are as follows:

[0064] Update weight of attribute feature weight W1 = S1 × W1;

[0065] Update weight of attribute feature weight W2 = S2 × W2;

[0066] Update weight of attribute feature weight W3 = S3 × W3.

[0067] In a possible implementation, to improve the accuracy of the matching results of point cloud data from various sources, for the point cloud data from each source, the semantic information of each point cloud in the point cloud data from each source will also be calculated, and the semantic information of each point cloud in the point cloud data from each source will be used as a feature attribute information of each point cloud data, so as to perform attribute alignment and matching of the point cloud data from each source based on the feature attribute including semantic information. When calculating the semantic information of each point cloud in the point cloud data from each source, the above-mentioned image data can be first subjected to semantic segmentation through SAM2 to obtain a two-dimensional semantic mask, and then the two-dimensional semantic mask is converted into a three-dimensional semantic mask. Next, the three-dimensional semantic mask is aligned and matched with the point cloud data from each source, and the semantic information in the three-dimensional semantic mask is assigned as a feature attribute to the point cloud in the point cloud data from each source that matches the three-dimensional semantic mask. In this way, the point cloud data from each source has the feature attribute of semantic information.

[0068] After obtaining the matching results of the point cloud data from each source, step S1300 can be executed to further fuse the matching results of the point cloud data from each source to obtain fused point cloud data. Specifically, the matching results will include multiple groups of matching point clouds. For each group of matching point clouds, each group of matching point clouds is fused into a single point cloud, and the value of each standard attribute feature of the fused point cloud is equal to the weighted average between the same standard attribute features of the multiple matching point clouds. Continuing with the above embodiment, when P1 and Q1 are matching point clouds, the position of the point cloud after their fusion is equal to the weighted average of position a and position b, the color is equal to the weighted average of color b and color c, and the normal vector is equal to the weighted average of normal vector c and normal vector f. After obtaining all the fused point cloud data, the fused point cloud data can be inserted into the point cloud set composed of the point cloud data from each source through interpolation or fitting to obtain the fused point cloud data.

[0069] After obtaining the fused point cloud data, step S1400 can be executed to perform semantic segmentation on the fused point cloud data based on the image data to obtain the semantic segmentation result of the target area. The specific steps can be as follows:

[0070] First, perform semantic analysis on the image data to obtain a two-dimensional semantic mask of the image data. Specifically, the image data can be input into the SAM2 model to output the two-dimensional semantic mask of the image data through the SAM2 model.

[0071] Second, convert the two-dimensional semantic mask into a three-dimensional semantic mask and align the three-dimensional semantic mask with the fused point cloud data. Specifically, first obtain the camera parameters corresponding to the image data. The camera parameters include the internal and external parameter information of the camera. For example, the focal length, principal point coordinates, distortion parameters, etc. of the camera. Then, use the camera parameters to convert the two-dimensional semantic mask into a three-dimensional semantic mask. Specifically, for the two-dimensional semantic mask that includes multiple pixel points, each pixel point has two-dimensional pixel coordinates and semantics, and the semantics are used to represent the category or boundary features of the pixel point. For each pixel point, use the camera parameters to convert its two-dimensional pixel coordinates into corresponding three-dimensional space coordinates, and bind the converted three-dimensional space coordinates with the corresponding semantics. After processing all the pixel points, the corresponding three-dimensional semantic mask can be obtained. That is, the three-dimensional semantic mask includes multiple spatial points, and semantic information is labeled for each spatial point. After obtaining the three-dimensional semantic mask, preprocessing operations such as denoising and smoothing will be performed on the point cloud data in the three-dimensional semantic mask, so as to output a more accurate and complete three-dimensional semantic mask. Finally, align and match the attributes of the preprocessed three-dimensional semantic mask with the fused point cloud data. The specific attribute alignment and matching process refers to step S1200 and will not be elaborated here.

[0072] Third, based on the three-dimensional semantic mask aligned with the fused point cloud data, perform semantic segmentation on the fused point cloud data to obtain the semantic segmentation result of the target area. Specifically, it can include the following steps: First, fuse the matching point clouds in the three-dimensional semantic mask and the fused point cloud data. The specific fusion method refers to the above and will not be elaborated here. Then, for the fused point cloud data, use the bidirectional group overlap algorithm to merge adjacent point clouds, reduce overlaps and gaps, and improve the integrity and accuracy of the point cloud data. Finally, use the region merging method to further extract and merge the semantic information of different regions, and finally obtain the complete 3D semantic segmentation result.

[0073] In the embodiment where the target area is a reservoir, after fusing the point cloud data above water and the point cloud data below water, the Figure 4 semantic segmentation result as shown will be obtained. From Figure 4 it can be seen that through the semantic segmentation method of the present disclosure, the point cloud features above water can be well displayed, and at the same time, the lake surface features in the point cloud data below water can be significantly shown, so that the semantic segmentation result can display more abundant scene information.

[0074] The present disclosure provides a semantic segmentation method for multi-source heterogeneous point clouds, comprising: obtaining image data of a target area and point cloud data from various sources; extracting attribute features from the point cloud data from various sources, and using a pre-built attribute alignment model to perform attribute alignment and matching on the point cloud data from various sources, thereby obtaining matching results for the point cloud data from various sources; fusing the matching results from various source point cloud data to obtain fused point cloud data; and performing semantic segmentation on the fused point cloud data based on the image data to obtain semantic segmentation results for the target area. The present disclosure improves the accuracy of semantic segmentation of multi-source heterogeneous point cloud data.

[0075] <Device Example>

[0076] Figure 5 FIG2 is a schematic block diagram of a semantic segmentation method and apparatus for multi-source heterogeneous point clouds according to an embodiment of the present disclosure. Figure 5 As shown, the device 100 includes:

[0077] The data acquisition module 110 is used to acquire image data of the target area and point cloud data from various sources;

[0078] The attribute alignment module 120 is used to extract attribute features of each source point cloud data, and use a pre-built attribute alignment model to perform attribute alignment and matching on each source point cloud data to obtain matching results for each source point cloud data;

[0079] The point cloud fusion module 130 is used to fuse the matching results of the point cloud data from each source to obtain fused point cloud data;

[0080] The semantic segmentation module 140 is configured to perform semantic segmentation on the fused point cloud data based on the image data to obtain a semantic segmentation result of the target area.

[0081] <Equipment Example>

[0082] Figure 6 FIG2 is a schematic block diagram of a semantic segmentation method and device for multi-source heterogeneous point clouds according to an embodiment of the present disclosure. Figure 6 As shown, a multi-source heterogeneous point cloud semantic segmentation method and apparatus 200 includes a processor 210 and a memory 220 for storing executable instructions of the processor 210. The processor 210 is configured to implement any of the aforementioned multi-source heterogeneous point cloud semantic segmentation methods when executing the executable instructions.

[0083] Here, it should be noted that the number of processors 210 can be one or more. At the same time, in the multi-source heterogeneous point cloud semantic segmentation method device 200 of the embodiments of the present disclosure, an input device 230 and an output device 240 can also be included. Among them, the processor 210, the memory 220, the input device 230, and the output device 240 can be connected through a bus or in other ways, and specific limitations are not made here.

[0084] As a computer-readable storage medium, the memory 220 can be used to store software programs, computer-executable programs, and various modules, such as: the programs or modules corresponding to the multi-source heterogeneous point cloud semantic segmentation method of the embodiments of the present disclosure. The processor 210 executes various functional applications and data processing of the multi-source heterogeneous point cloud semantic segmentation method device 200 by running the software programs or modules stored in the memory 220.

[0085] The input device 230 can be used to receive input numbers or signals. Among them, the signal can be a key signal related to the user settings and function control of the device / terminal / server. The output device 240 can include a display device such as a display screen.

[0086] <Embodiment of the storage medium>

[0087] According to the fourth aspect of the present disclosure, a non-volatile computer-readable storage medium is also provided, on which computer program instructions are stored, and when the computer program instructions are executed by the processor 210, the multi-source heterogeneous point cloud semantic segmentation method described in any one of the foregoing is implemented.

[0088] The above has described the embodiments of the present disclosure. The above description is exemplary and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes are obvious to those of ordinary skill in the art in the technical field without departing from the scope and spirit of the described embodiments. The selection of the terms used herein is intended to best explain the principles of the embodiments, the practical application, or the technical improvement of the technology in the market, or to enable other ordinary technical personnel in the technical field to understand the embodiments disclosed herein.

Claims

1. A semantic segmentation method for multi-source heterogeneous point clouds, characterized in that, Including: Obtain the image data of the target area and the point cloud data from each source; Extract the attribute features of the point cloud data from each source, and use the pre-constructed attribute alignment model to align and match the attribute features of the point cloud data from each source to obtain the matching results of the point cloud data from each source; Fuse the matching results of the point cloud data from each source to obtain the fused point cloud data; Perform semantic segmentation on the fused point cloud data based on the image data to obtain the semantic segmentation result of the target area; When extracting the attribute features of the point cloud data from each source, using the pre-constructed attribute alignment model to align and match the attribute features of the point cloud data from each source to obtain the matching results of the point cloud data from each source, it includes: Convert the attribute features of the point cloud data from each source into standard attribute features to obtain the standard attribute features of the point cloud data from each source; Based on the standard attribute features of the point cloud data from each source, match the point cloud data from each source to obtain the matching results of the point cloud data from each source; When, based on the standard attribute features of the point cloud data from each source, matching the point cloud data from each source to obtain the matching results of the point cloud data from each source, it includes: Determine the target point cloud data from the point cloud data from each source; Traverse each point cloud in the target point cloud data; For the currently traversed point cloud, calculate the similarity between the current point cloud and the standard attribute features of each point cloud in other source point cloud data in sequence, and use the point cloud in other source point cloud data with the similarity greater than the set threshold as the point cloud matching the current point cloud; After the traversal ends, the matching results of the point cloud data from each source can be obtained; When calculating the similarity between the current point cloud and the standard attribute features of each point cloud in other source point cloud data, it is implemented based on the preset attribute feature weights; The attribute feature weights are updated according to the similarity between the current point cloud and the standard attribute features of each point cloud in other source point cloud data.

2. The method according to claim 1, characterized in that, When performing semantic segmentation on the fused point cloud data based on the image data to obtain the semantic segmentation result of the target area, it includes: Perform semantic analysis on the image data to obtain the two-dimensional semantic mask of the image data; Convert the two-dimensional semantic mask into a three-dimensional semantic mask and align the three-dimensional semantic mask with the fused point cloud data; Based on the three-dimensional semantic mask aligned with the fused point cloud data, perform semantic segmentation on the fused point cloud data to obtain the semantic segmentation result of the target area.

3. The method according to claim 2, characterized in that, When performing semantic analysis on the image data to obtain the two-dimensional semantic mask of the image data, it is implemented based on the SAM2 model.

4. A semantic segmentation device for multi-source heterogeneous point clouds, characterized in that, Including: A data acquisition module for obtaining the image data of the target area and the point cloud data from each source; An attribute alignment module for extracting the attribute features of the point cloud data from each source, and using the pre-constructed attribute alignment model to align and match the attribute features of the point cloud data from each source to obtain the matching results of the point cloud data from each source; A point cloud fusion module for fusing the matching results of the point cloud data from each source to obtain the fused point cloud data; A semantic segmentation module, configured to perform semantic segmentation on the fused point cloud data based on the image data to obtain a semantic segmentation result of the target area; When extracting the attribute features of each source point cloud data and using a pre-constructed attribute alignment model to perform attribute alignment and matching on each source point cloud data to obtain the matching results of each source point cloud data, it includes: Converting the attribute features of each source point cloud data into standard attribute features to obtain the standard attribute features of each source point cloud data; Based on the standard attribute features of each source point cloud data, performing matching on each source point cloud data to obtain the matching results of each source point cloud data; When performing matching on each source point cloud data based on the standard attribute features of each source point cloud data to obtain the matching results of each source point cloud data, it includes: Determining target point cloud data from each source point cloud data; Traversing each point cloud in the target point cloud data; For the currently traversed point cloud, calculating the similarity between the current point cloud and the standard attribute features of each point cloud in other source point cloud data in sequence, and taking the point cloud in other source point cloud data with the similarity greater than the set threshold as the point cloud matching the current point cloud; After the traversal ends, the matching results of each source point cloud data can be obtained; When calculating the similarity between the current point cloud and the standard attribute features of each point cloud in other source point cloud data, it is implemented based on a preset attribute feature weight; The attribute feature weight is updated according to the similarity between the current point cloud and the standard attribute features of each point cloud in other source point cloud data.

5. A semantic segmentation device for multi-source heterogeneous point clouds, characterized in that, It includes: A processor; A memory for storing processor-executable instructions; Wherein, the processor is configured to implement the method described in any one of claims 1 to 3 when executing the executable instructions.

6. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that, The computer program instructions, when executed by the processor, implement the method described in any one of claims 1 to 3.

Images