Method and system for constructing 3D semantic change point cloud dataset based on deep learning
Through the pre-training of deep learning models and manual correction methods, the difficulty of constructing three-dimensional semantic change detection datasets was solved, and efficient and accurate semantic change detection of real urban scenes was achieved, reducing costs and difficulty.
Patent Information
- Application Number
- CN202510108627.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-23
- Publication Date
- 2025-09-30
- Estimated Expiration
- 2045-01-23
AI Technical Summary
Existing 3D semantic change detection datasets are difficult to construct and have poor authenticity. They cannot fully and accurately describe the semantic changes in real urban scenes, and the labeling cost is high and difficult.
By pre-training the deep learning model with open source datasets, semantic segmentation and change detection are performed on multi-time series point cloud data, combined with manual correction to obtain a well-labeled multi-time series point cloud dataset.
It reduces the cost and difficulty of dataset production, achieves a more comprehensive and accurate description of semantic changes in urban real scenes, and provides a rich data foundation.
Smart Images
Figure CN120071043B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of three-dimensional GIS, and specifically relates to a method and system for constructing a three-dimensional semantic change point cloud dataset based on deep learning. Background Art
[0002] Change detection is a crucial tool for observing urban renewal and monitoring urban land space. Because it can quickly identify changes within a city, it is widely used in urban planning, environmental protection, agricultural monitoring, disaster assessment, and other fields. With the continuous advancement of data collection methods, the cost and difficulty of collecting three-dimensional data have decreased, and it has gradually become one of the mainstream data types used to describe the appearance of cities. By comparing three-dimensional data from different time series, three-dimensional change detection can quickly identify changes within a city, locate the location of the changes, and provide a change map. However, this change map only indicates the location and existence of changes, but does not provide more detailed information about the changes. In urban management, it is necessary not only to know the location of changes, but also to clarify the type and extent of the changes. To achieve this, three-dimensional semantic change detection is necessary.
[0003] 3D semantic change detection simultaneously detects changes and identifies the category of the changed features. Currently, mainstream 3D semantic change detection is based on traditional methods, primarily using machine learning (random forest classifiers) to identify feature categories and then using algebraic differences based on the classification results to detect feature changes. While these methods are effective for 3D semantic change detection, they suffer from poor portability and generalizability. 3D semantic change detection methods based on deep learning can effectively address these issues by training efficient neural network models with datasets. However, their performance depends to a certain extent on the quality of the 3D change detection dataset.
[0004] 3D change detection datasets require not only annotating change information within multi-temporal point clouds but also annotating semantic category information within them. This makes their construction more difficult and costly than 3D change detection datasets. Currently, the primary method for constructing 3D change detection datasets is to generate virtual multi-temporal point clouds using simulation engines. Changes within these multi-temporal point clouds, such as new buildings or vegetation growth, are then manually created. Using 2D image mapping, semantic category information is then added to the point clouds at corresponding locations, enabling the annotation of both change and semantic information. While these methods are simple in principle, easy to operate, and inexpensive to construct, these virtual data suffer from simple geometry, monotonous textures, and low realism, making them incapable of comprehensively and accurately describing semantic changes within real urban scenes. Furthermore, due to the difficulty and cost of constructing 3D change detection datasets, the current lack of 3D real-world semantic change detection datasets significantly hinders the development of deep learning-based 3D semantic change detection methods.
[0005] In summary, the 3D semantic change detection method based on deep learning can quickly locate changes and accurately identify the categories of changed features. However, there is currently a lack of 3D real-scene semantic change detection datasets, and the construction of 3D change detection datasets still has the following problems: 1) Although the method of generating virtual data through a simulation engine can construct a 3D real-scene semantic change detection dataset at a low cost and high efficiency, the authenticity of the data is poor, which makes it unable to effectively describe the type and extent of changes in the city; 2) 3D semantic change detection requires not only labeling change information but also semantic information. The labeling workload increases exponentially with the number of observations, which greatly increases the cost and difficulty of dataset production.
[0006] Therefore, it is necessary to design a deep learning-based three-dimensional semantic change point cloud dataset construction method and system to address the above problems. Summary of the Invention
[0007] The purpose of the present invention is to address the problems existing in the prior art and provide a method and system for constructing a three-dimensional semantic change point cloud dataset based on deep learning. In order to address the problems that the existing data has weak authenticity and cannot describe real urban scenes, as well as the difficulty in labeling three-dimensional semantic change data, the present invention pre-trains a deep learning model with an open source dataset, performs semantic segmentation and change detection on the labeled multi-time series point cloud data, and then manually corrects the coarse classification results to finally obtain labeled multi-time series point cloud data, thereby completing the construction of the three-dimensional semantic change point cloud dataset and reducing the cost and difficulty of dataset labeling.
[0008] According to one aspect of this specification, a method for constructing a three-dimensional semantic change point cloud dataset based on deep learning is provided, comprising:
[0009] Define semantic categories for labeled multi-temporal point cloud data;
[0010] Based on the defined semantic categories, the 3D semantic segmentation dataset and semantic segmentation model are preprocessed. The preprocessed 3D semantic segmentation dataset is input into the preprocessed semantic segmentation model for pre-training. The pre-trained semantic segmentation model is used to perform semantic segmentation on the labeled multi-temporal point cloud data to obtain rough semantic information.
[0011] Based on the defined semantic categories, the 3D change detection dataset and 3D change detection model are adjusted. The adjusted 3D change detection dataset is input into the adjusted 3D change detection model for pre-training. Change detection is then performed on the annotated multi-time series point cloud data based on the pre-trained 3D change detection model to obtain rough change information.
[0012] Based on the obtained coarse semantic information and coarse change information, intra-class information correction and inter-class information correction are performed, and then the labeled multi-time series point cloud data are obtained through cross-validation to obtain a constructed three-dimensional semantic change point cloud dataset.
[0013] Furthermore, the 3D semantic segmentation dataset and semantic segmentation model are preprocessed, including:
[0014] The semantics in the 3D semantic segmentation dataset that are close to the defined semantic categories are used as target semantics. At the same time, based on the density, number of points, and spatial range of the point cloud data, the differences between the 3D semantic segmentation dataset and the multi-temporal point cloud data to be annotated are processed;
[0015] Modify the output layer of the semantic segmentation model so that the number of neurons in the output layer matches the number of target semantic categories.
[0016] Furthermore, the 3D change detection dataset and the 3D change detection model are adjusted, including:
[0017] Based on the differences between the 3D change detection dataset and the multi-time series point cloud data to be annotated, the category of the 3D change detection dataset is adjusted;
[0018] Fine-tune the initial parameters of the 3D change detection model and set the learning rate.
[0019] Furthermore, intra-class information correction and inter-class information correction are performed, including:
[0020] Based on the coarse semantic information, according to the defined semantic categories, the correctly classified data are retained and the misclassified data are classified into the correct category;
[0021] Based on the rough change information, check the data classified as changed and correct the false changed data therein; check the data classified as unchanged and correct the false unchanged data therein;
[0022] Based on semantic consistency and change contrast, verify whether the corrected coarse semantic information and coarse change information correspond to each other.
[0023] Furthermore, before the 3D semantic segmentation dataset is preprocessed, point cloud registration is also included:
[0024] Select the corresponding feature points in different time series point clouds and perform preliminary alignment on the multi-time series point cloud data;
[0025] The iterative closest point algorithm is used to align the multi-time series point cloud data after preliminary alignment to complete the point cloud registration.
[0026] According to one aspect of the present disclosure, a system for constructing a three-dimensional semantic change point cloud dataset based on deep learning is provided, comprising:
[0027] The semantic category module is used to define semantic categories for labeled multi-temporal point cloud data;
[0028] The coarse semantic information module is used to preprocess the 3D semantic segmentation dataset and semantic segmentation model based on the defined semantic categories, input the preprocessed 3D semantic segmentation dataset into the preprocessed semantic segmentation model for pre-training, and perform semantic segmentation on the labeled multi-temporal point cloud data based on the pre-trained semantic segmentation model to obtain coarse semantic information;
[0029] The coarse change information module is used to adjust the 3D change detection dataset and 3D change detection model based on the defined semantic categories, input the adjusted 3D change detection dataset into the adjusted 3D change detection model for pre-training, and perform change detection on the annotated multi-time series point cloud data based on the pre-trained 3D change detection model to obtain coarse change information.
[0030] The dataset construction module is used to perform intra-class information correction and inter-class information correction based on the obtained coarse semantic information and coarse change information, and then obtain the labeled multi-time series point cloud data through cross-validation to obtain a constructed three-dimensional semantic change point cloud dataset.
[0031] According to one aspect of this specification, an electronic device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and wherein the processor implements the steps of the method for constructing a three-dimensional semantic change point cloud dataset based on deep learning when executing the computer program.
[0032] According to one aspect of the present specification, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the computer program is executed by a processor, the steps of the method for constructing a three-dimensional semantic change point cloud dataset based on deep learning are implemented.
[0033] Compared with the prior art, the present invention has the following beneficial effects:
[0034] 1. To address the problem of poor data authenticity that cannot effectively describe the type and extent of changes in cities, this paper integrates deep learning network models into data annotation and proposes a new method for constructing a three-dimensional semantic change dataset, which achieves a more comprehensive and accurate description of semantic changes in real urban scenes.
[0035] 2. The present invention pre-trains a deep learning model through open source datasets, performs semantic segmentation and change detection on the labeled multi-time series point cloud data, and then manually corrects the coarse classification results to finally obtain labeled multi-time series point cloud data. This effectively reduces the cost and difficulty of producing three-dimensional change detection datasets, improves the efficiency of dataset annotation, and provides a rich data foundation for the design of three-dimensional semantic change detection methods. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0037] Figure 1 This is a flowchart of a method for constructing a three-dimensional semantic change point cloud dataset based on deep learning according to an embodiment of the present invention;
[0038] Figure 2 This is a flowchart of semi-automatic semantic information annotation according to an embodiment of the present invention;
[0039] Figure 3 This is a flowchart of semi-automatic change information annotation according to an embodiment of the present invention;
[0040] Figure 4 A sample diagram of a three-dimensional change detection dataset constructed for an embodiment of the present invention. DETAILED DESCRIPTION
[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0042] An embodiment of the present invention provides a method for constructing a three-dimensional semantic change point cloud dataset based on deep learning, including: defining semantic categories for annotated multi-temporal point cloud data; preprocessing a three-dimensional semantic segmentation dataset and a semantic segmentation model based on the defined semantic categories, inputting the preprocessed three-dimensional semantic segmentation dataset into the preprocessed semantic segmentation model for pre-training, and performing semantic segmentation on the annotated multi-temporal point cloud data based on the pre-trained semantic segmentation model to obtain rough semantic information; adjusting a three-dimensional change detection dataset and a three-dimensional change detection model based on the defined semantic categories, inputting the adjusted three-dimensional change detection dataset into the adjusted three-dimensional change detection model for pre-training, and performing change detection on the annotated multi-temporal point cloud data based on the pre-trained three-dimensional change detection model to obtain rough change information; performing intra-class information correction and inter-class information correction based on the obtained rough semantic information and rough change information, and then obtaining the annotated multi-temporal point cloud data through cross-validation to obtain a constructed three-dimensional semantic change point cloud dataset.
[0043] The method provided in the embodiment of the present invention can be implemented using computer software technology. Figure 1 The embodiment uses two time-series input point clouds as an example to illustrate the process of the present invention, and the specific implementation scheme of the embodiment uses the Python programming environment as an example.
[0044] An embodiment of the present invention provides a definition of semantic categories. The three-dimensional change detection dataset needs to be annotated with both semantic information and change information. Among them, the change information can be defined by simply whether a change has occurred, while the semantic category needs to be determined based on multiple factors such as the purpose and object of the change detection, the landscape appearance of the object to be detected, etc. The semantic category directly determines the type of change that the three-dimensional change detection dataset can cope with. A clear and accurate definition of semantic categories helps to accurately identify and distinguish different types of changes. Therefore, before annotating the three-dimensional semantic change detection data, it is first necessary to define the semantic category. The specific implementation process is described as follows:
[0045] The definition of semantic categories determines the types of semantic changes that a 3D change detection dataset can detect. Therefore, it's necessary to define relevant semantic categories based on the dataset's intended objects and objectives. In this embodiment, the method categorizes semantic categories into three main categories: man-made objects, natural objects, and landforms. Man-made objects primarily include buildings, roads, and infrastructure. Buildings are a common semantic category in both urban and rural environments, encompassing residential, commercial, and industrial buildings. Roads are a key component of man-made objects on the surface of the earth, serving as a vital link between various landforms. They primarily include highways, urban arterial roads, and rural paths. Infrastructure is crucial for ensuring public safety and maintaining the normal functioning of society, and primarily includes bridges, utility poles, and communication towers. Natural objects refer to entities naturally occurring in nature, untouched by large-scale human manipulation. They are the primary components of the earth's surface, primarily including vegetation and water bodies. Vegetation effectively reflects a region's greening rate and the quality of its ecological environment, and includes forests, grasslands, and farmland. Water bodies refer to natural complexes covered by surface water, including rivers and lakes. Landforms refer to the undulating shape of the earth's surface, and include subcategories such as mountains and valleys. Mountain ranges are defined as a series of ridges and valleys extending in a specific direction. Monitoring these ranges plays an important role in detecting geological disasters (such as landslides) or natural landscape changes. Monitoring valleys can effectively identify soil erosion. These three categories of objects effectively cover most of the major semantic categories within the study area.
[0046] Specifically, the implementation scheme of the embodiment of the present invention is as follows: Taking the Kowloon area of Hong Kong, China from 2017 to 2018 as an example, Hong Kong was in a stage of rapid urban renewal at that time, and there were a large number of changes in land features. This embodiment mainly defines the following semantic categories based on the urban appearance of the Kowloon area at that time. 1. Land surface. The land surface is the basic carrier of the entire Kowloon area of Hong Kong, China. It effectively connects all other land features, mainly including roads, sketches, land, and squares. 2. Buildings. Buildings are the main land feature type in the Kowloon area of Hong Kong, China, occupying most of the surface space. Its subcategories include residential, commercial land, office land, schools, hospitals, nursing homes, cultural and sports venues, and government police stations. 3. Vegetation. Vegetation reflects the greening and ecological environment of the Kowloon area of Hong Kong, China, mainly including shrubs, trees, street trees, and forests. 4. Vehicles. Vehicles are common objects on the land surface, mainly including motor vehicles and non-motor vehicles. 5. Street facilities. Street facilities are crucial for maintaining the normal functioning of society and primarily include streetlights, traffic lights, fire hydrants, guardrails, fences, and median strips. 6. Building facilities. Building facilities primarily refer to the accumulation of materials used for building construction or waste from demolition.
[0047] The embodiment of the present invention provides a coarse annotation of semantic information based on deep learning. First, it is necessary to clean the multi-time point cloud data to remove noise and outliers to ensure the accuracy of the data. Then, according to the defined semantic categories, a public 3D semantic segmentation dataset with similar semantic categories is used to train the most advanced 3D semantic segmentation model. Finally, the dataset to be annotated is used as a test set, and a pre-trained 3D semantic segmentation model is used to perform semantic segmentation on it to obtain the coarse semantic category information of the dataset to be annotated. The specific structure is as follows: Figure 2 The specific implementation process is described as follows:
[0048] Specifically, statistical filtering is first used to clean the point cloud. In statistical filtering, for each point in the point cloud, it is necessary to first determine a suitable neighborhood range. The size of the neighborhood range depends on factors such as the density of the point cloud, the size and shape of the target object, etc. Then, within the determined neighborhood, the statistical features of each point are calculated. The most commonly used feature is the distance from the point to other points in its neighborhood. Subsequently, the filter threshold is set based on the statistical features. Generally speaking, a multiple of the distance mean can be used as the threshold. Finally, all points in the point cloud are traversed and filtered according to the set threshold. Points determined to be noise points are deleted from the original point cloud data to obtain a cleaned point cloud.
[0049] Specifically, the semantic segmentation model is then pre-trained. The purpose of this step is to subsequently use the pre-trained model for semantic recognition of the data to be processed. This is essentially a form of transfer learning. First, a specific source model must be selected. First, it is important to clarify that the source model used for pre-training must solve the semantic segmentation task based on real-world 3D point clouds. The relevance of the source model's architecture, training methods, and data characteristics to the target semantic segmentation task must be carefully considered to ensure the source model's usability. The structure of the source model then needs to be adjusted based on the characteristics of the task. Adjusting the source model's architecture based on the requirements of the target task may require modifying the output layer settings to ensure that the number of neurons matches the number of target semantic categories. Subsequently, the target dataset needs to be pre-processed to align with the input requirements of the source model. Furthermore, factors such as point cloud density, point count, and spatial extent should be considered to mitigate the impact of inconsistencies between the training and target datasets. Finally, the semantic segmentation dataset is trained on an open-source dataset, and the model weights that yield the best performance are used to perform semantic segmentation on the target dataset to obtain a coarse classification result.
[0050] Specifically, the implementation scheme of this embodiment of the present invention is as follows: Taking the real-world point cloud data for the Kowloon area of Hong Kong, China in Step 1 as an example, this embodiment selects Sensaturban as the 3D semantic segmentation dataset and KPConv as the semantic segmentation model to achieve preliminary semantic category identification. Like the real-world point cloud for the Kowloon area of Hong Kong, China, Sensaturban is a large-scale, city-level photogrammetric real-world point cloud. It describes the semantic categories of areas such as Birmingham, UK, and includes 13 categories: ground, vegetation, buildings, walls, bridges, parking lots, tracks, roads, street furniture, vehicles, sidewalks, bicycles, and water. To match the semantic categories of the target dataset for the Kowloon area of Hong Kong, China, this embodiment processes the training dataset, classifying six object categories (ground, parking lots, sidewalks, tracks, roads, and water) into the land surface category; two object categories (buildings and bridges) into the building category; vegetation into the vegetation category; two object categories (vehicles and bicycles) into the transportation category; and two object categories (street furniture and walls) into the street furniture category. Note that buildings and facilities are not included, requiring subsequent manual correction of these objects. Before training KPConv, we also need to change the output categories of KPConv to 6 to meet the requirements of the target dataset. When training KPConv, we set the batch size to 12, used weighted cross entropy as the loss function, used Ranger as the model optimizer, and set the downsampling ratio to 0.2.
[0051] The embodiment of the present invention provides a rough annotation of change information based on deep learning. First, based on the multi-time series point cloud data that has been cleaned, point cloud registration is performed on these data, and data of different time series are matched to corresponding positions. Then, according to the urban style of the data to be annotated and the main types of changes that have occurred, a matching three-dimensional change detection data set is selected to pre-train the most advanced three-dimensional change detection model to obtain the optimal model weight. Finally, the data set to be annotated is used as a test set, and the pre-trained three-dimensional change detection model is used to perform change detection on the data set to obtain rough change information. The specific process is as follows: Figure 3 The specific implementation process is described as follows:
[0052] Specifically, point cloud registration is first required for the cleaned multi-sequence point clouds. Point cloud registration in this embodiment primarily involves two steps: manual alignment and automatic registration. Because the coordinate systems of different time-sequence point clouds may differ significantly, initial point clouds may be far apart, requiring manual alignment. Manual alignment involves manually selecting corresponding feature points from different time-sequence point clouds to align the multiple time-sequence point clouds and reduce the distance between them. The iterative closest point algorithm is then used to more accurately register the point clouds. First, the closest point in the target point cloud is found for each point in the source point cloud. This can be achieved by calculating the Euclidean distance between points. After finding the closest point pairs, the least squares method is used to calculate a rigid transformation matrix that aligns the two point clouds, minimizing the sum of the squared distances between the transformed points in the source point cloud and the closest points in the target point cloud. After obtaining the initial transformation matrix, this transformation is applied to the source point cloud, followed by finding the closest point pairs and calculating a new transformation matrix. This process is repeated (iteratively) until a convergence condition is met. The convergence condition can be that the distance change between point clouds is less than a certain threshold, or the number of iterations reaches a preset maximum value.
[0053] Specifically, the 3D change detection model is then pre-trained. Similar to pre-training the semantic segmentation model, the corresponding 3D change detection dataset and model must first be selected based on the properties of the target dataset to minimize the impact of data inconsistencies. For example, if the target data is a real-world point cloud from oblique photography, simulated data or street view data cannot be used as the training data; if the target data is a dense point cloud, sparse point cloud cannot be used as the training data. Next, based on the differences between the training and target datasets, the classification of the training dataset and the structure of the 3D change detection model must be adjusted to match the output requirements of the target dataset. The model can then be trained. Before training, it is necessary to determine whether to fine-tune all or some parameters. You can first fix the parameters reused from the source model and train only the newly added or modified parts of the target model (such as the output layer). After a certain training cycle, all parameters can be fine-tuned. During training, it is important to carefully adjust hyperparameters such as the learning rate. Since the model already has certain initial parameters, the learning rate can be set relatively low to avoid excessive modification of the source model's parameters. At the same time, strategies such as early stopping can be used to prevent overfitting. This means stopping training when the model's performance on the validation set no longer improves. Finally, using the pre-trained model to perform change detection on the target dataset can provide coarse change information.
[0054] Specifically, the implementation scheme of the embodiment of the present invention is: taking the real-life point cloud data of the Kowloon area of Hong Kong, China as an example. In order to conform to the characteristics of the real-life dense point cloud, this embodiment selects AHNCD as the three-dimensional change detection dataset. AHNCD describes the changes in the Utrecht area of the Netherlands from 2014 to 2022 through oblique photography of real-life point clouds. The change type is a binary change, i.e., change and no change, which is consistent with the change type of the target dataset and can be well adapted to the change detection of the target dataset. In addition, in order to achieve accurate change detection, the embodiment of the present invention selects the currently best performing three-dimensional change detection model PGN3DCD. During training, this embodiment uses negative log-likelihood as the loss function and uses StochasticGradient Descent as the model optimizer. The initial learning rate is set to 0.01 and the Batchsize is set to 10.
[0055] The embodiment of the present invention provides a correction for semantic changes. Based on the obtained category information and change information, the two types of information are manually corrected respectively. First, the intra-class information is corrected, that is, the misclassified data in the category information and change information are corrected respectively. Then, the inter-class information is corrected, and the data is corrected according to the correlation between the category information and the change information to ensure that the category information and the change information can effectively correspond. Finally, the data annotated by different annotators are cross-validated to ensure the accuracy of the final output data. The constructed sample is as follows: Figure 4 The specific implementation process is described as follows:
[0056] The coarse semantic category information and coarse change information obtained are refined separately. First, the coarse semantic category information is corrected. During the semantic category correction process, each category is checked according to the defined semantic categories. The specific method for checking is to retain the correctly classified data and manually classify the misclassified data into the corresponding correct category. Then, the coarse change information is corrected. During the change information correction process, the data classified as changed are first checked and false change samples are corrected. Then, the data classified as unchanged are checked and false unchanged samples are corrected. Subsequently, based on semantic consistency and change contrast, cross-validation is performed to verify that the semantic category and change information are effectively aligned. That is, if the categories of point clouds from different time series are inconsistent, then these point clouds are changing. If the point clouds from different time series have not changed, then the semantic categories of these point clouds should be consistent. If there is no effective correspondence, the semantic category and change information need to be corrected accordingly. Finally, cross-validation is performed on different manually annotated data to further improve data accuracy.
[0057] Specifically, the implementation scheme of this embodiment of the present invention utilizes CloudCompare as a platform tool for manual annotation. Based on the semantic category definitions described above, the resulting coarse semantic category information is inspected and corrected in the order of ground, buildings, vegetation, vehicles, street furniture, and building facilities. During the correction process, areas where different types of features meet are prone to misclassification, as the model typically fails to effectively identify the semantic categories in such areas. The resulting coarse change information then needs to be corrected. During the correction process, false unchanged samples often occur with small objects, such as vehicles and street furniture, which are too small to be detected by the model. False changed samples often occur with vegetation, primarily due to the difficulty maintaining consistent density across different time-series point clouds during vegetation data collection. Subsequently, inter-class consistency checks are performed on the semantic categories and change information. This process focuses on objects that have undergone replacement changes, such as demolition and reconstruction of houses (same-class changes), transformation of trees into houses (different-class changes), and transition areas between different types of features.
[0058] The implementation of each embodiment of the present invention is based on programmed processing performed by a device with processor functionality. Therefore, in practical engineering, the technical solutions and functions of each embodiment of the present invention are encapsulated into various modules. Based on this reality, and in addition to the aforementioned embodiments, an embodiment of the present invention provides a system for constructing a 3D semantically varied point cloud dataset based on deep learning. This system is used to execute the method for constructing a 3D semantically varied point cloud dataset based on deep learning described in the aforementioned method embodiments.
[0059] The system includes: a semantic category module, which is used to define semantic categories for labeled multi-temporal point cloud data; a coarse semantic information module, which is used to preprocess a three-dimensional semantic segmentation dataset and a semantic segmentation model based on the defined semantic categories, input the preprocessed three-dimensional semantic segmentation dataset into the preprocessed semantic segmentation model for pre-training, and perform semantic segmentation on the labeled multi-temporal point cloud data based on the pre-trained semantic segmentation model to obtain coarse semantic information; a coarse change information module, which is used to adjust a three-dimensional change detection dataset and a three-dimensional change detection model based on the defined semantic categories, input the adjusted three-dimensional change detection dataset into the adjusted three-dimensional change detection model for pre-training, and perform change detection on the labeled multi-temporal point cloud data based on the pre-trained three-dimensional change detection model to obtain coarse change information; a dataset construction module, which is used to perform intra-class information correction and inter-class information correction based on the obtained coarse semantic information and coarse change information, and then obtain the labeled multi-temporal point cloud data through cross-validation to obtain a constructed three-dimensional semantic change point cloud dataset.
[0060] The deep learning-based three-dimensional semantic change point cloud dataset construction system provided by the embodiment of the present invention addresses the problem that the authenticity of existing data is weak and cannot comprehensively and accurately describe the semantic changes in real urban scenes. It adopts several modules to pre-train the deep learning model through open source datasets, perform semantic segmentation and change detection on the labeled multi-time series point cloud data, and then manually correct the coarse classification results to finally obtain labeled multi-time series point cloud data, effectively reducing the cost and difficulty of producing three-dimensional change detection datasets.
[0061] Based on the same inventive concept as the above-mentioned embodiment, an embodiment of the present invention also provides an electronic device, including a memory and a processor, the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions, to implement a method for constructing a three-dimensional semantic change point cloud dataset based on deep learning as proposed in the above-mentioned embodiment.
[0062] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the program overcomes the problem of poor data authenticity, which results in its inability to effectively describe the type and extent of changes in the city. It effectively reduces the cost and difficulty of producing a three-dimensional change detection dataset, improves the efficiency of dataset annotation, and achieves a more comprehensive and accurate description of semantic changes in real urban scenes. The storage medium can be any non-volatile storage device such as a hard disk, solid-state drive, flash drive, or optical disk, and is used to store computer program code and necessary data files. The stored computer program includes: a semantic category module, a coarse semantic information module, a coarse change information module, and a dataset construction module.
[0063] Finally, it should be noted that the above specific embodiments are merely representative examples of the present invention. Obviously, the present invention is not limited to the above specific embodiments and is susceptible to numerous variations. Any simple modifications, equivalent variations, and modifications to the above specific embodiments based on the technical essence of the present invention shall be deemed to fall within the scope of protection of the present invention.
Claims
1. A method for constructing a 3D semantic change point cloud dataset based on deep learning, characterized in that: include: Define semantic categories for labeled multi-temporal point cloud data; Based on the defined semantic categories, the 3D semantic segmentation dataset and semantic segmentation model are preprocessed. The preprocessed 3D semantic segmentation dataset is input into the preprocessed semantic segmentation model for pre-training. The pre-trained semantic segmentation model is used to perform semantic segmentation on the labeled multi-temporal point cloud data to obtain rough semantic information. Based on the defined semantic categories, the 3D change detection dataset and 3D change detection model are adjusted. The adjusted 3D change detection dataset is input into the adjusted 3D change detection model for pre-training. Change detection is then performed on the annotated multi-time series point cloud data based on the pre-trained 3D change detection model to obtain rough change information. Based on the obtained coarse semantic information and coarse change information, intra-class information correction and inter-class information correction are performed, and then the labeled multi-time series point cloud data are obtained through cross-validation to obtain a constructed three-dimensional semantic change point cloud dataset.
2. The method for constructing a three-dimensional semantic change point cloud dataset based on deep learning according to claim 1, characterized in that: Preprocess the 3D semantic segmentation dataset and semantic segmentation model, including: The semantics in the 3D semantic segmentation dataset that are close to the defined semantic categories are used as target semantics. At the same time, based on the density, number of points, and spatial range of the point cloud data, the differences between the 3D semantic segmentation dataset and the multi-temporal point cloud data to be annotated are processed; Modify the output layer of the semantic segmentation model so that the number of neurons in the output layer matches the number of target semantic categories.
3. The method for constructing a three-dimensional semantic change point cloud dataset based on deep learning according to claim 1, characterized in that: Adjustments to the 3D change detection dataset and 3D change detection model are made, including: Based on the differences between the 3D change detection dataset and the multi-time series point cloud data to be annotated, the category of the 3D change detection dataset is adjusted; Fine-tune the initial parameters of the 3D change detection model and set the learning rate.
4. The method for constructing a 3D semantic change point cloud dataset based on deep learning according to claim 1, characterized in that: Perform intra-class information correction and inter-class information correction, including: Based on the coarse semantic information, according to the defined semantic categories, the correctly classified data are retained and the misclassified data are classified into the correct category; Based on the rough change information, check the data classified as changed and correct the false changed data therein; check the data classified as unchanged and correct the false unchanged data therein; Based on semantic consistency and change contrast, verify whether the corrected coarse semantic information and coarse change information correspond to each other.
5. The method for constructing a three-dimensional semantic change point cloud dataset based on deep learning according to claim 1, characterized in that: Before the 3D semantic segmentation dataset is preprocessed, point cloud registration is also included: Select the corresponding feature points in different time series point clouds and perform preliminary alignment on the multi-time series point cloud data; The iterative closest point algorithm is used to align the multi-time series point cloud data after preliminary alignment to complete the point cloud registration.
6. A system for constructing a 3D semantic change point cloud dataset based on deep learning, characterized by: include: The semantic category module is used to define semantic categories for labeled multi-temporal point cloud data; The coarse semantic information module is used to preprocess the 3D semantic segmentation dataset and semantic segmentation model based on the defined semantic categories, input the preprocessed 3D semantic segmentation dataset into the preprocessed semantic segmentation model for pre-training, and perform semantic segmentation on the labeled multi-temporal point cloud data based on the pre-trained semantic segmentation model to obtain coarse semantic information; The coarse change information module is used to adjust the 3D change detection dataset and 3D change detection model based on the defined semantic categories, input the adjusted 3D change detection dataset into the adjusted 3D change detection model for pre-training, and perform change detection on the annotated multi-time series point cloud data based on the pre-trained 3D change detection model to obtain coarse change information. The dataset construction module is used to perform intra-class information correction and inter-class information correction based on the obtained coarse semantic information and coarse change information, and then obtain the labeled multi-time series point cloud data through cross-validation to obtain a constructed three-dimensional semantic change point cloud dataset.
7. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for constructing a three-dimensional semantic change point cloud dataset based on deep learning are implemented.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for constructing a three-dimensional semantic change point cloud dataset based on deep learning are implemented.