Disaster scene point cloud data segmentation method and system

Abstract

This invention relates to the fields of artificial intelligence and point cloud segmentation, and provides a method and system for segmenting point cloud data in disaster scenarios. In the continuous learning of the student network, this invention effectively improves the network's ability to distinguish new object classes by resampling data from new object classes in the secondary subset. It generates pseudo-labels by combining the teacher network's prediction results with the original true labels of the new classes, and obtains the training loss based on the difference between the generated pseudo-labels and the student network's prediction results. By synchronously performing hierarchical pooling on the point cloud features of the teacher and student networks, and obtaining the distillation loss based on the difference between the pooled features, the network does not need to construct a total loss based on old classes, solving the problem of significantly increasing computer memory usage for old class retention. Simultaneously, by using the teacher network trained on the principal subset to guide the student network's learning, rapid identification of post-disaster objects in new classes can be achieved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and point cloud segmentation, and in particular to a method and system for segmenting point cloud data in disaster scenarios. Background Technology

[0002] With the rapid development of disaster prevention and mitigation technologies, the use of computer algorithms to process point cloud data can provide efficient and accurate assistance for disaster assessment and decision-making. As a key technology for identifying object types and segmenting instance shapes, point cloud instance segmentation is an important part of this process.

[0003] Currently, point cloud instance segmentation based on deep learning typically uses implicit fitting functions of neural networks. It leverages point cloud data to drive the network, learns complex patterns and high-dimensional features, and generates item classifications and instance masks. It is mainly aimed at closed scenes, that is, research and application are only carried out on indoor and street scene data with known categories. It lacks the ability to distinguish newly emerging object features and categories.

[0004] Disaster scenarios, as a special type of scenario, are characterized by complex environments, diverse elements, variable features, and high urgency. When a disaster occurs, it is necessary to effectively distinguish the features and categories of newly emerging objects. For open scene points, methods based on deep learning point cloud instance segmentation combined with continuous learning are still in their early stages. Currently, the only publicly available literature (Boudjoghra M EA, Al Khatib S, Lahoud J, et al. 3D Indoor Instance Segmentation in an Open-World[J]. Advances in Neural Information Processing Systems, 2024, 36) addresses methods used in indoor open scenes. For outdoor open disaster scenarios, this method has the following shortcomings:

[0005] (1) The constructed data does not contain objects specific to the disaster scene and lacks depth information of the outdoor scene, making it impossible to identify all disaster elements in the disaster scene.

[0006] (2) A continuous learning method based on old class sample data is adopted. When a new object category is added, a data playback anti-forgetting mechanism is adopted. That is, the old class data and the new class dataset are not divided. The old class data and the new class data that have been trained are combined to form a dataset. The network builds loss from the saved old class for continuous learning. In the case of a large amount of data in remote sensing point cloud scene, the old class sample data needs to be stored, which will bring a heavy additional storage burden and thus greatly increase the computer memory. At the same time, the dataset synthesized by the old and new classes increases the complexity and intensity of training, making it impossible to quickly identify disaster scene elements and thus making it difficult to meet the urgent requirements.

[0007] To address these technical issues, a segmentation method and system for point cloud data in disaster scenarios are proposed. Summary of the Invention

[0008] The purpose of this invention is to provide a method and system for segmenting point cloud data in disaster scenarios, aiming to solve the problems mentioned in the background art.

[0009] One embodiment of the present invention provides a segmentation method for point cloud data in disaster scenarios, the method comprising the following steps:

[0010] Obtain manually labeled regular urban point cloud data; obtain manually labeled open disaster scene point cloud data, wherein the open disaster scene point cloud data includes a learning set and a validation set, wherein the learning set includes a principal subset and a secondary subset, wherein the annotations of the principal subset only include common objects in general outdoor scenes, and the annotations of the secondary subset only include one or more disaster scene objects.

[0011] A network structure for segmenting the conventional urban point cloud data is constructed; the network is pre-trained on the conventional urban point cloud data to obtain network pre-training parameters; the pre-trained network structure and pre-training parameters are transferred to the principal subset;

[0012] The migrated networks are used as teacher networks and student networks, respectively. The student networks are used for continuous learning in the sub-subsets. When continuous learning begins in each sub-subset, the network parameters obtained from the continuous learning of the previous sub-subset are used to initialize the teacher network and student network, and then the teacher network is frozen.

[0013] The point cloud data in the secondary subset is resampled, with undersampling of background samples belonging to common objects and oversampling of target samples belonging to disaster scene objects;

[0014] The resampled point cloud data is synchronously input into the feature extractors of the teacher network and the student network to obtain the point cloud features of the teacher network and the student network respectively.

[0015] The point cloud features of the obtained teacher network and student network are input into the decoder to generate prediction results for the teacher network and student network respectively. Using the prediction results of the teacher network and the original true label annotation of the target class, pseudo labels are generated. The training loss is calculated based on the difference between the pseudo labels and the prediction results of the student network. The point cloud features of the teacher network and student network are simultaneously subjected to hierarchical pooling to obtain hierarchical pooling features of the teacher network and student network respectively. The distillation loss is calculated based on the difference between the two hierarchical pooling features.

[0016] The distillation loss and the training loss are summed to obtain the total loss. Backpropagation is performed based on the total loss to train the student network. The parameters of the student network are optimized. After training is completed on all subsets, the optimized parameters of the student network are obtained.

[0017] The point cloud data on the validation set is processed after the student network is initialized using the optimized parameters. The obtained semantic-level predictions and instance-level masks are compared with the labeled true categories and masks to evaluate the point cloud data segmentation effect.

[0018] Another embodiment of the present invention is implemented as follows: a segmentation system for point cloud data in disaster scenarios is provided, the system comprising:

[0019] Point cloud data acquisition and annotation unit: The point cloud data acquisition and annotation unit is used to acquire manually annotated conventional urban point cloud data; and to acquire open disaster scene point cloud data and perform manual annotation. The open disaster scene point cloud data includes a learning set and a validation set. The learning set includes a primary subset and a secondary subset. The labels of the primary subset only include common objects in general outdoor scenes, and the labels of the secondary subset only include one or more disaster scene objects.

[0020] A network pre-training unit is used to pre-train the network on manually labeled conventional urban point cloud data to obtain network pre-training parameters.

[0021] A network migration unit is used to migrate the network and pre-trained parameters on a principal subset of manually labeled open disaster scenario point cloud data.

[0022] A continuous learning unit for networks, wherein the transferred networks are respectively used as teacher networks and student networks, and continuous learning is performed on sub-subsets using the student network to obtain student network parameters; the continuous learning unit for networks specifically includes:

[0023] Initialization module: used to initialize the teacher network and student network with the network parameters obtained from the previous sub-subset when the student network starts continuous learning at each sub-subset, and then freeze the teacher network;

[0024] Secondary subset data resampling module: used to resample point cloud data in secondary subsets, including undersampling of background samples belonging to common objects and oversampling of target samples belonging to objects in disaster scenes;

[0025] Point cloud feature extraction module: used to synchronously input the resampled point cloud data into the teacher network and student network feature extractors to obtain the point cloud features of the teacher network and student network respectively;

[0026] Training loss calculation module: used to generate pseudo-labels by combining the prediction results of the teacher network with the original true labels of the target class; and to calculate the training loss by using the pseudo-labels and the prediction results of the student network.

[0027] Distillation loss calculation module: used to synchronously perform hierarchical pooling on the point cloud features of the teacher network and student network to obtain pooled features; and use the pooled features to calculate distillation loss.

[0028] The network parameter optimization module is used to backpropagate and train the student network using the total loss obtained by summing the distillation loss and the training loss. It optimizes the parameters of the student network and obtains the optimized parameters of the student network after training is completed on all subsets.

[0029] Network Application Evaluation Unit: The network application unit is used to process point cloud data on the student network initialization validation set using the optimized parameters, and compare the obtained semantic-level predictions and instance-level masks with the true categories and masks labeled to evaluate the point cloud data segmentation effect.

[0030] Compared with the prior art, the beneficial effects of the present invention are:

[0031] (1) For actual disaster application scenarios, a point cloud segmentation dataset is constructed and the dataset is divided into multiple subsets. Each main subset contains common objects in general scenarios, and each sub-subset contains specific objects in disaster scenarios. The disaster severity of specific objects in disaster scenarios is further refined. While retaining the ability to process common objects, the dataset learns to learn newly added post-disaster objects or objects that need to be focused on after a disaster, so as to realize the recognition of all objects in disaster scenarios.

[0032] (2) In response to the need for data segmentation of specific object classes (new classes) after a disaster, during the continuous learning process, the new class objects in the sub-task data are resampled to obtain samples that are of special interest to the new class data. This avoids the problem of class imbalance in subsequent learning tasks due to the small proportion of new class data in the whole data, thereby effectively promoting the network to improve its ability to distinguish new classes.

[0033] (3) During continuous learning, the teacher network prediction results are used to generate pseudo-labels by combining the real labels of the target class. The difference between the pseudo-labels and the prediction results of the student network is used to obtain the training loss. By performing synchronous hierarchical pooling on the features extracted by the teacher network and the student network, the difference between the pooling features is used to obtain the distillation loss, and then the loss of continuous network learning is obtained. This allows the network to build loss without relying on old classes, solving the problem of the large increase in computer memory due to the continued existence and addition / reduction of old classes. Since both the training loss and the distillation loss are related to the teacher network, and the teacher network has already trained the principal subset (including regular objects) during pre-training, the teacher network can be used to guide the student network to learn about newly added post-disaster objects or objects of focus. By guiding the generation of student network features through the teacher network features, it helps the student network to retain the review of old knowledge while learning new classes, alleviate the problem of catastrophic forgetting, and achieve rapid identification of disaster elements, thereby meeting the requirements of urgency. Attached Figure Description

[0034] Figure 1 A flowchart of a segmentation method provided in one embodiment;

[0035] Figure 2 A flowchart of the segmentation method in another embodiment;

[0036] Figure 3 A flowchart of the segmentation method in another embodiment;

[0037] Figure 4 A flowchart of the segmentation method in another embodiment;

[0038] Figure 5 A flowchart of the segmentation method in another embodiment;

[0039] Figure 6 This is a schematic diagram of the distillation feature structure in a segmentation method in one embodiment;

[0040] Figure 7 This is a diagram illustrating the effect of the continuous learning method in the segmentation method of one embodiment;

[0041] Figure 8 Here is a structural block diagram of the segmentation system provided in another embodiment;

[0042] Figure 9 This is a block diagram of the continuous learning structure in one embodiment. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0044] like Figure 1 As shown, this invention provides a method for segmenting point cloud data in disaster scenarios, comprising:

[0045] S101 Point Cloud Data Acquisition and Labeling: Acquire manually labeled conventional urban point cloud data; acquire manually labeled open disaster scene point cloud data, wherein the open disaster scene point cloud data includes a learning set and a validation set, wherein the learning set includes a primary subset and a secondary subset, wherein the labeling of the primary subset includes only common objects in general outdoor scenes, and the labeling of the secondary subset includes only one or more disaster scene objects.

[0046] S102 Network Pre-training and Transfer: Construct a network structure for segmenting the conventional urban point cloud data; perform pre-training on the conventional urban point cloud data to obtain network pre-training parameters; transfer the pre-trained network structure and pre-training parameters to the principal subset;

[0047] S103 Teacher network and student network initialization: The migrated network is used as the teacher network and student network respectively. The student network is used to perform continuous learning in the sub-subset. When continuous learning begins in each sub-subset, the network parameters obtained from the continuous learning of the previous sub-subset are used to initialize the teacher network and student network, and then the teacher network is frozen.

[0048] S104 Subset Data Resampling: The point cloud data in the subset is resampled, including undersampling of background samples that belong to common objects and oversampling of target samples that belong to objects in disaster scenes;

[0049] S105 Point cloud feature extraction of teacher network and student network: The resampled point cloud data is synchronously input into the teacher network and student network feature extractor to obtain the point cloud features of the teacher network and student network respectively.

[0050] S106 Training Loss and Distillation Loss Calculation: Input the obtained point cloud features of the teacher network and student network into the decoder to generate prediction results for the teacher network and student network respectively; use the prediction results of the teacher network and combine them with the original true label annotation of the target class to generate pseudo labels, and calculate the training loss based on the difference between the pseudo labels and the prediction results of the student network; perform hierarchical pooling on the point cloud features of the teacher network and student network simultaneously to obtain hierarchical pooling features of the teacher network and student network respectively, and calculate the distillation loss based on the difference between the two hierarchical pooling features.

[0051] S107 Student Network Parameter Optimization: The distillation loss and the training loss are summed to obtain the total loss. The student network is trained by backpropagation based on the total loss to optimize the student network parameters. After training is completed on all subsets, the optimized parameters of the student network are obtained.

[0052] S108: Evaluation of point cloud data segmentation effect in disaster scenarios: Using the optimized parameters, the point cloud data on the validation set is processed after initialization of the student network to obtain semantic-level predictions and instance-level masks. The obtained semantic-level predictions and instance-level masks are compared with the labeled true categories and masks to evaluate the point cloud data segmentation effect.

[0053] In this embodiment of the invention, for step S101, typical test environment and case UAV laser point cloud data are used as raw data, covering complete multi-view images and point cloud information of Beichuan County; by labeling disaster object data, semantic classes are defined as: background, vegetation, vehicles, water, roads, houses, damaged houses and ruins, and instance classes are defined as: collapsed houses, roads, and ruins; the main subset includes 5 types of objects (background, vegetation, vehicles, water, houses), and the secondary subset includes 3 types of objects (roads, ruins, damaged houses). Classes not included in each subset are set as background classes; at the same time, the degree of disaster in the objects in the secondary subset is finely divided, thereby training the network to make a refined judgment on the degree of disaster;

[0054] For step S102, since there is relatively little data on disaster scenarios, directly using the initialized network for training can easily lead to overfitting or insufficient generalization. Therefore, model pre-training is first carried out on a large urban dataset, including eight networks such as PointGroup, HAIS, SoftGroup, DyCo3D, DKNet, BSeg, Mask3D, and OneFormer3D as the backbone network. The BSeg, Mask3D, DKNet, and SoftGroup networks that performed well during the pre-training process are then transferred to the principal subset of the disaster scenario point cloud data.

[0055] For step S103, when continuous learning on disaster scenario point cloud data begins, the transferred network is used as the teacher network and the student network respectively. The student network is used to train on the sub-subset. When training begins on each sub-subset, the teacher network and the student network are initialized using the parameters after training on the previous sub-subset. Then the teacher network is frozen, that is, when training begins on each sub-subset, the teacher network and the student network are exactly the same.

[0056] For step S104, since the current subset only contains background and target classes, and the background class consists of true background classes and pseudo background classes (vegetation, vehicles, water, houses) that have been learned in the pre-training task, the target class accounts for a small proportion of the entire data. This leads to class imbalance in subsequent learning tasks, which can easily cause overfitting to the background class (old class) and neglect of the target class (new class). Therefore, before each round of training, it is necessary to oversample the target class samples of objects in the disaster scene in the current subtask dataset to pay special attention to the newly added target class (new class) data, thereby effectively prompting the network to improve its ability to distinguish new classes.

[0057] For step S105, the point cloud features of the teacher network and the student network are obtained by using the feature extractors of the teacher network and the student network respectively. For the same subset, the point cloud features of the teacher network and the student network are the same in the first round of training. Starting from the second round of training, the point cloud features of the teacher network and the student network are different due to the changes in the student network.

[0058] For step S106, based on a dynamic threshold, background class labels judged as low-quality in the prediction results of the teacher network are removed to improve the reliability of the prediction. Original true label annotations of the target class data are added to generate pseudo-labels. The training loss is obtained based on the difference between the pseudo-labels and the prediction results of the student network. Distillation loss is obtained by simultaneously performing hierarchical pooling on the point cloud features of each layer obtained by the teacher and student networks, and evaluating the difference between the pooling features of each layer. In the first round of training for each subset, the distillation loss is 0 because the pooling features of the teacher and student networks are the same. From the second round of training onwards, distillation loss occurs due to changes in the student network. The training loss and distillation loss are then added together. The total loss is obtained to train and optimize the student network. Since both the training loss and distillation loss are related to the teacher network, and the teacher network has already been trained on the principal subset (including regular objects) during pre-training, it can be used to guide the student network to learn about newly added post-disaster objects or objects of focus. By guiding the generation of student network features through the teacher network features, the student network can retain the review of old knowledge while learning new classes, alleviating the problem of catastrophic forgetting. This allows the network to avoid building loss based on old classes, solving the problem of the large increase in computer memory required for the continuation of old classes. At the same time, by using the teacher network trained on the principal subset to guide the student network's learning, it is possible to quickly identify post-disaster objects in new classes.

[0059] As another preferred embodiment of the present invention, a method for segmenting point cloud data in disaster scenarios is provided. In step S101, the network structure includes a structure built with a sparse U-Net network as the backbone network.

[0060] In this embodiment, the network is pre-trained on the conventional urban point cloud instance segmentation data. The preferred backbone networks with better performance are BSeg, Mask3D, DKNet and SoftGroup. At the same time, the pre-training parameters of various networks are obtained. These backbone networks can effectively deal with outdoor scene point cloud data. These networks are selected for subsequent learning experiments.

[0061] like Figure 2 As shown, in another preferred embodiment of the present invention, a method for segmenting point cloud data in disaster scenarios is provided. Step S102, the step of transferring the network and pre-trained parameters over the principal subset, includes:

[0062] S201: Initialize the network using pre-trained parameters;

[0063] S202: A multilayer perceptron network is connected after the network feature extractor, and LeakyReLU is used as the activation function. The feature offsets extracted by the network on each principal subset are obtained through training.

[0064] S203: Summing the network-extracted features with the feature offsets to obtain new features;

[0065] S204: Based on new features, with a pre-training learning rate of 0.2-0.3 times, fine adjustments are made to the pre-trained network parameters and network structure after training on each principal subset.

[0066] In this embodiment, to reduce the impact of the domain migration process on the original network, a multilayer perceptron (MLP) network is connected after the network feature extractor to learn the feature offsets extracted by the network on different datasets. The network is then migrated to the point cloud data of the disaster scene and trained on the principal subset based on the new features and pre-trained network parameters. The network structure is trained with a small learning rate and fine adjustments are made to achieve a rapid response to the disaster scene.

[0067] The ablation experiments in Table 1-2 are based on the BSeg network. "Baseline" refers to training directly without pre-training; "transferring w / o MLP" refers to pre-training followed by transfer learning without adding an MLP; and "transferring w / MLP" refers to pre-training followed by MLP feature shifts for transfer learning. The quantitative results in Table 1-2 show that using the pre-trained and feature-shifted MLP network significantly improves instance segmentation metrics AP, AP50, and AP25, as well as semantic segmentation metrics Accuracy and mIoU. Furthermore, it achieves optimal results for all object segmentation metrics, indicating that the pre-training method and the feature-shifted MLP have a positive impact on instance and semantic segmentation.

[0068] Table 3-4 presents a comparative experiment on model transfer (adding MLP feature shift features after pre-training) of four networks for disaster scenarios. The quantitative results in Table 3-4 show that after using the pre-trained and feature-shifted MLP network, better results were achieved in semantic segmentation of various object metrics.

[0069] Table 1. Experimental results of model transfer ablation for disaster scenarios.

[0070]

[0071] Table 2. Cross-Union Comparison Results of Model Transfer Ablation Experiments for Disaster Scenarios

[0072]

[0073] Table 3. Comparative Experiment of Model Transfer Learning with Four Networks for Disaster Scenarios (Transferring w / MLP)

[0074]

[0075] Table 4. Cross-Union Ratio (CUI) Results of Model Transfer Experiments for Disaster Scenarios (w / MLP)

[0076]

[0077] like Figure 3 The method for segmenting point cloud data in disaster scenarios, provided as another preferred embodiment of the present invention, includes step 104, which involves resampling the segmented point cloud instance data in the secondary subset, comprising:

[0078] S301: Count the number of points belonging to the target class and the number of points not belonging to the target class in the point cloud data;

[0079] S302: Calculate the ratio of the number of target class points to the number of non-target class points as the sampling probability of non-target class samples, and take the value 1 as the sampling probability of target class samples;

[0080] S303: Sample and generate training samples;

[0081] In this embodiment, the target class is resampled based on the sampling probabilities of non-target class samples and target class samples. This overcomes the problem of class imbalance in subsequent learning tasks caused by the small proportion of the target class in the entire data, which can easily lead to overfitting to old classes and neglecting new classes. The subsample set D... t The probability ρ of selecting each sample ti for:

[0082]

[0083] Where: T is the target class, N t Where N is the number of samples in the target class, N0 is the number of samples in other classes, and D is the number of samples in other classes. t For some second subset, d t For a certain sample.

[0084] like Figure 4 As shown, in another preferred embodiment of the present invention, a method for segmenting point cloud data in disaster scenarios is provided. Step 105, which involves generating pseudo-labels by combining the prediction results of the teacher network with the original true labels of the target class, and calculating the training loss based on the difference between the pseudo-labels and the prediction results of the student network, includes:

[0085] S401: Use the teacher network to predict semantic probabilities and calculate the difference between the average predicted probability and the maximum predicted probability for each type of object;

[0086] Calculate the ratio between the difference between the average predicted probability and the maximum predicted probability. If it is greater than a fixed value, take the minimum predicted probability as the semantic threshold; otherwise, take the maximum value between the minimum predicted probability and the fixed threshold as the semantic threshold.

[0087] Predicted points below the semantic threshold are classified as background points, while predicted points above the semantic threshold are processed according to the actual prediction results. The true category labeled by the target class is added to generate pseudo-semantic labels.

[0088] S402: Use pseudo-semantic labels to remove mask points belonging to the background class in the teacher network prediction mask;

[0089] Using a teacher network to predict mask scores, the difference between the average predicted mask score and the maximum predicted score is calculated.

[0090] Calculate the ratio between the predicted average score and the predicted maximum score difference. If it is greater than a fixed value, take the minimum mask score as the mask threshold; otherwise, take the maximum value between the minimum mask score and the fixed threshold as the mask threshold.

[0091] Masks smaller than the mask threshold are removed, and masks larger than the mask threshold are processed as actual predicted masks. The actual mask labeled with the target class is added to generate a pseudo instance mask.

[0092] S403: Use the teacher network's predicted instance mask score as the pseudo-mask score, and calculate the intersection-union ratio between the pseudo-instance mask corresponding to the pseudo-mask score and the corresponding instance mask of the student network as the loss weight.

[0093] S404: If the network has a mask center location prediction branch, use the teacher network to predict the center location and calculate the mean and standard deviation of the predicted center.

[0094] The difference between the average value of the mask center and the predicted center is calculated using the pseudo-instance mask. If the difference is greater than the standard deviation, the calculated mask center is removed. If the difference is less than the standard deviation, the center is retained as the pseudo-center position.

[0095] S405: Compare the pseudo-semantic labels, pseudo-instance masks, pseudo-mask scores, and pseudo-center positions with the student network prediction results to calculate the training loss.

[0096] In this embodiment, for step S401, the current training subset includes a target class and a background class. The background class includes a pseudo-background class (regular objects, which have been learned in the pre-training task) and a true background class. The teacher network can be used to perform semantic prediction on regular objects. The set dynamic threshold can select an appropriate threshold according to the probability distribution. When all probabilities of a certain class (or a certain instance) are small, a fixed threshold is used. When there are points with high probabilities and the distribution within the class is relatively concentrated, the prediction of that class is considered more reliable, and the minimum value is used as the threshold. When there are points with high probabilities and the distribution within the class is relatively scattered, the low-scoring prediction is considered unreliable, and the maximum value of the minimum value and the fixed threshold is used as the threshold to remove low-scoring thresholds. Based on the dynamic semantic threshold, semantic labels smaller than the semantic threshold are considered low-quality and unreliable, and are classified as background class and removed. Semantic labels larger than the mask threshold are retained, and the true category labeled by the target class is added to generate pseudo-semantic labels.

[0097] For step S402, based on the generation process of pseudo-semantic labels, the mask points classified as true background class in the teacher network prediction mask are first removed; then the pseudo background class masks that are less than the mask threshold are also removed, and the masks that are greater than the mask threshold are retained as actual prediction masks, and the real mask labeled with the target class label is added to generate pseudo instance masks.

[0098] For step S403, the larger the crossover ratio between the pseudo-instance mask corresponding to the pseudo-mask score and the corresponding instance mask of the student network, the greater the fit between the pseudo-instance mask and the corresponding instance mask of the student network, and the greater the designed training loss.

[0099] For step S404, if the network has a mask center location prediction branch, the difference between the pseudo-center of the mask and the average value of the teacher network prediction center is calculated using the pseudo-instance mask and compared with the standard deviation to determine whether to retain the center as the pseudo-center location.

[0100] For step S405, the pseudo-semantic labels, pseudo-instance masks, pseudo-mask scores, and pseudo-center locations are compared with the results predicted by the student network to obtain the training loss.

[0101]

[0102] In the formula: The results are calculated from the semantic judgment result, instance mask, instance center, and instance score, respectively.

[0103] To achieve better training results, the difference comparison method is the same as the training loss acquisition method in the pre-training of the network (teacher network). Thus, while the network learns new classes, it reviews the knowledge of old classes. Under the guidance of the teacher network, the network can construct the distillation loss without relying on old classes, thus solving the problem that the continued use of old classes greatly increases computer memory.

[0104] like Figure 5-6 As shown, in another preferred embodiment of the present invention, a method for segmenting point cloud data in disaster scenarios is provided. In step 105, the point cloud features of the teacher network and the student network are simultaneously subjected to hierarchical pooling to obtain the hierarchical pooling features of the teacher network and the student network, respectively. The step of calculating the distillation loss based on the difference between the two hierarchical pooling features includes:

[0105] For low-level features, S501 uses dilated average pooling layers with different jumps to extract multi-scale pooling features.

[0106] For high-level features, S502 uses a second-order pooling layer to extract context-dependent pooling features.

[0107] S503 calculates L2 loss as distillation loss based on the pooling characteristics of each layer of the teacher network and student network.

[0108] In this embodiment, for step 501, low-level convolutional features contain rich spatial details (such as edges, textures and local structural information), but they are sensitive to position and have a low level of semantic abstraction. Therefore, dilated spatial pyramid pooling (ASPP) is used to process the features. By applying dilated convolutional operations with different dilation rates in parallel, ASPP can effectively capture multi-scale local contextual information without significantly reducing the spatial resolution of the feature map.

[0109] For step 502, high-level convolutional features, due to layer-by-layer abstraction by deep networks, possess strong semantic meaning (representing object parts, wholes, or scene concepts) and spatial robustness (insensitive to changes in position and scale). However, spatial detail information is significantly attenuated. Therefore, global second-order pooling is used to process the features. This describes how different feature channels co-variate by calculating the covariance matrix.

[0110] For step 503, the differences between pooling features at each layer of the teacher network and the student network are evaluated using L2 loss. As the number of training rounds increases, the weight of the KL divergence L2 loss gradually decreases to prevent the original knowledge from limiting the network's learning. At the same time, the deeper the feature, the higher its importance is, and it is given a higher weight. The loss weights for each layer are as follows:

[0111]

[0112] In the formula: λ is a constant; n l N l Here, γ represents the current layer and the total number of layers for this type of pooling method; γ is a constant less than 1, and n... e N e This represents the current training round number and the total number of rounds.

[0113] By designing dynamic loss weights, different weights are applied to the distillation loss at different layers and rounds to obtain the final distillation loss. This allows the teacher network features to guide student feature generation. As the network learns new classes, it reviews knowledge of older classes, enabling the construction of the distillation loss without relying on old classes. This solves the problem of significantly increasing computer memory usage when reusing old classes.

[0114] The total loss l is obtained by adding the original training loss to the distillation loss:

[0115] l = l train +Ση KDi l KDi

[0116] In the formula: l KDi The loss per layer of distillation is calculated from the L2 loss.

[0117] Table 5 presents the experimental results after pseudo-label generation, data resampling, and feature distillation during continuous learning. It can be seen from the quantitative data in Table 5 that adding data resampling on the basis of pseudo-label generation can effectively improve the network's ability to distinguish new classes. After generating pseudo-labels for training, data resampling and distillation of features between the teacher network and the student network can guide the student network with the features of the teacher network, thus alleviating the catastrophic forgetting problem.

[0118] Table 5. Results of ablation experiments and cross-union comparisons of the continuous learning segmentation framework for point clouds in disaster scenarios.

[0119]

[0120] Figure 7Qualitative results of continuous learning performance are presented, where the leftmost images represent the original image, semantic labels, and mask labels, respectively. (a) shows the result after adding the pseudo-label generation module, (b) shows the result after adding the data resampling module, and (c) shows the result after adding the feature distillation module. Figure 7 As can be seen, in the continuous learning process, based on the generation of pseudo-labels, data resampling and feature distillation are successively added, making the predicted semantic-level predictions and instance-level masks increasingly closer to the labeled true categories and masks, thus further demonstrating the effectiveness of the continuous learning method in this invention.

[0121] This invention transfers the pre-trained network to both the student and teacher networks. The student network is trained using data from the secondary subset. During continuous learning, pseudo-labels are generated by combining the teacher network's predictions with the target class's true labels. The difference between the pseudo-labels and the student network's predictions is used to obtain the training loss. Simultaneous hierarchical pooling is performed on the features extracted from both the teacher and student networks. Distillation loss is obtained by utilizing the differences between the pooled features, thus yielding the loss for continuous network learning. Since both the training loss and distillation loss are related to the teacher network, and the teacher network has already been trained on the principal subset (including regular objects) during pre-training, the teacher network can guide the student network to learn about newly added post-disaster objects or objects of particular interest. Guiding student network feature generation through teacher network features helps the student network retain old knowledge while learning new classes, mitigating catastrophic forgetting. This allows the network to avoid building loss based on old classes, solving the problem of significantly increasing computer memory usage due to the addition and removal of old classes.

[0122] After continuous learning across all subsets, the optimized student network is used to process point cloud data on the validation set. The obtained semantic-level predictions and instance-level masks are compared with the labeled ground truth categories and masks to evaluate the point cloud data segmentation effect. According to the intersection-union ratio (IU) data obtained from the experiment in Table 6, among the four networks BSeg, Mask3D, DKNet, and SoftGroup, the IU value of the prediction results using the DKNet network is the highest, indicating that it has the greatest fit with the labeled ground truth categories and masks, and the best effect on object segmentation and discrimination in disaster scenes. Furthermore, using the DKNet network, the IU values ​​for vegetation, water, vehicles, houses, and ruins are close to or exceed 0.7, demonstrating that optimizing and improving the DKNet network after training using the continuous learning described in this invention can effectively segment and distinguish objects in disaster scenes, thus providing strong support for emergency rescue decision-making in disaster areas.

[0123] Table 6. Comparative Experiment Results of Intersection over Union (IoU) for Constructing Continuous Learning Segmentation Frameworks for Point Clouds in Disaster Scenarios

[0124]

[0125] like Figure 8-9 As shown, another embodiment of the present invention provides a segmentation system for point cloud data in disaster scenarios, the system comprising:

[0126] The S601 point cloud data acquisition and annotation unit is used to acquire manually annotated conventional urban point cloud data; and to acquire open disaster scene point cloud data and perform manual annotation. The open disaster scene point cloud data includes a learning set and a validation set. The learning set includes a primary subset and a secondary subset. The labels of the primary subset only include common objects in general outdoor scenes, and the labels of the secondary subset only include one or more disaster scene objects.

[0127] S602 Network Pre-training Unit, the network pre-training unit is used to pre-train the network on manually labeled conventional urban point cloud data to obtain network pre-training parameters;

[0128] S603 Network Migration Unit, the network migration unit is used to migrate the network and pre-trained parameters on the principal subset of manually labeled open disaster scene point cloud data;

[0129] S604 Network Continuous Learning Unit, which is used to treat the transferred network as both a teacher network and a student network, and to perform continuous learning on a sub-subset using the student network to obtain student network parameters; the network continuous learning unit specifically includes:

[0130] S6041 Initialization Module: Used to initialize the teacher network and student network with the network parameters obtained from the previous sub-subset when the student network starts continuous learning at each sub-subset, and then freeze the teacher network;

[0131] S6042 Subset Data Resampling Module: Used to resample point cloud data in the subset, including undersampling of background samples belonging to common objects and oversampling of target samples belonging to disaster scene objects;

[0132] S6043 Point Cloud Feature Extraction Module: Used to synchronously input the resampled point cloud data into the teacher network and student network feature extractors to obtain the point cloud features of the teacher network and student network respectively;

[0133] S6044 Training Loss Calculation Module: Used to generate pseudo-labels by combining the prediction results of the teacher network with the original true labels of the target class; and to calculate the training loss by using the pseudo-labels and the prediction results of the student network.

[0134] S6045 Distillation Loss Calculation Module: Used to synchronously perform hierarchical pooling on the point cloud features of the teacher network and the student network to obtain pooling features; and to calculate the distillation loss by utilizing the difference between the pooling features of the two networks.

[0135] S6046 Student Network Parameter Optimization Module; used to train the student network by backpropagation using the total loss obtained by summing the distillation loss and training loss, optimize the student network parameters, and obtain the optimized parameters of the student network after training on all subsets;

[0136] The S605 network application unit uses the optimized parameters to process point cloud data on the student network initialization validation set, and compares the obtained semantic-level predictions and instance-level masks with the true categories and masks labeled to evaluate the point cloud data segmentation effect.

[0137] In this embodiment, the description of the above system corresponds to the segmentation method. For a description of the above segmentation method, please refer to the above text, which will not be repeated here.

[0138] As another preferred embodiment of the present invention, a segmentation system for point cloud data in disaster scenarios is provided, wherein the network migration unit specifically includes:

[0139] Transfer initialization module: Initializes the network using pre-trained parameters;

[0140] Domain transfer module: By connecting a multilayer perceptron network after the network feature extractor and using LeakyReLU as the activation function, the feature offsets extracted by the network on each principal subset are learned; and the network-extracted features are summed with the feature offsets to obtain new features;

[0141] Transfer fine-tuning module: Using new features and a value of 0.2-0.3 times the pre-training learning rate, fine-tunes the parameters and structure of the pre-trained network transferred to each principal subset.

[0142] In this embodiment, the description of the above system corresponds to the segmentation method. For a description of the above segmentation method, please refer to the above text, which will not be repeated here.

[0143] As another preferred embodiment of the present invention, a segmentation system for point cloud data in disaster scenarios is provided, wherein the training loss calculation module includes:

[0144] The pseudo-semantic label generation module uses the teacher network to predict semantic probabilities, calculates the ratio between the average predicted probability and the maximum predicted probability of each object class, and if it is greater than a fixed value, takes the minimum predicted probability as the semantic threshold; otherwise, takes the maximum value between the minimum predicted probability and the fixed threshold as the semantic threshold. Predicted points below the semantic threshold are classified as background, and predicted points above the semantic threshold are processed according to the actual predicted category. The module also adds the real category labeled by the target class to generate pseudo-semantic labels.

[0145] The pseudo-instance mask generation module uses pseudo-semantic tags to remove mask points belonging to the background class from the teacher network prediction mask; it uses the teacher network prediction mask score to calculate the ratio between the predicted average score and the predicted maximum score difference. If the ratio is greater than a fixed value, the minimum mask score is taken as the mask threshold; otherwise, the maximum value between the minimum mask score and the fixed threshold is taken as the mask threshold. Masks smaller than the mask threshold are removed, and masks larger than the mask threshold are processed as actual prediction masks and real masks labeled with the target class are added to generate pseudo-instance masks.

[0146] The pseudo-mask scoring generation module calculates the maximum intersection-union ratio (CIU) between the student network prediction mask and the pseudo-instance mask, uses the mask score of the pseudo-instance mask as the pseudo-mask score, and uses the CIU as the loss weight.

[0147] Pseudo-center location generation module: If the network has a mask center location prediction branch, the teacher network is used to predict the center location, and the mean and standard deviation of the predicted center are calculated; the difference between the mask center and the mean of the predicted center is calculated using the pseudo-instance mask. If it is greater than the standard deviation, the calculated mask center is removed; if it is less than the standard deviation, the center is retained as the pseudo-center location.

[0148] Training loss acquisition module: The training loss is calculated by comparing the pseudo-semantic labels, pseudo-instance masks, pseudo-mask scores, and pseudo-center locations with the student network prediction results.

[0149] In this embodiment, the description of the above system corresponds to the segmentation method. For a description of the above segmentation method, please refer to the above text, which will not be repeated here.

[0150] As another preferred embodiment of the present invention, a segmentation system for point cloud data in disaster scenarios is provided, wherein the distillation loss calculation module includes:

[0151] The low-level pooling module uses dilated average pooling layers with different jumps to extract multi-scale pooling features.

[0152] High-level pooling module: Uses second-order pooling layers to extract context-dependent pooling features;

[0153] Distillation loss acquisition module: Based on the pooling characteristics of each layer of the teacher network and student network, calculate L2 loss as distillation loss.

[0154] In this embodiment, the description of the above system corresponds to the segmentation method. For a description of the above segmentation method, please refer to the above text, which will not be repeated here.

[0155] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0156] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0157] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A method for segmenting point cloud data in a disaster scene, characterized in that, The method includes the following steps: Obtain manually labeled regular urban point cloud data; obtain manually labeled open disaster scene point cloud data, wherein the open disaster scene point cloud data includes a learning set and a validation set, wherein the learning set includes a principal subset and a secondary subset, wherein the annotations of the principal subset only include common objects in general outdoor scenes, and the annotations of the secondary subset only include one or more disaster scene objects. A network structure for segmenting the conventional urban point cloud data is constructed; the network is pre-trained on the conventional urban point cloud data to obtain network pre-training parameters; the pre-trained network structure and pre-training parameters are transferred to the principal subset; The migrated networks are used as teacher networks and student networks, respectively. The student networks are used for continuous learning in the sub-subsets. When continuous learning begins in each sub-subset, the network parameters obtained from the continuous learning of the previous sub-subset are used to initialize the teacher network and student network, and then the teacher network is frozen. The point cloud data in the secondary subset is resampled, with undersampling of background samples belonging to common objects and oversampling of target samples belonging to disaster scene objects; The resampled point cloud data is synchronously input into the feature extractors of the teacher network and the student network to obtain the point cloud features of the teacher network and the student network respectively. The point cloud features of the obtained teacher network and student network are input into the decoder to generate prediction results for the teacher network and student network respectively. Using the prediction results of the teacher network and the original true label annotation of the target class, pseudo labels are generated. The training loss is calculated based on the difference between the pseudo labels and the prediction results of the student network. The point cloud features of the teacher network and student network are simultaneously subjected to hierarchical pooling to obtain hierarchical pooling features of the teacher network and student network respectively. The distillation loss is calculated based on the difference between the two hierarchical pooling features. The distillation loss and the training loss are summed to obtain the total loss. Backpropagation is performed based on the total loss to train the student network. The parameters of the student network are optimized. After training is completed on all subsets, the optimized parameters of the student network are obtained. Using the optimized parameters, the point cloud data on the validation set is processed after initializing the student network to obtain semantic-level predictions and instance-level masks. The semantic-level predictions and instance-level masks are then compared with the true categories and masks labeled on the validation set to evaluate the point cloud data segmentation effect.

2. The method of claim 1, wherein, The network structure includes a structure built with a sparse U-Net network as the backbone network.

3. The method of claim 1, wherein, The steps for transferring the network and pre-trained parameters to the principal subset include: Initialize the network using pre-trained parameters; A multilayer perceptron network is connected after the network feature extractor, and LeakyReLU is used as the activation function. The network feature extraction offset on each principal subset is obtained through training. The new feature is obtained by summing the features extracted by the network with the feature offset; Based on the new features, the parameters and structure of the pre-trained network transferred to each principal subset are finely adjusted by using a value of 0.2-0.3 times the pre-training learning rate.

4. The method of claim 1, wherein, The step of resampling the point cloud data in the secondary subset includes: Count the number of points in the point cloud data that belong to the target class and the background class; Calculate the ratio of the number of target class points to the number of background class points as the sampling probability of the background class sample, and take the value 1 as the sampling probability of the target class sample; Training samples are generated through sampling.

5. The method of claim 1, wherein, The steps of generating pseudo-labels by combining the prediction results of the teacher network with the original true labels of the target class, and calculating the training loss based on the difference between the pseudo-labels and the prediction results of the student network include: Using a teacher network to predict semantic probabilities, the difference between the average predicted probability and the maximum predicted probability for each object class is calculated. Calculate the ratio between the difference between the average predicted probability and the maximum predicted probability. If it is greater than a fixed value, take the minimum predicted probability as the semantic threshold; otherwise, take the maximum value between the minimum predicted probability and the fixed threshold as the semantic threshold. Predicted points below the semantic threshold are classified as background points, while predicted points above the semantic threshold are processed according to the actual prediction results. The true category labeled by the target class is added to generate pseudo-semantic labels. Using pseudo-semantic labels, mask points belonging to the background class in the teacher network prediction mask are removed; Using a teacher network to predict mask scores, the difference between the average predicted mask score and the maximum predicted score is calculated. Calculate the ratio between the predicted average score and the predicted maximum score difference. If it is greater than a fixed value, take the minimum mask score as the mask threshold; otherwise, take the maximum value between the minimum mask score and the fixed threshold as the mask threshold. Masks smaller than the mask threshold are removed, and masks larger than the mask threshold are processed as actual predicted masks. The actual mask labeled with the target class is added to generate a pseudo instance mask. The teacher network predicted instance mask scores were used as pseudo mask scores, and the cross-union ratio between the pseudo instance mask corresponding to the pseudo mask score and the corresponding instance mask of the student network was calculated as the loss weight. If the network has a mask center location prediction branch, the teacher network is used to predict the center location, and the mean and standard deviation of the predicted center are calculated. The difference between the average value of the mask center and the predicted center is calculated using the pseudo-instance mask. If the difference is greater than the standard deviation, the calculated mask center is removed. If the difference is less than the standard deviation, the center is retained as the pseudo-center position. The training loss is calculated by comparing the pseudo-semantic labels, pseudo-instance masks, pseudo-mask scores, and pseudo-center positions with the prediction results of the student network.

6. The method of claim 1, wherein, The step of simultaneously performing hierarchical pooling on the point cloud features of the teacher network and the student network to obtain hierarchical pooling features of the teacher network and the student network respectively, and calculating the distillation loss based on the difference between the two hierarchical pooling features includes: For low-level features, multi-scale pooling features are extracted using dilated average pooling layers with different jumps. For high-level features, a second-order pooling layer is used to extract context-dependent pooling features; Based on the pooling characteristics of each layer of the teacher network and student network, L2 loss is calculated as distillation loss.

7. A segmentation system for point cloud data in disaster scenarios, characterized in that, The system includes: Point cloud data acquisition and annotation unit: The point cloud data acquisition and annotation unit is used to acquire manually annotated conventional urban point cloud data; and to acquire open disaster scene point cloud data and perform manual annotation. The open disaster scene point cloud data includes a learning set and a validation set. The learning set includes a primary subset and a secondary subset. The labels of the primary subset only include common objects in general outdoor scenes, and the labels of the secondary subset only include one or more disaster scene objects. A network pre-training unit is used to pre-train the network on manually labeled conventional urban point cloud data to obtain network pre-training parameters. A network migration unit is used to migrate the network and pre-trained parameters on a principal subset of manually labeled open disaster scenario point cloud data. A continuous learning unit for networks, wherein the transferred networks are respectively used as teacher networks and student networks, and continuous learning is performed on sub-subsets using the student network to obtain student network parameters; the continuous learning unit for networks specifically includes: Initialization module: used to initialize the teacher network and student network with the network parameters obtained from the previous continuous learning of each subset when the student network begins continuous learning; and then freezes the teacher network. Secondary subset data resampling module: used to resample point cloud data in secondary subsets, including undersampling of background samples that belong to common objects and oversampling of target samples that belong to objects in disaster scenes; Point cloud feature extraction module: used to synchronously input the resampled point cloud data into the teacher network and student network feature extractors to obtain the point cloud features of the teacher network and student network respectively; Training loss calculation module: used to generate pseudo-labels by combining the prediction results of the teacher network with the original true labels of the target class; and to calculate the training loss by using the pseudo-labels and the prediction results of the student network. Distillation loss calculation module: used to synchronously perform hierarchical pooling on the point cloud features of the teacher network and the student network to obtain pooled features; and to calculate the distillation loss by utilizing the difference between the pooled features of the two networks. The network parameter optimization module is used to backpropagate and train the student network using the total loss obtained by summing the distillation loss and the training loss, optimize the parameters of the student network, and obtain the optimized parameters of the student network after training on all subsets. Network Application Evaluation Unit: The network application unit is used to process point cloud data on the student network initialization validation set using the optimized parameters, and compare the obtained semantic-level predictions and instance-level masks with the true categories and masks labeled to evaluate the point cloud data segmentation effect.

8. A segmentation system for point cloud data in disaster scenarios according to claim 7, characterized in that, The network migration unit specifically includes: Transfer initialization module: Initializes the network using pre-trained parameters; Domain transfer module: By connecting a multilayer perceptron network after the network feature extractor and using LeakyReLU as the activation function, the feature offsets extracted by the network on each principal subset are learned; and the network-extracted features are summed with the feature offsets to obtain new features; Transfer Fine-tuning Module: Utilizing new features, this module fine-tunes the parameters and structure of the pre-trained network transferred to each principal subset at a value of 0.2-0.3 times the pre-training learning rate.

9. A segmentation system for point cloud data in disaster scenarios according to claim 7, characterized in that: The training loss calculation module includes: The pseudo-semantic label generation module uses the teacher network to predict semantic probabilities, calculates the ratio between the average predicted probability and the maximum predicted probability of each object class, and if it is greater than a fixed value, takes the minimum predicted probability as the semantic threshold; otherwise, takes the maximum value between the minimum predicted probability and the fixed threshold as the semantic threshold. Predicted points below the semantic threshold are classified as background, and predicted points above the semantic threshold are processed according to the actual predicted category. The module also adds the real category labeled by the target class to generate pseudo-semantic labels. The pseudo-instance mask generation module uses pseudo-semantic tags to remove mask points belonging to the background class from the teacher network prediction mask; it uses the teacher network prediction mask score to calculate the ratio between the predicted average score and the predicted maximum score difference. If the ratio is greater than a fixed value, the minimum mask score is taken as the mask threshold; otherwise, the maximum value between the minimum mask score and the fixed threshold is taken as the mask threshold. Masks smaller than the mask threshold are removed, and masks larger than the mask threshold are processed as actual prediction masks and real masks labeled with the target class are added to generate pseudo-instance masks. The pseudo-mask scoring generation module calculates the maximum intersection-union ratio (CIU) between the student network prediction mask and the pseudo-instance mask, uses the mask score of the pseudo-instance mask as the pseudo-mask score, and uses the CIU as the loss weight. Pseudo-center location generation module: If the network has a mask center location prediction branch, the teacher network is used to predict the center location, and the mean and standard deviation of the predicted center are calculated; the difference between the mask center and the mean of the predicted center is calculated using the pseudo-instance mask. If it is greater than the standard deviation, the calculated mask center is removed; if it is less than the standard deviation, the center is retained as the pseudo-center location. Training loss acquisition module: The training loss is calculated by comparing the pseudo-semantic labels, pseudo-instance masks, pseudo-mask scores, and pseudo-center positions with the student network prediction results.

10. A segmentation system for point cloud data in disaster scenarios according to claim 7, characterized in that, The distillation loss calculation module includes: The low-level pooling module uses dilated average pooling layers with different jumps to extract multi-scale pooling features. High-level pooling module: Uses second-order pooling layers to extract context-dependent pooling features; Distillation loss acquisition module: Based on the pooling characteristics of each layer of the teacher network and student network, calculate L2 loss as distillation loss.

Images