An indoor mobile robot scene perception method based on multi-granularity point cloud segmentation

Abstract

The present application relates to a kind of indoor mobile robot scene perception methods based on multi-granularity point cloud segmentation, including constructing rough segmentation dataset and fine segmentation dataset;From a plurality of candidate three-dimensional point cloud semantic segmentation network, selected reference network, construct teacher network and extract rough segmentation feature, rough segmentation probability and rough segmentation predicted value;Fine segmentation feature, fine segmentation probability and fine segmentation predicted value are extracted by constructing student network;Design loss function, based on the fine-grained class label in the rough segmentation feature extracted by teacher network, fine segmentation dataset, train student network;Using the point cloud segmentation model of training completion, the point cloud semantic segmentation result of indoor scene is generated, to realize the perception of indoor scene by mobile robot.The present application can realize hierarchical semantic segmentation to indoor scene point cloud data, improve the scene understanding ability of mobile robot in path planning, object interaction and other tasks.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of indoor scene perception, and in particular to an indoor mobile robot scene perception method based on multi-granularity point cloud segmentation. BACKGROUND

[0002] In recent years, with the rapid popularization of indoor service robots, warehouse logistics robots and household sweeping robots, the demand for autonomous operation of mobile robots in complex indoor environments is increasing. Scene perception, as a prerequisite for robots to realize autonomous navigation, obstacle avoidance and human-computer interaction, its core lies in accurate semantic understanding of the surrounding environment. Traditional methods mainly rely on two-dimensional laser radar or visual SLAM to construct geometric maps, although relatively accurate spatial profiles can be obtained, but lack of semantic information.

[0003] Due to the wide application of RGB-D cameras and low-cost laser radars, the acquisition of three-dimensional point cloud data has become more convenient, and the semantic segmentation technology based on point cloud has gradually become a research hotspot. This technology assigns semantic labels to three-dimensional point clouds collected by laser radars, providing structured understanding of the scene for robots, and has become a key means of environment perception.

[0004] In the existing indoor scene application process, mobile robots often need to face two types of tasks, "obstacle avoidance" and "interaction". Among them, the obstacle avoidance task only needs to distinguish the ground, passable area and obstacles, which belongs to coarse-grained point cloud segmentation; while the interaction task needs to identify specific furniture or objects, which belongs to fine-grained point cloud segmentation. Most of the existing point cloud segmentation methods adopt a single-granularity labeling system, which cannot meet the different level task requirements of robots. If a model is independently trained for each task, it will significantly increase the storage and computing burden, and it is difficult to realize cross-task knowledge sharing.

[0005] Knowledge distillation, as an effective means of model compression and migration, has been used in image classification, target detection and other fields, but its research in three-dimensional point cloud semantic segmentation is still in its infancy. Existing work mainly focuses on using large offline networks as teachers and lightweight online networks as students, and transferring the similarity between classes through temperature-scaled softmax distribution to reduce the performance loss of small models. However, most of these methods are designed for single-granularity labels and cannot meet the needs of robots for multi-granularity semantics, and lack of systematic solutions for collaborative use of "coarse-fine" labels. SUMMARY

[0006] In view of the deficiencies of the prior art, the present application provides an indoor mobile robot scene perception method based on multi-granularity point cloud segmentation, which solves the problems of lack of semantic information caused by the dependence of traditional scene perception methods on two-dimensional laser radar or visual SLAM to construct geometric maps, the limitation of traditional point cloud segmentation methods to a single granularity annotation system and the lack of systematic solutions for collaborative use of "coarse-fine" labels in existing knowledge distillation frameworks.

[0007] To solve the above technical problems, the present application provides the following technical solutions: an indoor mobile robot scene perception method based on multi-granularity point cloud segmentation, comprising the following steps:

[0008] S1, based on the collected indoor scene three-dimensional point cloud data, constructing a coarse segmentation dataset and a fine segmentation dataset;

[0009] S2, selecting a reference network from a plurality of candidate three-dimensional point cloud semantic segmentation networks based on the coarse segmentation dataset and the fine segmentation dataset, constructing a teacher network and extracting coarse segmentation features, coarse segmentation probabilities and coarse segmentation prediction values;

[0010] S3, constructing a student network and extracting fine segmentation features, fine segmentation probabilities and fine segmentation prediction values; the constructed student network and the constructed teacher network together constitute a point cloud segmentation model based on the teacher-student network architecture;

[0011] S4, designing a loss function, training the student network based on the coarse segmentation features extracted by the teacher network and the fine-grained class labels in the fine segmentation dataset, and the teacher network and the trained student network together constitute a trained point cloud segmentation model;

[0012] S5, using a mobile robot to scan and obtain three-dimensional point cloud data of an indoor scene, generating point cloud semantic segmentation results of the indoor scene based on the trained point cloud segmentation model, including coarse-grained prediction values and fine-grained prediction values, to realize path planning and object interaction tasks of the mobile robot in the indoor scene.

[0013] Further, in step S1, the specific process comprises the following steps:

[0014] S11, for the indoor obstacle avoidance task, the three-dimensional point cloud data containing coarse-grained categories in the indoor scene three-dimensional point cloud data is taken as coarse segmentation data, and is divided into several categories;

[0015] S12, for the indoor interaction task, the three-dimensional point cloud data containing fine-grained categories in the indoor scene three-dimensional point cloud data is taken as fine segmentation data, and is further subdivided on the basis of the categories divided by the coarse segmentation data;

[0016] S13, according to the categories divided in steps S11-S12, point cloud processing software is used to assign coarse-grained category labels and fine-grained category labels to the coarse segmentation data and the fine segmentation data respectively, so as to complete the construction of the coarse segmentation data set and the fine segmentation data set.

[0017] Further, in step S2, the specific process includes the following steps:

[0018] S21, the constructed coarse segmentation data set and fine segmentation data set are sent into a plurality of candidate three-dimensional point cloud semantic segmentation networks, the performance of each candidate three-dimensional point cloud semantic segmentation network is evaluated and compared comprehensively, and a candidate three-dimensional point cloud semantic segmentation network with excellent computing efficiency under the premise of ensuring high accuracy is preferentially selected as a reference network;

[0019] S22, the reference network is pre-trained using the coarse segmentation data set, and the network parameters are optimized until convergence;

[0020] S23, the pre-trained reference network is used as a teacher network and its weight is fixed, the point cloud data in the fine segmentation data set is received at the input end thereof, the features of the last layer and the category probability output at the end are taken as the output of the teacher network, and are defined as coarse segmentation features and coarse segmentation probability respectively;

[0021] S24, Argmax operation is performed on the coarse segmentation probability to obtain high-precision coarse segmentation prediction values, and a reliable understanding ability of scene coarse-grained semantics is formed.

[0022] Further, in step S3, the specific process includes the following steps:

[0023] S31, the reference network with the initialized weight is used as a student network;

[0024] S32, the point cloud data in the fine segmentation data set is input into the student network, the features of the last layer and the category probability output at the end are taken as the output of the student network, and are defined as fine segmentation features and fine segmentation probability respectively;

[0025] S33, Argmax operation is performed on the fine segmentation probability to obtain high-precision fine segmentation prediction values, and ensure the fine-grained semantic recognition ability.

[0026] Further, in step S4, the specific process includes the following steps:

[0027] S41, the coarse segmentation features extracted by the teacher network are used as soft labels to supervise the fine segmentation features extracted by the student network, a KL divergence is introduced to construct a knowledge distillation loss, and the student network is supervised and trained;

[0028] S42, introduce cross-entropy loss, calculate the error between the fine-grained class label in the fine segmentation dataset and the fine segmentation probability output by the student network, and further supervise the training of the student network;

[0029] S43, combine the knowledge distillation loss of step S41 and the cross-entropy loss of step S42 to construct a total loss function, train the student network with the total loss function, and realize the joint supervision of the teacher network and the fine-grained class label on the student network.

[0030] Further, in step S41, the specific process includes the following steps:

[0031] S411, convert the coarse segmentation feature and the fine segmentation feature into corresponding log probability distributions under temperature respectively by temperature scaling and softmax normalization: , :

[0032] ;

[0033] ;

[0034] In the above formula, denotes the softmax normalization operation, denotes the application of the softmax normalization operation in the last dimension;

[0035] S412, calculate the KL divergence of the coarse segmentation feature and the fine segmentation feature under temperature : ;

[0036] In the above formula, denotes the KL divergence cumulative sum,

[0037] denotes the batch size, denotes the number of point clouds, denotes the number of channels of the coarse and fine segmentation features, and the parameter is the base of the natural logarithm; S413, enlarge the calculated by

[0038] times and divide by the total number of elements to compensate for the gradient amplitude change caused by temperature scaling in step S411, to obtain the final knowledge distillation loss :

[0039] ​​ ;

[0040] In the above formula Indicates a knowledge distillation loss calculation operation.

[0041] Further, in step S5, the specific process includes the following steps:

[0042] S51, deploy a laser radar on a mobile robot, scan an indoor scene in real time, and obtain continuous three-dimensional point cloud data;

[0043] S52, input the three-dimensional point cloud data obtained in step S51 into the trained point cloud segmentation model, and generate coarse-grained prediction values and fine-grained prediction values through the teacher network and the trained student network respectively;

[0044] S53, based on the coarse-grained prediction values And the fine-grained prediction values The multi-level semantic information provided by the mobile robot realizes intelligent perception of the indoor scene, and when performing the path planning task, the mobile robot only processes the coarse-grained prediction values, and when performing the object interaction task, the mobile robot directly processes the fine-grained prediction values, and finally effectively performs the path planning and object interaction tasks in the indoor scene.

[0045] Further, the present application also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the indoor mobile robot scene perception method based on multi-granularity point cloud segmentation.

[0046] Further, the present application also provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the indoor mobile robot scene perception method based on multi-granularity point cloud segmentation.

[0047] Through the above technical solution, the present application provides an indoor mobile robot scene perception method based on multi-granularity point cloud segmentation, which at least has the following beneficial effects:

[0048] (1) The present application constructs a point cloud segmentation data set containing coarse-grained and fine-grained semantic labels, and gives the mobile robot hierarchical scene understanding ability, so that it can simultaneously process obstacle avoidance (coarse-grained) and object interaction (fine-grained) tasks;

[0049] (2) The indoor mobile robot scene perception method based on multi-granularity point cloud segmentation can realize efficient and accurate indoor point cloud semantic segmentation by introducing a teacher-student network architecture and organically connecting coarse-grained prior knowledge and fine-grained supervision information, and based on multi-level semantic segmentation results, a mobile robot can make accurate decisions in path planning and object interaction tasks, thereby realizing stable and reliable intelligent applications in a complex indoor environment. BRIEF DESCRIPTION OF DRAWINGS

[0050] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and serve to explain the principles of the application, and do not limit the application. In the drawings:

[0051] Fig. 1 A flowchart of the indoor mobile robot scene perception method based on multi-granularity point cloud segmentation is shown.

[0052] Fig. 2 A structure diagram of the point cloud segmentation model based on the teacher-student network architecture is shown. DETAILED DESCRIPTION

[0053] To make the above purposes, features and advantages of the present application more apparent, specific embodiments of the present application will be described in detail below with reference to the accompanying drawings and specific embodiments. The implementation process of how to apply technical means to solve technical problems and achieve technical effects of the present application can be fully understood and implemented.

[0054] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment methods can be completed by programs instructing related hardware, therefore, the present application can adopt a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt a computer program product in the form of a computer usable storage medium (including but not limited to a disk memory, a CD-ROM, an optical memory, etc.) containing computer usable program codes.

[0055] Please refer to Figs. 1-2 , which shows a specific embodiment of the present embodiment, the present embodiment builds a coarse segmentation dataset and a fine segmentation dataset, builds a teacher-student network architecture based on the coarse and fine segmentation datasets, uses the trained point cloud segmentation model to generate point cloud semantic segmentation results, realizes the perception of indoor scenes by a mobile robot, and effectively performs path planning and object interaction tasks in an indoor scene.

[0056] Please refer to Fig. 1 , the present embodiment proposes an indoor mobile robot scene perception method based on multi-granularity point cloud segmentation, which includes the following steps:

[0057] S1, constructing a coarse segmentation dataset and a fine segmentation dataset based on the collected indoor scene three-dimensional point cloud data;

[0058] As a preferred embodiment of step S1, the specific process includes the following steps:

[0059] S11, for the indoor obstacle avoidance task, the three-dimensional point cloud data containing coarse-grained categories in the indoor scene three-dimensional point cloud data is taken as coarse segmentation data, and the three-dimensional point cloud in the coarse segmentation data is divided into multiple categories including ground, furniture, and doors;

[0060] S12, for the indoor interaction task, the three-dimensional point cloud data containing fine-grained categories in the indoor scene three-dimensional point cloud data is taken as fine segmentation data, and the fine segmentation data is further subdivided on the basis of the categories divided by the coarse segmentation data, such as furniture being subdivided into five subcategories including tables, chairs, and sofas;

[0061] S13, according to the categories divided in steps S11-S12, point cloud processing software is used to assign coarse-grained category labels and fine-grained category labels to the coarse segmentation data and the fine segmentation data respectively, thereby completing the construction of the coarse segmentation dataset and the fine segmentation dataset.

[0062] In this embodiment, by constructing a point cloud segmentation dataset containing coarse-grained and fine-grained semantic labels, the mobile robot is given a hierarchical scene understanding capability, so that it can simultaneously process obstacle avoidance (coarse-grained) and object interaction (fine-grained) tasks.

[0063] S2, selecting a reference network from multiple candidate three-dimensional point cloud semantic segmentation networks based on the coarse segmentation dataset and the fine segmentation dataset, constructing a teacher network and extracting coarse segmentation features, coarse segmentation probabilities, and coarse segmentation prediction values;

[0064] As a preferred embodiment of step S2, the specific process includes the following steps:

[0065] S21, input the coarse segmentation dataset and the fine segmentation dataset into each candidate three-dimensional point cloud semantic segmentation network respectively for training and testing, for each candidate three-dimensional point cloud semantic segmentation network, on the coarse segmentation dataset, calculate the accuracy of its semantic segmentation (here, the commonly used mean intersection over union mIoU is used to measure the accuracy of segmentation), and record the calculation efficiency in the training and inference process, such as the time required for training, the time for inferring a single frame of point cloud, and the parameter amount of the model, so as to evaluate the consumption of calculation resources; similarly, on the fine segmentation dataset, the above accuracy and calculation efficiency are also evaluated. Next, the performance of each candidate network on the coarse and fine segmentation datasets is compared comprehensively, and those candidate three-dimensional point cloud semantic segmentation networks which have high calculation efficiency (such as less training and inference time, moderate parameter amount) under the premise of ensuring high segmentation accuracy are preferentially selected as the benchmark network;

[0066] S22, pre-training the benchmark network using the coarse segmentation dataset, and optimizing the network parameters until convergence;

[0067] S23, using the pre-trained benchmark network as a teacher network and fixing its weights, inputting the point cloud data in the fine segmentation dataset as input data into the teacher network, taking the features of the last layer and the class probability output at the end as the output of the teacher network, and defining them as coarse segmentation features and coarse segmentation probability respectively;

[0068] S24, performing Argmax operation on the coarse segmentation probability , that is, taking the index of the maximum probability value in the coarse segmentation probability to obtain a high-precision coarse segmentation prediction value, forming a reliable understanding ability of the scene coarse-grained semantics.

[0069] S3, constructing a student network to extract fine segmentation features, fine segmentation probability and fine segmentation prediction value; the constructed student network and the constructed teacher network together constitute a point cloud segmentation model based on the teacher-student network architecture;

[0070] As a preferred embodiment of step S3, the specific process includes the following steps:

[0071] S31, taking the benchmark network with initialized weights as the student network;

[0072] S32, inputting the point cloud data in the fine segmentation dataset as input data into the student network, taking the features of the last layer and the class probability output at the end as the output of the student network, and defining them as fine segmentation features and fine segmentation probability respectively;

[0073] S33, performing an Argmax operation on the fine segmentation probability to obtain a high-precision fine segmentation prediction value, ensuring fine-grained semantic recognition capability.

[0074] S4, designing a loss function based on the coarse segmentation features extracted by the teacher network and the fine-grained class labels in the fine segmentation dataset to train the student network.

[0075] As a preferred embodiment of step S4, the specific process includes the following steps:

[0076] S41, using the coarse segmentation features extracted by the teacher network As a soft label, the fine segmentation features extracted by the student network are supervised , and a knowledge distillation loss is constructed by introducing KL divergence to supervise and train the student network.

[0077] As a preferred embodiment of step S41, the specific process includes the following steps:

[0078] S411, converting the coarse segmentation features and the fine segmentation features into corresponding log probability distributions under temperature through temperature scaling and softmax normalization , :

[0079] ;

[0080] ;

[0081] In the above formula, denotes a softmax normalization operation, denotes applying a softmax normalization operation in the last dimension (i.e., the channel dimension); the coarse segmentation features and the fine segmentation features are both tensors with dimensions , where denotes the batch size, denotes the number of point clouds, denotes the number of channels of the coarse and fine segmentation features.

[0082] S412, calculating the KL divergence of the coarse segmentation features and the fine segmentation features under temperature based on the log probability distributions :

[0083] ;

[0084] In the above formula,​ Represents the cumulative sum of KL divergence, with parameters is the base of the natural logarithm;

[0085] S413, the calculated result enlarge The result is multiplied by the total number of elements to compensate for the gradient magnitude change caused by temperature scaling in step S411, yielding the final knowledge distillation loss. :

[0086] ;

[0087] In the above formula This indicates the operation for calculating knowledge distillation loss.

[0088] S42. Introduce cross-entropy loss to calculate fine-grained class labels in the finely segmented dataset. fine segmentation probability of student network output The error between them is used to further supervise the training of the student network; the process expression is:

[0089] ;

[0090] In the above formula, Represents cross-entropy loss, This indicates the cross-entropy loss calculation operation. This indicates the number of categories in the finely segmented dataset;

[0091] S43, Knowledge distillation loss in combined step S41 and the cross-entropy loss in step S42 Construct the total loss function Its expression is:

[0092] ;

[0093] In the above formula, The hyperparameter is initially set to 1 and gradually decreased to 0 during training to dynamically balance the influence of prior knowledge from the teacher network and observational data on the student network.

[0094] With total loss function In this embodiment, the AdamW optimizer is used to update the student network parameters, with the learning rate set to 0.01 and the weight decay coefficient set to 0.01. Iteratively update the student network parameters until the total loss function is reached. The training continues until convergence, at which point cloud segmentation model is finally obtained, realizing joint supervision of the student network by the teacher network and fine-grained category labels.

[0095] In this embodiment, by introducing the teacher-student network architecture, coarse-grained prior knowledge and fine-grained supervised information are organically combined, efficient and accurate indoor point cloud semantic segmentation is realized, based on the multi-level semantic segmentation results, the mobile robot can make accurate decisions in path planning and object interaction tasks, so as to realize stable and reliable intelligent application in complex indoor environment. The structure of the point cloud segmentation model based on the teacher-student network architecture constructed can be seen from Fig. 2 .

[0096] S5, using the mobile robot to scan and obtain three-dimensional point cloud data of the indoor scene, based on the trained point cloud segmentation model, generating point cloud semantic segmentation results of the indoor scene, including coarse-grained prediction value and fine-grained prediction value , to realize path planning and object interaction tasks of the mobile robot in the indoor scene.

[0097] As a preferred embodiment of step S5, the specific process includes the following steps:

[0098] S51, deploying a laser radar on the mobile robot, scanning the indoor scene in real time to obtain continuous three-dimensional point cloud data;

[0099] S52, inputting the three-dimensional point cloud data obtained in step S51 into the trained point cloud segmentation model, generating coarse-grained prediction value and fine-grained prediction value by the teacher network and the trained student network respectively;

[0100] S53, based on the multi-level semantic information provided by the coarse-grained prediction value and the fine-grained prediction value , the mobile robot realizes intelligent perception of the indoor scene. Specifically, when performing path planning task, since only obstacles and passable areas need to be clearly distinguished, the mobile robot only processes the coarse-grained prediction value at this time. This way does not need to process too fine semantic information, which can greatly reduce the amount of calculation, making the path planning process more efficient and rapid; when performing object interaction task, detailed information such as specific object category needs to be accurately identified, the mobile robot directly processes the fine-grained prediction value , so as to accurately interact with the object, and finally effectively perform path planning and object interaction tasks in the indoor scene.

[0101] The embodiment of the present application also provides an electronic device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, the computer program is configured to realize the steps of the indoor mobile robot scene perception method based on multi-granularity point cloud segmentation.

[0102] The embodiment of the present application further provides a storage medium, which is a computer readable storage medium, and a computer program is stored on the storage medium, and the computer program is executed by a processor to implement the steps of the indoor mobile robot scene perception method based on multi-granularity point cloud segmentation.

[0103] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction.

[0104] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a list of executable instructions for implementing logic functions, which can be specifically embodied in any computer readable medium for use by or in conjunction with an instruction execution system, device or apparatus, such as a computer-based system, a system including a processor or other system that can fetch and execute instructions from the instruction execution system, device or apparatus.

[0105] The above embodiments have been described in detail, and the principles and embodiments of the present application have been described by applying specific examples; the above embodiment descriptions are only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific embodiments and application scope will be changed; in view of the above, the content of the present specification should not be understood as a limitation of the present application.

Claims

1. A method for indoor mobile robot scene perception based on multi-granularity point cloud segmentation, characterized in that, The method comprises the following steps: S1, based on the collected indoor scene three-dimensional point cloud data, constructing coarse segmentation data set and fine segmentation data set; S11, for indoor obstacle avoidance task, the three-dimensional point cloud data containing coarse granularity category in the indoor scene three-dimensional point cloud data is taken as coarse segmentation data, and is divided into several categories; S12, for indoor interaction task, the three-dimensional point cloud data containing fine granularity category in the indoor scene three-dimensional point cloud data is taken as fine segmentation data, and is further subdivided on the basis of the categories divided by the coarse segmentation data; S13, according to the categories divided in steps S11-S12, the coarse segmentation data and the fine segmentation data are respectively given coarse granularity category label and fine granularity category label by point cloud processing software, so as to complete the construction of the coarse segmentation data set and the fine segmentation data set; S2, based on the coarse segmentation data set and the fine segmentation data set, selecting the reference network from multiple candidate three-dimensional point cloud semantic segmentation networks, constructing the teacher network and extracting the coarse segmentation feature, the coarse segmentation probability and the coarse segmentation prediction value; S3, constructing the student network, extracting the fine segmentation feature, the fine segmentation probability and the fine segmentation prediction value; the constructed student network and the constructed teacher network jointly constitute the point cloud segmentation model based on the teacher-student network architecture; S4, designing the loss function, training the student network based on the coarse segmentation feature extracted by the teacher network and the fine granularity category label in the fine segmentation data set, and the teacher network and the trained student network jointly constitute the trained point cloud segmentation model; S5, using the mobile robot to scan and obtain the three-dimensional point cloud data of the indoor scene, generating the point cloud semantic segmentation result of the indoor scene based on the trained point cloud segmentation model, including coarse granularity prediction value and fine granularity prediction value, so as to realize the path planning and object interaction task of the mobile robot in the indoor scene.

2. The indoor mobile robot scene perception method based on multi-granularity point cloud segmentation according to claim 1, characterized in that: The specific process of step S2 comprises the following steps: S21, the constructed coarse segmentation data set and fine segmentation data set are sent into multiple candidate three-dimensional point cloud semantic segmentation networks, the performance of each candidate three-dimensional point cloud semantic segmentation network is evaluated and compared comprehensively, and the candidate three-dimensional point cloud semantic segmentation network with excellent calculation efficiency under the premise of ensuring high accuracy is preferentially selected as the reference network; S22, the reference network is pre-trained using the coarse segmentation data set, and the network parameters are optimized until convergence; S23, the pre-trained reference network is taken as the teacher network and its weight is fixed, the point cloud data in the fine segmentation data set is received at the input end of the teacher network, the feature of the last layer and the category probability output at the end are taken as the output of the teacher network, and are defined as coarse segmentation feature and coarse segmentation probability respectively; S24, the Argmax operation is performed on the coarse segmentation probability to obtain the high-precision coarse segmentation prediction value, and the reliable understanding ability of the scene coarse granularity semantic is formed.

3. The indoor mobile robot scene perception method based on multi-granularity point cloud segmentation according to claim 1, characterized in that: The specific process of step S3 comprises the following steps: S31, the reference network with initialized weight is taken as the student network; S32, the point cloud data in the fine segmentation data set is input into the student network, the feature of the last layer and the category probability output at the end are taken as the output of the student network, and are defined as fine segmentation feature and fine segmentation probability respectively; S33, performing Argmax operation on the fine segmentation probability to obtain a high-precision fine segmentation prediction value, and ensuring fine-grained semantic recognition capability.

4. The indoor mobile robot scene perception method based on multi-granularity point cloud segmentation according to claim 1, characterized in that: The specific process of step S4 includes the following steps: S41, taking the coarse segmentation features extracted by the teacher network as soft labels to supervise the fine segmentation features extracted by the student network, introducing KL divergence to construct a knowledge distillation loss, and supervising the training of the student network; S42, introducing cross-entropy loss to calculate the error between the fine-grained class labels in the fine segmentation data set and the fine segmentation probability output by the student network, and further supervising the training of the student network; S43, combining the knowledge distillation loss of step S41 and the cross-entropy loss of step S42 to construct a total loss function, training the student network with the total loss function, and realizing the joint supervision of the teacher network and the fine-grained class labels on the student network.

5. The indoor mobile robot scene perception method based on multi-granularity point cloud segmentation according to claim 4, characterized in that: The specific process of step S41 includes the following steps: S411, convert coarse segmentation features and fine segmentation features into respective log probability distributions at respective temperatures and , :​​ ； ； In the above formula, denotes a softmax normalization operation, denotes applying a softmax normalization operation in the last dimension; S412, based on the log probability distribution , Compute coarse segmentation features and fine segmentation features KL divergence at temperature : ； In the above formula, denotes the KL divergence cumulative sum, denotes the batch size, denotes the number of point clouds, denotes the number of channels for coarse, fine segmentation features, parameter is the base of the natural logarithm; S413, the calculated result enlarge The result is multiplied by the total number of elements to compensate for the gradient magnitude change caused by temperature scaling in step S411, yielding the final knowledge distillation loss. : ； In the above formula represents a knowledge distillation loss calculation operation.

6. The indoor mobile robot scene perception method based on multi-granularity point cloud segmentation according to claim 1, characterized in that: The specific process of step S5 includes the following steps: S51, deploying a laser radar on the mobile robot to scan the indoor scene in real time and obtain continuous three-dimensional point cloud data; S52, inputting the three-dimensional point cloud data obtained in step S51 into the trained point cloud segmentation model, and generating coarse-grained prediction values and fine-grained prediction values through the teacher network and the trained student network respectively; S53, based on the coarse-grained prediction value with the fine-grained prediction value The mobile robot realizes intelligent perception of the indoor scene based on the multi-level semantic information provided, and in the execution of the path planning task, the mobile robot only processes the coarse-grained prediction value, and in the execution of the object interaction task, the mobile robot directly processes the fine-grained prediction value, and finally effectively executes the path planning and object interaction tasks in the indoor scene.

7. An electronic device, comprising: The device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the indoor mobile robot scene perception method based on multi-granularity point cloud segmentation according to any one of claims 1-6.

8. A storage medium, characterized by The storage medium is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by the processor to realize the steps of the indoor mobile robot scene perception method based on multi-granularity point cloud segmentation according to any one of claims 1-6.

Images