Tooth position segmentation method and device, equipment and storage medium

Abstract

The invention discloses a tooth position segmentation method and device, equipment and a storage medium. The tooth position segmentation method comprises the following steps: acquiring tooth point cloud data; performing segmentation processing on the tooth point cloud data based on a pre-trained tooth position segmentation model to obtain a tooth position segmentation result; the tooth position segmentation model comprises a double-branch feature extraction module, a plurality of fusion modules, an up-sampling fusion module and a classification module; inputting the tooth point cloud data into a double-branch feature extraction module for feature extraction, and outputting a plurality of stage features; each stage feature is input into a corresponding fusion module to obtain a plurality of stage fusion features, and each fusion module performs spatial dimension and channel dimension fusion processing on the semantic feature and the structural feature in one stage feature; and inputting the fusion features of each stage into an up-sampling fusion module for up-sampling processing and feature splicing processing to obtain target features, inputting the target features into a classification module for classification processing, and outputting a tooth position segmentation result. And the accuracy of tooth position segmentation is improved.

Description

Technical Field

[0001] The present invention relates to the field of oral technology, and in particular to a tooth segmentation method, device, equipment and storage medium. Background Art

[0002] With the rapid development of oral digital technology, tooth segmentation technology has become one of its core links. This technology provides dental technicians with detailed tooth morphology information by extracting high-precision 3D tooth models from 3D oral scan data. The acquisition of this information helps to optimize treatment planning, surgical guide design, and improve the execution process of orthodontic and implant surgery. The application of tooth segmentation technology not only greatly improves the accuracy of diagnosis, but also makes the treatment plan more personalized to meet the specific needs of patients.

[0003] Traditional tooth segmentation methods mainly rely on dentists to perform manual or semi-automatic marking, which is not only labor-intensive but also time-consuming. Professional dentists usually need to spend 15 to 30 minutes to mark a single jaw, and this efficiency is far from meeting the needs of modern oral medicine. Although deep learning has made some progress in assisting three-dimensional tooth segmentation, the tooth segmentation task still faces challenges. The similarity of the shape and appearance of adjacent teeth, as well as the unclear boundary between teeth and gums, especially missing teeth and crowding, pose challenges to the accuracy and robustness of tooth segmentation algorithms. These difficulties have led to unsatisfactory segmentation results of existing algorithms when dealing with complex clinical cases. Summary of the invention

[0004] The invention provides a tooth position segmentation method, device, equipment and storage medium to solve the problems of low efficiency and low accuracy of tooth position segmentation.

[0005] According to one aspect of the present invention, a tooth position segmentation method is provided, the method comprising:

[0006] Obtain tooth point cloud data;

[0007] Based on the pre-trained tooth segmentation model, the tooth point cloud data is segmented and processed to obtain the tooth segmentation result;

[0008] Among them, the tooth segmentation model includes a dual-branch feature extraction module, multiple fusion modules, an upsampling fusion module and a classification module;

[0009] Input the tooth point cloud data into the dual-branch feature extraction module for feature extraction, and output multiple stage features in the feature extraction process, where each stage feature includes semantic features and structural features;

[0010] The features of each stage are input into the corresponding fusion modules respectively to obtain multi-stage fusion features. Each fusion module performs fusion processing of the semantic features and structural features in the features of one stage in the spatial dimension and the channel dimension.

[0011] The fused features of each stage are input into the upsampling fusion module for upsampling and feature concatenation to obtain the target features;

[0012] The target features are input into the classification module for classification processing, and the tooth segmentation results are output.

[0013] According to another aspect of the present invention, a tooth position segmentation device is provided, the device comprising:

[0014] A tooth point cloud data acquisition module, used for acquiring tooth point cloud data;

[0015] A tooth segmentation result acquisition module is used to segment the tooth point cloud data based on a pre-trained tooth segmentation model to obtain a tooth segmentation result;

[0016] Among them, the tooth segmentation model includes a dual-branch feature extraction module, multiple fusion modules, an upsampling fusion module and a classification module;

[0017] Input the tooth point cloud data into the dual-branch feature extraction module for feature extraction, and output multiple stage features in the feature extraction process, where each stage feature includes semantic features and structural features;

[0018] The features of each stage are input into the corresponding fusion modules respectively to obtain multi-stage fusion features. Each fusion module performs fusion processing of the semantic features and structural features in the features of one stage in the spatial dimension and the channel dimension.

[0019] The fused features of each stage are input into the upsampling fusion module for upsampling and feature concatenation to obtain the target features;

[0020] The target features are input into the classification module for classification processing, and the tooth segmentation results are output.

[0021] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0022] at least one processor; and

[0023] a memory communicatively connected to at least one processor; wherein,

[0024] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by a processor so that the at least one processor can execute the tooth position segmentation method according to any embodiment of the present invention.

[0025] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the tooth segmentation method according to any embodiment of the present invention when executed.

[0026] The technical solution of the embodiment of the present invention is to obtain tooth point cloud data; segment the tooth point cloud data based on a pre-trained tooth segmentation model to obtain a tooth segmentation result; wherein the tooth segmentation model includes a dual-branch feature extraction module, multiple fusion modules, an upsampling fusion module and a classification module; input the tooth point cloud data into the dual-branch feature extraction module for feature extraction, and output multiple stage features in the feature extraction process, wherein each stage feature includes a semantic feature and a structural feature; input the features of each stage into the corresponding fusion module respectively to obtain multiple stage fusion features, and each fusion module performs spatial dimension and channel dimension fusion processing on the semantic features and structural features in a stage feature; input the fusion features of each stage into the upsampling fusion module for upsampling processing and feature splicing processing to obtain the target feature; input the target feature into the classification module for classification processing, and output the tooth segmentation result, thereby solving the problems of low efficiency and low accuracy of tooth segmentation and improving the efficiency and accuracy of tooth segmentation.

[0027] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 is a flow chart of a tooth segmentation method provided in Embodiment 1 of the present invention;

[0030] Figure 2 It is a flow chart of segmenting tooth point cloud data based on a tooth position segmentation model provided by an embodiment of the present invention;

[0031] Figure 3 This is a flow chart of a stage fusion feature acquisition provided by an embodiment of the present invention;

[0032] Figure 4 This is a flow chart of an upsampling fusion module acquiring target features provided by an embodiment of the present invention;

[0033] Figure 5 It is a structural schematic diagram of a tooth segmentation model provided by an embodiment of the present invention;

[0034] Figure 6 It is a structural schematic diagram of a tooth position segmentation device provided in Embodiment 2 of the present invention;

[0035] Figure 7 It is a structural schematic diagram of an electronic device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0038] Embodiment 1

[0039] Figure 1 This is a flow chart of a tooth segmentation method provided in the first embodiment of the present invention. This embodiment is applicable to the case of automatic tooth segmentation. The method can be executed by a tooth segmentation device. The tooth segmentation device can be implemented in the form of hardware and / or software. The tooth segmentation device can be configured in the electronic device provided in the embodiment of the present invention. The electronic device can be a server, a computer, a mobile terminal or a medical device, etc. For example, the mobile terminal can be a mobile phone, a tablet computer, etc. For example, the medical device can be a device equipped with a processor, such as an oral medical device, which can be equipped with a point cloud scanning device and / or an oral surgical instrument, etc. Figure 1 As shown, the method includes:

[0040] S110, obtaining tooth point cloud data.

[0041] In this embodiment, the tooth point cloud data may be data obtained by scanning the oral cavity through an intraoral scanning device, for example, it may be oral 3D model data, which can be used in oral medicine fields such as orthodontics, implantation, and restoration. Optionally, the oral scanning device may be an oral scanner. The tooth point cloud data includes point cloud data of a plurality of nodes, and the nodes in the tooth point cloud data may be nodes including tooth areas and nodes including background areas, wherein the background area may include but is not limited to the gum area and the gum area. In this embodiment, the tooth point cloud data is segmented to distinguish the point cloud data of the tooth area from the point cloud data of the background area.

[0042] The tooth point cloud data may include point data corresponding to multiple nodes, and the point data of each node may include attribute information and normal information of the node, wherein the attribute information may include one or more of color information and coordinate information, for example, the color information may be the RGB information of the node, the coordinate information may be the three-dimensional coordinates of the node, and the normal information may represent the orientation of the node. For example, the tooth point cloud data obtained by the oral scanner is M=(P, C, S), which consists of a set of points x i =(p i , c i ,s i ). Among them, P represents the three-dimensional coordinates of all nodes in the tooth point cloud data, that is, the position on the x, y, and z axes; C represents the color information of all nodes in the tooth point cloud data, that is, RGB information; S represents the normal features of all nodes in the tooth point cloud data, that is, the normal features on the x, y, and z axes; p i represents the three-dimensional coordinates of node i; c i Indicates the color information of node i; s i Represents the normal information of node i. The oral cavity is scanned by an intraoral scanning device to obtain tooth point cloud data including multiple nodes, which can provide a data basis for subsequent processing.

[0043] After acquiring the tooth point cloud data, the tooth point cloud data needs to be preprocessed, for example, one or more of alignment processing, centroid centering processing, scale normalization processing and resampling processing.

[0044] Optionally, after acquiring the tooth point cloud data, one or more of the following items are also included: aligning the point cloud data of multiple nodes in the tooth point cloud data with a predefined tooth coordinate system; performing centroid centering on the point cloud data of multiple nodes in the tooth point cloud data; performing scale normalization on the point cloud data of multiple nodes in the tooth point cloud data; and performing resampling on the point cloud data of multiple nodes in the tooth point cloud data.

[0045] Specifically, the point cloud data of multiple nodes in the tooth point cloud data are aligned with the predefined tooth coordinate system, which can be done by aligning the coordinate system of the point cloud data of multiple nodes in the oral scan data with the predefined tooth coordinate system through an iterative closest point algorithm, wherein the predefined tooth coordinate system may include standard oral data, and the standard oral data includes the point cloud data of multiple nodes. The specific process is as follows: obtaining the point cloud data of a node among multiple nodes in the tooth point cloud data; performing the closest point matching step, by finding the corresponding relationship between each node in the tooth point cloud data and the closest point in the predefined tooth coordinate system, establishing a matching pair between the node in the tooth point cloud data and the node in the predefined tooth coordinate system, and aligning the tooth point cloud data in the predefined tooth coordinate system through the above matching pair. Specifically, the gradient descent method is used to calculate the error between the matching node pairs, and the optimal transformation parameters are determined to minimize the error; according to the calculated optimal transformation parameters, the position and posture of the point cloud data of multiple nodes in the tooth point cloud data are adjusted to better align with the predefined tooth coordinate system. This process is repeated in the iterative process until the preset convergence condition is reached to ensure the accuracy and stability of the alignment result. Finally, the point cloud data of multiple nodes in the registered tooth point cloud data are output, achieving precise alignment with the predefined tooth coordinate system.

[0046] Specifically, the centroid centering process of the point cloud data of multiple nodes in the tooth point cloud data can be to move the centroid of the upper and lower jaws to the origin of the coordinate system. The specific process is as follows: Calculate the coordinate average value of all nodes in the tooth point cloud data, expressed as where p i represents the three-dimensional coordinates of point i, and n represents the number of nodes in the tooth point cloud data; the coordinates of each node are subtracted from the average coordinate value to achieve centroid centralization, that is, where p′ i is the coordinate vector centered around the centroid.

[0047] Specifically, the scale normalization processing is performed on the point cloud data of multiple nodes in the tooth point cloud data. The minimum and maximum values ​​of the Y axis are calculated for the point cloud data after centroiding, and the coordinate values ​​of each node are normalized so that they fall within the range of [-1, 1]. The formula is expressed as:

[0048]

[0049] Among them, p″ i is the normalized coordinate vector; min(·) represents the minimum function; max(·) represents the maximum function.

[0050] Specifically, the point cloud data of multiple nodes in the tooth point cloud data are resampled, and the farthest point sampling algorithm can be used to evenly distribute the point cloud data and keep the number of point clouds fixed at N, thereby optimizing the distribution and quality of the point cloud data of multiple nodes in the tooth point cloud data. The specific process is as follows: randomly select a point from the point cloud data of multiple nodes in the tooth point cloud data as the starting point set, denoted as P0; calculate the shortest distance from each remaining point in the point cloud data of multiple nodes in the tooth point cloud data to the starting point set P0, and the distance is determined by calculating the Euclidean distance from the point to all points in the point set and taking the minimum value; among all the unselected points, select a point that is farthest from the existing point set and add it to the point set. Repeat the distance calculation and farthest point selection steps until the size of the point set reaches the predetermined N points. Through resampling processing, the number of nodes obtained by resampling is reduced, the amount of calculation in the tooth position segmentation process is reduced, and the uniformity of node layout is obtained by optimizing resampling.

[0051] S120, performing segmentation processing on the tooth point cloud data based on the pre-trained tooth segmentation model to obtain a tooth segmentation result.

[0052] The tooth segmentation model includes a dual-branch feature extraction module, multiple fusion modules, an upsampling fusion module, and a classification module. Figure 2 , Figure 2 FIG. 1 is a flow chart of segmenting tooth point cloud data based on a tooth position segmentation model provided by an embodiment of the present invention. Figure 2 As shown, the tooth point cloud data is segmented based on the tooth position segmentation model, including:

[0053] S121. Input the tooth point cloud data into a dual-branch feature extraction module for feature extraction, and output multiple stage features in the feature extraction process, wherein each stage feature includes a semantic feature and a structural feature.

[0054] S122, input the features of each stage into the corresponding fusion modules respectively to obtain multiple stage fusion features, and each fusion module performs fusion processing on the semantic features and structural features in one stage feature in terms of spatial dimension and channel dimension.

[0055] S123, input the fused features of each stage into the upsampling fusion module for upsampling processing and feature splicing processing to obtain the target features.

[0056] S124, input the target features into the classification module for classification processing, and output the tooth segmentation result.

[0057] In this embodiment, the tooth segmentation model can be a neural network model, which can be used to segment the tooth point cloud data and output the tooth segmentation result. The tooth segmentation result includes the tooth area and the background area in the tooth point cloud data. The tooth area and the background area can be characterized by different marks. For example, the background area can be marked as 0. The tooth area can include multiple tooth positions. Different marks are set in the tooth area according to the tooth position. For example, the marks of different tooth positions in the tooth area can be digital marks. Exemplarily, when the upper and lower jaws are not distinguished, the corresponding tooth positions of the upper and lower jaws can be set with the same tooth position mark, and each tooth position mark can be 1-16; Exemplarily, when the upper and lower jaws are distinguished, different tooth position marks are set for each tooth position in the upper and lower jaws, and each tooth position mark can be 1-32.

[0058] Optionally, the tooth segmentation model may be a Point Transformer model. The model structure of the tooth segmentation model is not limited here and may be created according to user requirements.

[0059] Optionally, the tooth segmentation model includes a dual-branch feature extraction module, multiple fusion modules, an upsampling fusion module and a classification module. Among them, the dual-branch feature extraction module can be a module for extracting features from tooth point cloud data, and different branches are used to extract features from different types of input information in the tooth point cloud data to obtain different types of feature data, such as semantic features and structural features. During the feature extraction process, any branch in the dual-branch feature extraction module can perform feature extraction in stages to obtain feature data for each stage, that is, multiple stage features. During the feature extraction process, each branch in the dual-branch feature extraction module processes the input information at different depths at different stages, and corresponding stage features corresponding to different depths are obtained to improve the comprehensiveness of feature types and feature depths.

[0060] The fusion module is used to fuse different types of feature data, and the fusion module here can be one or more. The fusion module can fuse different types of feature data. Specifically, it can fuse different types of feature data in each stage feature to obtain stage fusion features, which can better understand the features of tooth point cloud data. The stage fusion features can be feature data obtained by fusing the semantic features and structural features of each stage through the fusion module, and the stage fusion features of each stage have different spatial scales.

[0061] The upsampling fusion module can be a module that performs upsampling, splicing, and other processing on the stage fusion features of multiple fusion modules to obtain target features, wherein the target features can be feature data of the same size as the tooth point cloud data input to the tooth position segmentation model obtained through the upsampling fusion module. Through the upsampling fusion module, different stage fusion features can be processed, which improves the comprehensiveness of the tooth point cloud data features and is beneficial to improving the accuracy of the tooth position segmentation model. The classification module can be a module that classifies the target features. For example, the classification module can include a one-dimensional convolution layer and a softmax layer, and the target features are sequentially input into the one-dimensional convolution layer and the softmax layer to obtain the tooth position segmentation result of the tooth point cloud data.

[0062] In order to respectively extract features of attribute information and normal information in tooth point cloud data, the dual-branch feature extraction module may include feature extraction modules with different network structures.

[0063] Optionally, the dual-branch feature extraction module includes: a semantic feature extraction module and a structural feature extraction module. The network structures of the semantic feature extraction module and the structural feature extraction module may be different.

[0064] Specifically, the semantic feature extraction module can be used to extract the features of the attribute information input by the tooth segmentation model, and the semantic feature extraction module can be a neural network model. Optionally, the semantic feature extraction module can include multiple semantic feature extraction submodules, each of which is connected in sequence, each of which can extract the features of the attribute information at different stages, and each of which has the same structure. Each semantic feature extraction submodule can include a multilayer perceptron, a Point Transformer Block module, and a Transition Down module. The output of each semantic feature extraction submodule is calculated as follows:

[0065] F i+1 =f TD (f PTB (F i ))

[0066] Among them, F i+1 represents the output of the semantic feature extraction submodule at stage i+1, F i represents the output of the semantic feature extraction submodule at stage i, f PTB represents the Point Transformer Block module, f TD Indicates the Transition Down module.

[0067] The Point Transformer Block module includes a Point Transformer layer, a linear layer before and after, and adopts a residual structure. The Point Transformer layer is responsible for performing self-attention operations and feature conversion of tooth point cloud data to capture the complex geometric relationships and features in the tooth point cloud data. Before the Point Transformer layer, a front linear layer is set to map the features of the input tooth point cloud data to a new feature space to provide preprocessing for the self-attention operation. After the Point Transformer layer, a back linear layer is set to further map and transform the output of the layer to meet the needs of subsequent network layers or tasks.

[0068] The processing flow of the Point Transformer layer is as follows: By adding the position encoding δ = θ (p i -p j ) to obtain spatial information, where p i , p j are the three-dimensional coordinates corresponding to the two nodes, θ(·) is the multi-layer perceptron layer; using linear transformation ψ(·) calculates the query vector q and key vector k of point i respectively. Then, the features are aggregated by the γ(·) mapping function to generate the attention vector. The attention weight is obtained by the SoftMax function and multiplied by the numerical vector v to obtain the output feature y. i The calculation formula is as follows:

[0069]

[0070] Among them, ρ(·) represents the SoftMax function, γ(·) represents the multi-layer perceptron, ψ(·), α(·) denote linear layers, and ⊙ denotes dot products.

[0071] The Transition Down module aims to aggregate features and reduce the resolution of tooth point cloud data through a downsampling process. The processing flow is as follows: the farthest point sampling is used for the input tooth point cloud data, and the sampling rate is 4 to obtain a sub-point set of the input point set; for each point in the sub-point set, the K nearest neighbor algorithm is used to obtain k points in its neighborhood, where the value of k is set to 16; for the k points in the neighborhood of each point, a multi-layer perceptron is used to extract features, which includes a linear layer and a batch normalization layer, as well as a ReLU activation function; the output features are subjected to local maximum pooling, that is, the feature vectors of the k points in the neighborhood are replaced by the maximum value of each dimension in the output feature vector, thereby completing the process of downsampling and feature aggregation.

[0072] The structural feature extraction module extracts geometric structure information and local edge area information of the tooth point cloud data by capturing the change characteristics between normal lines. The structural feature extraction module can be a neural network model. Optionally, the structural feature extraction module can include multiple structural feature extraction submodules, each of which is connected in sequence, each of which can extract features of normal information at different stages, and each of which has the same structure. Optionally, the structural feature extraction submodule can include a multi-layer perceptron, an EdgeConv module, and a TransitionDown module. The structural feature extraction submodule is based on the normal information S of the tooth point cloud data. in ∈(s i ) as input data, and output multi-scale geometric features through the hierarchical stacking EdgeConv module. The output of the structural feature extraction submodule is calculated as follows:

[0073] S i+1 =f edge (S i )

[0074] Among them, S i+1 represents the output of the structural feature extraction submodule at stage i+1; S i represents the output of the structural feature extraction submodule at stage i; f edge Represents the EdgeConv module.

[0075] In order to extract semantic features and structural features at different depths, the attribute information in the tooth point cloud data may be subjected to multiple stages of semantic feature extraction, and the normal information in the tooth point cloud data may be subjected to multiple stages of structural feature extraction.

[0076] Optionally, the tooth point cloud data is input into a dual-branch feature extraction module for feature extraction, and features of multiple stages in the feature extraction process are output, including: inputting attribute information in the tooth point cloud data into a semantic feature extraction module for semantic feature extraction, and outputting semantic features of multiple stages in the semantic feature extraction process; inputting normal information in the tooth point cloud data into a structural feature extraction module for structural feature extraction, and outputting structural features of multiple stages in the structural feature extraction process.

[0077] In order to obtain semantic features of multiple stages, the semantic feature extraction module may include multiple semantic feature extraction submodules. Optionally, the semantic feature extraction module includes multiple semantic feature extraction submodules connected in sequence, and each semantic feature extraction submodule outputs the semantic features of one stage. Specifically, since the semantic feature extraction module may include multiple semantic feature extraction submodules, the attribute information in the tooth point cloud data is input into the first semantic feature extraction submodule for semantic feature extraction to obtain the semantic features of this stage, and the extracted semantic features of this stage are input into the semantic feature extraction submodule of the next stage to obtain the semantic features of the next stage, and so on, to obtain the semantic features of multiple stages.

[0078] Exemplarily, the semantic feature extraction module includes a first semantic feature extraction submodule, a second semantic feature extraction submodule and a third semantic feature extraction submodule, wherein the first semantic feature extraction submodule, the second semantic feature extraction submodule and the third semantic feature extraction submodule are connected in sequence, the attribute information in the tooth point cloud data is input into the first semantic feature extraction submodule for semantic feature extraction to obtain a first semantic feature, the first semantic feature is input into the second semantic feature extraction submodule for semantic feature extraction to obtain a second semantic feature, the second semantic feature is input into the third semantic feature extraction submodule for semantic feature extraction to obtain a third semantic feature. The number of semantic feature extraction submodules can be three or five, and can be set according to needs, and is not limited here.

[0079] In order to obtain structural features of multiple stages, the structural feature extraction module may include multiple structural feature extraction submodules. Optionally, the structural feature extraction module includes multiple structural feature extraction submodules connected in sequence, and each structural feature extraction submodule outputs the structural features of one stage. Specifically, since the structural feature extraction module may include multiple structural feature extraction submodules, the normal information in the tooth point cloud data is input into the first structural feature extraction submodule for structural feature extraction to obtain the structural features of this stage, and the structural features of this stage are input into the structural feature extraction submodule of the next stage to obtain the structural features of the next stage, and so on, to obtain the structural features of multiple stages. Optionally, the multiple stages in the semantic feature extraction process correspond one-to-one to the multiple stages in the structural feature extraction process.

[0080] Exemplarily, the structural feature extraction module includes a first structural feature extraction submodule, a second structural feature extraction submodule and a third structural feature extraction submodule, wherein the first structural feature extraction submodule, the second structural feature extraction submodule and the third structural feature extraction submodule are connected in sequence, the normal information in the tooth point cloud data is input into the first structural feature extraction submodule for structural feature extraction to obtain the first structural feature, the first structural feature is input into the second structural feature extraction submodule for structural feature extraction to obtain the second structural feature, and the second structural feature is input into the third structural feature extraction submodule for structural feature extraction to obtain the third structural feature. The number of structural feature extraction submodules can be three or five, and can be set according to requirements, and there is no limitation here. In order to ensure that the semantic features of a certain stage are fused with the structural features of that stage, the number of structural feature extraction submodules and semantic feature extraction submodules is the same.

[0081] In order to make full use of the respective advantages of the semantic feature extraction module and the structural feature extraction module, the spatial attention fusion module and the channel attention fusion module can be used to fuse the semantic features and structural features in the spatial dimension and channel dimension respectively to increase the comprehensiveness of the feature data.

[0082] Optionally, the tooth segmentation model can have multiple fusion modules, each fusion module has the same structure, the input end of each stage fusion module is respectively connected to the output end of the semantic feature extraction submodule and the output end of the structural feature extraction submodule corresponding to the stage, and the output end of each stage fusion module is respectively connected to the input end of the upsampling fusion module corresponding to the stage. Each semantic feature extraction submodule and structural feature extraction submodule of each stage corresponds to one fusion module.

[0083] The fusion module includes: a spatial attention fusion module and a channel attention fusion module. Specifically, the fusion module can fuse the semantic features extracted by the semantic feature extraction module and the structural features extracted by the structural feature extraction module.

[0084] The spatial attention fusion module is used to perform spatial dimension fusion processing on semantic features and structural features, and the spatial attention fusion module can be a self-attention mechanism module. Optionally, the spatial attention fusion module can include a multi-layer perceptron and a self-attention mechanism. The channel attention fusion module is used to perform channel dimension fusion processing on semantic features and structural features. In some embodiments, the spatial attention fusion module and the channel attention fusion module can be set in parallel, and the output results of the spatial attention fusion module and the channel attention fusion module are spliced ​​as stage fusion features; in some embodiments, the spatial attention fusion module and the channel attention fusion module can be connected in series, and the connection order of the spatial attention fusion module and the channel attention fusion module may not be limited here, and the input semantic features and structural features are fused in turn to output stage fusion features.

[0085] Optionally, the features of each stage are respectively input into the corresponding fusion modules to obtain multiple stage fusion features, including: through the spatial attention fusion module, for any stage feature, determining the initial splicing features of the semantic features and structural features in the stage feature; determining the spatial attention weight based on the initial splicing feature, and performing feature fusion in the spatial dimension based on the spatial attention weight and the initial splicing feature to obtain the spatial fusion feature; through the channel attention fusion module, extracting the context representation features of the spatial fusion feature in the channel dimension; determining the channel attention weight based on the context representation feature; and fusing the spatial fusion features based on the channel attention weight to obtain the stage fusion feature.

[0086] Specifically, the initial splicing feature can be a feature obtained by splicing semantic features and structural features in the channel dimension. Optionally, the initial splicing feature includes splicing features of multiple nodes. The spatial attention weight can characterize the importance of a node to its neighborhood nodes. Through the spatial attention fusion module, the semantic features and structural features of each stage are spliced ​​in the channel dimension to obtain the initial splicing feature of the stage; based on the initial splicing feature, the spatial attention weight is calculated according to the correlation between different tooth point cloud data; and the initial splicing feature after spatial attention weighting is calculated.

[0087] For example, the features of the semantic feature extraction module and the structural feature extraction module at stage i are F i ∈(B,N i , C i ) and S i ∈(B,N i , C i ), where N i represents the number of point clouds at stage i, C iRepresents the dimension of the feature at stage i. The processing flow of the spatial attention fusion module is as follows: the semantic features and structural features are spliced ​​and fused in the channel dimension to obtain the initial spliced ​​features, and the multi-layer perceptron is used to reduce the dimension of the initial spliced ​​features to remove redundant information and obtain the output feature A i ∈(B,N i , C i ), the calculation formula is as follows:

[0088]

[0089] Among them, f mlp represents the output of the multi-layer perceptron, Represents the channel dimension feature concatenation operation.

[0090] Optionally, a spatial attention weight is determined based on the initial splicing feature, and feature fusion in the spatial dimension is performed based on the spatial attention weight and the initial splicing feature to obtain a spatial fusion feature, including: determining the spatial attention weight of each neighboring node based on the splicing feature of each node and the splicing features of multiple neighboring nodes of the node; determining the spatial enhancement feature of each node based on the spatial attention weights of multiple neighboring nodes of each node and the splicing features of multiple neighboring nodes; determining the spatial fusion feature based on the splicing features of multiple nodes in the initial splicing feature and the spatial enhancement features of the multiple nodes.

[0091] Specifically, the multiple neighboring nodes of a node may be determined by, for example, a K nearest neighbor algorithm. i , find its K nearest neighbor points through the K nearest neighbor algorithm Each node here may be each node in all nodes in the initial splicing feature, or each node in the local nodes obtained by sampling multiple nodes in the initial splicing feature. The sampling process of multiple nodes in the initial splicing feature may be based on the farthest point sampling method to sample a preset number of nodes.

[0092] Optionally, the neighborhood nodes of each node can be determined based on the distance between each node, and the distance here can be determined based on the coordinate information between each node. Optionally, a node directed graph is constructed. The directed graph can be a graph composed of multiple nodes in the tooth point cloud data and the edges between the nodes. The edges in the directed graph represent the similarity between adjacent nodes. Accordingly, the distance between nodes can be represented based on the edge information of the nodes in the node directed graph, and the neighborhood nodes of each node can be further determined. The node directed graph can be expressed as G = (V, E), where V is a set of points, and E is a set of directed edges, which represents the connection relationship between points, and is mathematically described as {(i, j i1 ),....,(i,j iK)}. A directed graph can be a graph composed of multiple nodes in the tooth point cloud data and the edges between the nodes. The edge between any two nodes is set with a weight, which can be used to represent the relationship between the nodes. The larger the weight of the edge between the two nodes, the higher the similarity between the two nodes and the stronger the relationship between the two nodes; the smaller the weight of the edge between the two nodes, the lower the similarity between the two nodes and the weaker the relationship between the two nodes. Define the edge feature function e ij =h Θ (x n , x q -x n ), used to represent the weight between nodes. Θ (·) represents a nonlinear function composed of a learnable parameter Θ. Such an edge feature function combines global shape information and local domain information at the same time, that is, combines the overall shape information of the tooth and the local geometric information. From then on, the edge feature function can be determined based on a preset multi-layer perceptron, for example, the concatenation features of any two nodes are input into the above multi-layer perceptron, and the edge feature function of any two nodes is output.

[0093] The semantic features and structural features of each node are spliced, for example, the splicing can be performed in the channel dimension, to obtain the splicing features of each node; the semantic features and structural features of each neighboring node of the node are spliced, for example, the splicing can be performed in the channel dimension, to obtain the splicing features of each neighboring node.

[0094] The spatial attention weight corresponding to each neighborhood node is determined by the splicing features of multiple neighborhood nodes corresponding to each node. For example, the spatial attention weight corresponding to any neighborhood node can be determined based on the ratio between the product of the splicing feature of the node and the splicing feature of any neighborhood node and the sum of the products of the splicing feature of the node and the splicing features of multiple neighborhood nodes.

[0095] For multiple neighboring nodes of each node, the splicing features of each neighboring node are spatially enhanced by the spatial attention weight corresponding to each neighboring node to obtain the enhanced features of each neighboring node. The sum of the enhanced features of multiple neighboring nodes of each node can be determined as the spatial enhanced features of each node. For example, the splicing features of multiple neighboring nodes of the node and their corresponding spatial attention weights can be multiplied to obtain the spatial enhanced features of the node. The splicing features of multiple nodes and their corresponding spatial enhanced features are summed or weighted summed to obtain spatial fusion features.

[0096] For example, in order to make the tooth segmentation model focus on the tooth morphological features and geometric features, the attention weight matrix is ​​calculated according to the correlation between different tooth point cloud data, and the splicing features of multiple neighborhood points are weighted fused. i Perform farthest point sampling to obtain A i For each point in the sub-point set, the K nearest neighbor algorithm is used to obtain k points in its neighborhood and output B i ∈(B,K,N i , C i ). For each point b i , by querying its K-nearest neighbor points {b i1 , b i2 , ..., b ik} to construct local neighborhoods. These neighborhood points together with the center point form the local graph structure. The self-attention mechanism is used to calculate the spatial attention weight of each node to its neighborhood points. This is achieved by calculating the similarity of the features of the point and its neighborhood points. Let f i For point p i The splicing features, f ij is the neighborhood point b ij The concatenated features of the spatial attention weight a ij It can be calculated by the following formula:

[0097]

[0098] Use the calculated attention weight a ij The features of weighted neighborhood points are used to obtain the spatial fusion features f′ of each neighborhood point. ij :

[0099] f′ ij =a ij ·f ij

[0100] Aggregate the spatial fusion features of all neighborhood points to obtain the enhanced center point p i Spatial enhancement features:

[0101]

[0102] The enhanced spatial enhancement feature f′ i Combined with the initial concatenation feature f, it can be done by simple concatenation or weighted sum:

[0103] f″ i =f i +f′ i

[0104] Among them, f″ i Represents spatial fusion features.

[0105] Optionally, the contextual representation features of the spatial fusion features in the channel dimension include an average representation feature and a maximum representation feature in the channel dimension; wherein the average representation feature is obtained by performing average pooling processing on the spatial fusion features, and the maximum representation feature is obtained by performing maximum pooling processing on the spatial fusion features.

[0106] Specifically, the average representation feature can characterize the average level of the input tooth point cloud data in the channel dimension, and can be obtained by aggregating the spatial information of the spatial fusion feature through the average pooling operation. The maximum representation feature can characterize the maximum level of the input tooth point cloud data in the channel dimension, and can be obtained by aggregating the spatial information of the spatial fusion feature through the maximum pooling operation. Optionally, the channel attention weight is determined based on the contextual representation feature, and the spatial fusion feature is fused based on the channel attention weight, including: determining the channel attention weight based on the average representation feature and the maximum representation feature; determining the channel enhancement feature based on the channel attention weight and the spatial fusion feature; and obtaining the stage fusion feature based on the channel enhancement feature and the spatial fusion feature.

[0107] Specifically, the channel attention weight can characterize the importance between node channels. The contextual representation feature can characterize the contextual information of the spatial fusion feature in the tooth point cloud data, and can include global spatial fusion features and local spatial fusion features. Through the channel attention fusion module, the contextual representation features of the spatial fusion feature in the channel dimension are extracted, and the contextual representation features of the spatial fusion feature in the channel dimension are captured. For example, the dependency between channels can be captured by a multi-layer perceptron with shared parameters, and the channel attention weight is obtained through the sigmoid function. The spatial fusion feature under the channel attention weight is calculated to obtain the stage fusion feature. By capturing the dependency between channels, the average representation feature and the maximum representation feature are input into the neural network model for processing to obtain the channel attention weight, and the product between the channel attention weight and the spatial fusion feature is calculated to obtain the channel enhancement feature. The channel enhancement feature and the spatial fusion feature are calculated for splicing operation, for example, it can be residual splicing to obtain the stage fusion feature. For example, the average representation feature and the maximum representation feature are input into a shared multi-layer perceptron to further capture the dependency between channels, obtain two output feature vectors, perform vector addition and sum operation on the two output feature vectors, and obtain the channel attention weight through the sigmoid function. The calculation formula is as follows:

[0108] a cas =σ(f MLP (fmax)+f MLP (favg))

[0109] Among them, a casrepresents the channel attention weight, σ represents the sigmoid function, f MLP is the output of the multi-layer perceptron with shared parameters, fmax represents the maximum representation feature, and favg represents the average representation feature. The spatial fusion feature f″ i and channel attention weight a cas Perform element-level multiplication operations to obtain enhanced features after channel attention screening. Use the idea of ​​residual connection to add the spatial fusion features and enhanced features to obtain the final output result f out :

[0110]

[0111] in, represents the residual connection operation, Represents an element-wise multiplication operation. For example, see Figure 3 , Figure 3 It is a flow chart of acquiring stage fusion features provided by an embodiment of the present invention.

[0112] Optionally, the method also includes: upsampling the input features through an upsampling fusion module to obtain upsampling features; the input features are the fusion features of the last stage or the output features of the last upsampling fusion module; the upsampling features are concatenated with the fusion features of the last stage corresponding to the input features to obtain the output features of the upsampling fusion module; the output features of the last upsampling fusion module are the target features.

[0113] Specifically, the tooth position segmentation model may include multiple upsampling fusion modules, each of which has the same structure, and the input end of each upsampling fusion module is respectively connected to the output end of the fusion module of the corresponding stage and the output end of the fusion module of the previous stage, and the output end of each upsampling fusion module is connected to the input end of the upsampling fusion module of the previous stage. The input feature can be the output feature of the previous upsampling fusion module, or the fusion feature of the last stage. The output feature can be the input feature processed by the upsampling fusion module. The upsampling feature can be the feature obtained by upsampling the input feature. The upsampling fusion module can perform upsampling and fusion operations on the input feature to obtain the target feature required for the tooth position segmentation model. The target feature can be data of the same size as the tooth point cloud data of the input tooth position segmentation model after the upsampling fusion module. The upsampling process can use a neural network model. Optionally, the upsampling process can use a multi-layer perceptron.

[0114] Exemplarily, the tooth segmentation model includes N upsampling fusion modules. Taking the Nth upsampling fusion module as an example, the input feature is the fusion feature of the last stage, the input feature is upsampled to obtain the upsampled feature of the Nth upsampling fusion module, the upsampled feature of the Nth upsampling fusion module is spliced ​​with the N-1th stage fusion feature, for example, the splicing process can be channel dimension, to obtain the output feature of the Nth upsampling fusion module; the output feature of the Nth upsampling fusion module is upsampled to obtain the upsampled feature of the N-1th upsampling fusion module, the upsampled feature of the N-1th upsampling fusion module is spliced ​​with the N-2th stage fusion feature. Splicing processing, for example, can be splicing processing in the channel dimension to obtain the output features of the N-1th upsampling fusion module; upsampling operation is performed on the output features of the N-1th upsampling fusion module to obtain the upsampling features of the N-2th upsampling fusion module, and the upsampling features of the N-2th upsampling fusion module are spliced ​​with the N-3th stage fusion features, for example, splicing processing can be performed in the channel dimension to obtain the output features of the N-2th upsampling fusion module, and so on, to obtain the output features of the first upsampling fusion module, and the output features of the first upsampling fusion module are used as the target features, and processed by the classification module to obtain the tooth segmentation result, thereby realizing the segmentation of the tooth. For example, see Figure 4 and Figure 5 , Figure 4 is a flow chart of an upsampling fusion module obtaining target features provided by an embodiment of the present invention, Figure 5 Schematic diagram of the structure of a tooth segmentation model provided by an embodiment of the present invention. Figure 4 As shown, f out_i represents the stage fusion feature of stage i, represents the stage fusion feature of stage i-1, It represents the output of the upsampling fusion module at stage i-1. The stage fusion features of stage i are sequentially processed by the multi-layer perceptron and trilinear interpolation, and then added to the stage fusion features of stage i-1 to obtain the output of the upsampling fusion module at stage i-1.

[0115] The target feature may include target sub-features of multiple nodes, and the target sub-features of multiple nodes are classified by the classification module to obtain classification results of multiple nodes, and the classification results of multiple nodes form tooth segmentation results. The classification result of any node may include various tooth types and background types.

[0116] The technical solution of this embodiment is to obtain tooth point cloud data; segment the tooth point cloud data based on a pre-trained tooth segmentation model to obtain a tooth segmentation result; wherein the tooth segmentation model includes a dual-branch feature extraction module, multiple fusion modules, an upsampling fusion module and a classification module; input the tooth point cloud data into the dual-branch feature extraction module for feature extraction, and output multiple stage features in the feature extraction process, wherein each stage feature includes semantic features and structural features; input each stage feature into the corresponding fusion module respectively to obtain multiple stage fusion features, and each fusion module performs spatial dimension and channel dimension fusion processing on the semantic features and structural features in a stage feature; input each stage fusion feature into the upsampling fusion module for upsampling processing and feature splicing processing to obtain the target feature; input the target feature into the classification module for classification processing, and output the tooth segmentation result, thereby improving the efficiency and accuracy of tooth segmentation.

[0117] Embodiment 2

[0118] Figure 6 Schematic diagram of the structure of a tooth segmentation device provided in the second embodiment of the present invention. Figure 6 As shown, the device comprises:

[0119] A tooth point cloud data acquisition module 210 is used to acquire tooth point cloud data;

[0120] The tooth segmentation result acquisition module 220 is used to perform segmentation processing on the tooth point cloud data based on the pre-trained tooth segmentation model to obtain the tooth segmentation result.

[0121] On the basis of the above embodiment, optionally, the tooth point cloud data acquisition module 210 also includes a data preprocessing module 211, which is used to: perform data preprocessing on the tooth point cloud data, such as one or more of alignment processing, centroid centering processing, scale normalization processing and resampling processing.

[0122] Optionally, the tooth segmentation result acquisition module 220 further includes a dual-branch feature extraction module 221, multiple fusion modules 222, an upsampling fusion module 223 and a classification module 224;

[0123] The dual-branch feature extraction module 221 is used to input the tooth point cloud data into the dual-branch feature extraction module 221 for feature extraction, and output multiple stage features in the feature extraction process, wherein each stage feature includes a semantic feature and a structural feature;

[0124] The fusion module 222 is used to input the features of each stage into the corresponding fusion module 222 to obtain multiple stage fusion features. Each fusion module performs fusion processing of the semantic features and structural features in a stage feature in the spatial dimension and the channel dimension.

[0125] The upsampling fusion module 223 is used to input the fusion features of each stage into the upsampling fusion module 223 for upsampling processing and feature splicing processing to obtain the target features;

[0126] The classification module 224 is used to input the target features into the classification module 224 for classification processing and output the tooth position segmentation result.

[0127] Optionally, the tooth segmentation result acquisition module 220 is also used to: input the attribute information in the tooth point cloud data into the semantic feature extraction module for semantic feature extraction, and output the semantic features of multiple stages in the semantic feature extraction process; input the normal information in the tooth point cloud data into the structural feature extraction module for structural feature extraction, and output the structural features of multiple stages in the structural feature extraction process.

[0128] Optionally, the tooth segmentation result acquisition module 220 is also used to: determine the initial splicing features of the semantic features and structural features in the stage features for any stage features through the spatial attention fusion module; determine the spatial attention weight based on the initial splicing feature, and perform feature fusion in the spatial dimension based on the spatial attention weight and the initial splicing feature to obtain the spatial fusion feature; extract the context representation features of the spatial fusion features in the channel dimension through the channel attention fusion module; determine the channel attention weight based on the context representation features; and fuse the spatial fusion features based on the channel attention weight to obtain the stage fusion feature.

[0129] Optionally, the tooth segmentation result acquisition module 220 is also used to: determine the spatial attention weight of each neighboring node based on the splicing features of each node and the splicing features of multiple neighboring nodes of the node; determine the spatial enhancement feature of each node based on the spatial attention weights of multiple neighboring nodes of each node and the splicing features of multiple neighboring nodes; determine the spatial fusion feature based on the splicing features of multiple nodes in the initial splicing features and the spatial enhancement features of multiple nodes.

[0130] Optionally, the tooth segmentation result acquisition module 220 is also used to: determine the channel attention weight based on the average representation feature and the maximum representation feature; determine the channel enhancement feature based on the channel attention weight and the spatial fusion feature; and obtain the stage fusion feature based on the channel enhancement feature and the spatial fusion feature.

[0131] Optionally, the device also includes a target feature acquisition module, which is used to: upsample the input features through the upsampling fusion module to obtain upsampled features; the input features are the fusion features of the last stage or the output features of the last upsampling fusion module; the upsampled features are concatenated with the fusion features of the last stage corresponding to the input features to obtain the output features of the upsampling fusion module; the output features of the last upsampling fusion module are the target features.

[0132] The tooth segmentation device provided in the embodiment of the present invention can execute the tooth segmentation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0133] Embodiment 3

[0134] Figure 7 1 is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0135] like Figure 7 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0136] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0137] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the tooth segmentation method.

[0138] In some embodiments, the tooth segmentation method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the tooth segmentation method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the tooth segmentation method in any other appropriate manner (for example, by means of firmware).

[0139] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0140] The computer program for implementing the tooth segmentation method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0141] Embodiment 4

[0142] Embodiment 4 of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute a tooth position segmentation method, the method comprising:

[0143] Acquire tooth point cloud data; segment and process the tooth point cloud data based on a pre-trained tooth segmentation model to obtain a tooth segmentation result; wherein the tooth segmentation model includes a dual-branch feature extraction module, multiple fusion modules, an upsampling fusion module and a classification module; input the tooth point cloud data into the dual-branch feature extraction module for feature extraction, and output multiple stage features in the feature extraction process, wherein each stage feature includes semantic features and structural features; input each stage feature into the corresponding fusion module respectively to obtain multiple stage fusion features, and each fusion module performs spatial dimension and channel dimension fusion processing on the semantic features and structural features in a stage feature; input each stage fusion feature into the upsampling fusion module for upsampling processing and feature splicing processing to obtain the target feature; input the target feature into the classification module for classification processing, and output the tooth segmentation result.

[0144] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0145] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0146] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0147] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0148] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0149] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A tooth segmentation method, characterized in that: include: Obtain tooth point cloud data; Segmenting the tooth point cloud data based on a pre-trained tooth segmentation model to obtain a tooth segmentation result; The tooth segmentation model includes a dual-branch feature extraction module, multiple fusion modules, an upsampling fusion module and a classification module; Inputting the tooth point cloud data into the dual-branch feature extraction module for feature extraction, and outputting multiple stage features in the feature extraction process, wherein each stage feature includes a semantic feature and a structural feature; Input each of the stage features into the corresponding fusion module to obtain multiple stage fusion features, and each fusion module performs fusion processing of the semantic features and the structural features in one stage feature in the spatial dimension and the channel dimension; Inputting the fused features of each stage into the upsampling fusion module for upsampling and feature concatenation to obtain target features; The target features are input into the classification module for classification processing, and the tooth position segmentation result is output.

2. The method according to claim 1, characterized in that The tooth point cloud data includes attribute information and normal information corresponding to a plurality of nodes respectively; The dual-branch feature extraction module includes: a semantic feature extraction module and a structural feature extraction module; The step of inputting the tooth point cloud data into the dual-branch feature extraction module for feature extraction and outputting multiple stage features in the feature extraction process includes: Inputting the attribute information in the tooth point cloud data into the semantic feature extraction module to extract semantic features, and outputting semantic features at multiple stages in the semantic feature extraction process; The normal information in the tooth point cloud data is input into the structural feature extraction module to extract structural features, and the structural features of multiple stages in the structural feature extraction process are output.

3. The method according to claim 2, characterized in that The semantic feature extraction module includes a plurality of semantic feature extraction submodules connected in sequence, each of which outputs a semantic feature of a stage; The structural feature extraction module includes a plurality of structural feature extraction submodules connected in sequence, and each of the structural feature extraction submodules outputs a stage of structural features.

4. The method according to claim 1, characterized in that: The fusion module includes: a spatial attention fusion module and a channel attention fusion module; The step of inputting each of the stage features into the corresponding fusion module to obtain multiple stage fusion features includes: By means of the spatial attention fusion module, for any stage feature, an initial splicing feature of the semantic feature and the structural feature in the stage feature is determined; a spatial attention weight is determined based on the initial splicing feature, and feature fusion of the spatial dimension is performed based on the spatial attention weight and the initial splicing feature to obtain a spatial fusion feature; Through the channel attention fusion module, the context representation features of the spatial fusion features in the channel dimension are extracted; the channel attention weights are determined based on the context representation features; and the spatial fusion features are fused based on the channel attention weights to obtain the stage fusion features.

5. The method according to claim 4, characterized in that The initial splicing features include splicing features of multiple nodes; The determining of the spatial attention weight based on the initial splicing feature, and performing feature fusion of the spatial dimension based on the spatial attention weight and the initial splicing feature to obtain the spatial fusion feature, includes: Determine the spatial attention weight of each of the neighboring nodes based on the splicing features of each of the nodes and the splicing features of a plurality of neighboring nodes of the node; Determine a spatial enhancement feature of each of the nodes based on the spatial attention weights of the plurality of neighboring nodes of each of the nodes and the concatenated features of the plurality of neighboring nodes; The spatial fusion feature is determined based on the splicing features of the plurality of nodes in the initial splicing feature and the spatial enhancement features of the plurality of nodes.

6. The method according to claim 4, characterized in that The context representation features of the spatial fusion features in the channel dimension include the average representation features and the maximum representation features in the channel dimension; The determining of the channel attention weight based on the context representation feature, and fusing the spatial fusion feature based on the channel attention weight, comprises: Determine the channel attention weight based on the average representation feature and the maximum representation feature; Determine a channel enhancement feature based on the channel attention weight and the spatial fusion feature; The stage fusion feature is obtained based on the channel enhancement feature and the spatial fusion feature.

7. The method according to claim 1, characterized in that The method further comprises: The upsampling fusion module is used to upsample the input features to obtain upsampled features; the input features are the fusion features of the last stage or the output features of the previous upsampling fusion module; The up-sampled features are concatenated with the fusion features of the previous stage corresponding to the input features to obtain the output features of the up-sampled fusion module; The output feature of the last upsampling fusion module is the target feature.

8. A tooth segmentation device, characterized in that: include: A tooth point cloud data acquisition module, used for acquiring tooth point cloud data; A tooth segmentation result acquisition module is used to perform segmentation processing on the tooth point cloud data based on a pre-trained tooth segmentation model to obtain a tooth segmentation result; The tooth segmentation model includes a dual-branch feature extraction module, multiple fusion modules, an upsampling fusion module and a classification module; Inputting the tooth point cloud data into the dual-branch feature extraction module for feature extraction, and outputting multiple stage features in the feature extraction process, wherein each stage feature includes a semantic feature and a structural feature; Input each of the stage features into the corresponding fusion module to obtain multiple stage fusion features, and each fusion module performs fusion processing of the semantic features and the structural features in one stage feature in the spatial dimension and the channel dimension; Inputting the fused features of each stage into the upsampling fusion module for upsampling and feature concatenation to obtain target features; The target features are input into the classification module for classification processing, and the tooth position segmentation result is output.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the tooth position segmentation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the tooth position segmentation method according to any one of claims 1 to 7 when executed.

Images