Tooth microscopic image segmentation method, device, equipment and medium

Abstract

The present disclosure relates to a method, device, equipment and medium for segmenting a tooth microscopic image. A first disturbed image and a second disturbed image of a tooth microscopic image are obtained; based on a feature extraction module in a pre-trained semantic segmentation model, feature extraction processing is performed on the first disturbed image and the second disturbed image respectively to obtain an image feature vector, wherein the semantic segmentation model is a quantization model with a precision less than a preset precision; based on the image segmentation module in the semantic segmentation model, segmentation processing is performed on the image feature vector to obtain a tooth region and a root canal orifice region in the tooth microscopic image. In this way, the semantic segmentation model is used to perform feature extraction and segmentation processing on the two types of disturbed images in turn to achieve accurate segmentation of the tooth microscopic image and determine the tooth region and the root canal orifice region. At the same time, since the semantic segmentation model is a quantization model with a precision less than a preset precision, while maintaining the model performance, the amount of calculation and memory usage are reduced, thereby improving the real-time performance of the segmentation.

Description

Technical Field

[0001] The present disclosure relates to the field of image segmentation technology, and in particular to a method, device, equipment and medium for segmenting tooth microscopic images. Background Art

[0002] When performing root canal repair, the user relies on the video images presented by the microscope (including multiple frames of tooth microscopic images) to determine the root canal opening and the tooth, and customizes the root canal repair plan based on the root canal opening and the tooth to achieve root canal repair.

[0003] Due to the complexity of oral conditions, different users' teeth present different geometric states, which brings greater challenges to the root canal repair process. Therefore, it is necessary to provide a segmentation method with high real-time and accuracy to segment the tooth area and the root canal area from the tooth microscopic image to assist in root canal repair. Summary of the invention

[0004] In order to solve the above technical problems, the present disclosure provides a tooth microscopic image segmentation method, device, equipment and medium.

[0005] In a first aspect, the present disclosure provides a tooth microscopic image segmentation method, the method comprising:

[0006] Acquire a first disturbed image and a second disturbed image of a tooth microscopic image;

[0007] Based on a feature extraction module in a pre-trained semantic segmentation model, feature extraction processing is performed on the first perturbed image and the second perturbed image respectively to obtain image feature vectors, wherein the semantic segmentation model is a quantization model with a precision less than a preset precision;

[0008] Based on the image segmentation module in the semantic segmentation model, the image feature vector is segmented to obtain the tooth area and the root canal orifice area in the tooth microscopic image.

[0009] In a second aspect, the present disclosure provides a tooth microscopic image segmentation device, the device comprising:

[0010] An image acquisition module, used for acquiring a first disturbed image and a second disturbed image of a tooth microscopic image;

[0011] A feature extraction module, configured to perform feature extraction processing on the first perturbed image and the second perturbed image respectively based on a feature extraction module in a pre-trained semantic segmentation model to obtain an image feature vector, wherein the semantic segmentation model is a quantization model with a precision less than a preset precision;

[0012] The segmentation module is used to perform segmentation processing on the image feature vector based on the image segmentation module in the semantic segmentation model to obtain the tooth area and the root canal orifice area in the tooth microscopic image.

[0013] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the device comprising:

[0014] one or more processors;

[0015] a storage device for storing one or more programs,

[0016] When one or more programs are executed by one or more processors, the one or more processors implement the method provided in the first aspect.

[0017] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method provided in the first aspect is implemented.

[0018] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0019] A method, device, equipment and medium for segmenting a tooth microscopic image according to an embodiment of the present disclosure obtains a first disturbed image and a second disturbed image of a tooth microscopic image; based on a feature extraction module in a pre-trained semantic segmentation model, feature extraction is performed on the first disturbed image and the second disturbed image respectively to obtain an image feature vector, wherein the semantic segmentation model is a quantization model with a precision less than a preset precision; based on the image segmentation module in the semantic segmentation model, segmentation is performed on the image feature vector to obtain a tooth region and a root canal orifice region in the tooth microscopic image. In this way, the semantic segmentation model is used to perform feature extraction and segmentation on the two types of disturbed images of the tooth microscopic image in turn to achieve accurate segmentation of the tooth microscopic image and determine the tooth region and the root canal orifice region. At the same time, since the semantic segmentation model is a quantization model with a precision less than a preset precision, while maintaining the model performance, the amount of calculation and memory usage are significantly reduced, thereby improving the real-time performance of the segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0022] Figure 1 A schematic diagram of a process of a tooth microscopic image segmentation method provided in an embodiment of the present disclosure;

[0023] Figure 2 A schematic diagram of a flow chart of another tooth microscopic image segmentation method provided in an embodiment of the present disclosure;

[0024] Figure 3 A flowchart of a semantic segmentation model training method provided in an embodiment of the present disclosure;

[0025] Figure 4 A schematic diagram of the model structure of an initial neural network model provided in an embodiment of the present disclosure;

[0026] Figure 5 A schematic diagram of the structure of a tooth microscopic image segmentation device provided in an embodiment of the present disclosure;

[0027] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0029] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0030] In order to segment the tooth area and the root canal area from the tooth microscopic image to assist in root canal repair, the following is combined with Figures 1 to 4 The tooth microscopic image segmentation method provided in the embodiment of the present disclosure is described. In the embodiment of the present disclosure, the tooth microscopic image segmentation method can be executed by an electronic device. The electronic device can include a device with communication function such as a tablet computer, a desktop computer, a laptop computer, etc., and can also include a device simulated by a virtual machine or a simulator.

[0031] Figure 1 A schematic flow chart of a tooth microscopic image segmentation method provided in an embodiment of the present disclosure is shown.

[0032] like Figure 1 As shown, the tooth microscopic image segmentation method may include the following steps.

[0033] S110, acquiring a first disturbed image and a second disturbed image of a tooth microscopic image.

[0034] In this embodiment, the electronic device acquires a real-time video stream under a dental microscope at a specific speed (e.g., acquiring 30 frames of images per second), and performs weak perturbation processing and strong perturbation processing on each frame of tooth microscopic image in the real-time video stream to obtain a first perturbation image after weak perturbation processing and a second perturbation image after strong perturbation processing. Optionally, the strong perturbation processing can be performed on the basis of the weak perturbation processing. In other words, the second perturbation image can be determined based on the first perturbation image.

[0035] In order to better adapt to the root canal repair process, the tooth microscopic image in the real-time video stream is randomly scaled to an image of 1024×1920 size, and the aspect ratio is kept unchanged during the scaling process. If the image size after scaling is larger than 1024×1920, the size of 1024×1920 is randomly cropped. If the image size after scaling is smaller than 1024×1920, the element value is randomly filled with 0 around the tooth microscopic image to obtain a tooth microscopic image with an image size of 1024×1920. After weak perturbation processing and strong perturbation processing are performed on the tooth microscopic image with an image size of 1024×1920, a first perturbation image with an image size of 1024×1920 and a second perturbation image with an image size of 1024×1920 are generated.

[0036] In some embodiments, the specific implementation method of S110 includes but is not limited to the following methods: performing perturbation processing on the geometric parameters of the tooth microscopic image to obtain a first perturbed image; performing perturbation processing on the image data of the tooth microscopic image to obtain a second perturbed image.

[0037] In other embodiments, the specific implementation method of S110 includes but is not limited to the following methods: performing perturbation processing on the geometric parameters of the tooth microscopic image to obtain a first perturbed image; performing perturbation processing on the image data of the first perturbed image to obtain a second perturbed image.

[0038] Among them, the geometric parameters include scale parameters and rotation parameters, and the image data includes color, blur, grayscale, etc.

[0039] Specifically, the electronic device performs weak disturbance processing such as scaling, cropping, and flipping on the tooth microscopic image to adjust the geometric parameters of the tooth microscopic image to obtain a first disturbed image. At the same time, the tooth microscopic image is subjected to color dithering, blurring, grayscale conversion, and CutMix data enhancement processing to adjust the image data of the tooth microscopic image to obtain a second disturbed image.

[0040] S120. Based on a feature extraction module in a pre-trained semantic segmentation model, perform feature extraction processing on the first perturbed image and the second perturbed image respectively to obtain an image feature vector, wherein the semantic segmentation model is a quantization model with a precision less than a preset precision.

[0041] In order to reduce the computational complexity and memory usage of the segmentation process, the electronic device calls a quantization model with a precision less than a preset accuracy as a semantic segmentation model, and uses a feature extraction module in the semantic segmentation model to extract features from the first perturbed image and the second perturbed image, respectively, to obtain an image feature vector that combines the features of the first perturbed image and the features of the second perturbed image.

[0042] Among them, the preset precision refers to high-precision floating-point precision, specifically a 32-bit floating-point format (FP32), and the quantization model can be half-precision floating-point precision, specifically a 16-bit floating-point format (FP16).

[0043] Optionally, the quantization model can be a TensorRT model formed after conversion based on a real-time semantic segmentation model (such as PIDNet).

[0044] The feature extraction module performs preliminary feature extraction on the first perturbed image and the second perturbed image to obtain an image feature vector. Optionally, the feature extraction module includes but is not limited to a Stem Layer. The image feature vector includes a feature vector of the first perturbed image and a feature vector of the second perturbed image.

[0045] S130. Based on the image segmentation module in the semantic segmentation model, the image feature vector is segmented to obtain the tooth area and the root canal orifice area in the tooth microscopic image.

[0046] In this embodiment, the electronic device inputs the image feature vector output by the feature extraction module into the image segmentation module, so as to use the image segmentation module to segment the image feature vector and output the segmented area of ​​the tooth microscopic image as the tooth area and the root canal orifice area.

[0047] Furthermore, the electronic device determines a root canal repair plan based on the tooth area and the root canal opening area, and assists the user to repair the root canal based on the root canal repair plan.

[0048] A tooth microscopic image segmentation method according to an embodiment of the present disclosure obtains a first disturbed image and a second disturbed image of a tooth microscopic image; based on a feature extraction module in a pre-trained semantic segmentation model, feature extraction is performed on the first disturbed image and the second disturbed image respectively to obtain an image feature vector, wherein the semantic segmentation model is a quantization model with a precision less than a preset precision; based on the image segmentation module in the semantic segmentation model, segmentation is performed on the image feature vector to obtain a tooth region and a root canal orifice region in the tooth microscopic image. In this way, the semantic segmentation model is used to perform feature extraction and segmentation on the two types of disturbed images of the tooth microscopic image in turn to achieve accurate segmentation of the tooth microscopic image and determine the tooth region and the root canal orifice region. At the same time, since the semantic segmentation model is a quantization model with a precision less than a preset precision, while maintaining the model performance, the amount of calculation and memory usage are significantly reduced, thereby improving the real-time performance of the segmentation.

[0049] In another embodiment of the present disclosure, the segmentation process of the image segmentation module is explained in detail.

[0050] Figure 2 A flow chart of another tooth microscopic image segmentation method provided in an embodiment of the present disclosure is shown.

[0051] like Figure 2 As shown, the tooth microscopic image segmentation method may include the following steps.

[0052] S210, acquiring a first disturbed image and a second disturbed image of a tooth microscopic image.

[0053] S220. Based on a feature extraction module in a pre-trained semantic segmentation model, perform feature extraction processing on the first perturbed image and the second perturbed image respectively to obtain an image feature vector, wherein the semantic segmentation model is a quantization model with a precision less than a preset precision.

[0054] Among them, S210 to S220 are similar to S110 to S120 and are not described in detail here.

[0055] S230, based on the semantic segmentation network in the image segmentation module of the semantic segmentation model, performing semantic segmentation processing on the image feature vector to obtain image semantic features.

[0056] In this embodiment, the semantic segmentation network includes: a proportional branch subnetwork, an integral branch subnetwork and a differential branch subnetwork; accordingly, the specific implementation method of S230 includes but is not limited to the following methods: based on the proportional branch subnetwork in the semantic segmentation network, the image feature vector is processed to obtain high-resolution features; based on the integral branch subnetwork in the semantic segmentation network, the image feature vector is processed to obtain context features; based on the differential branch subnetwork in the semantic segmentation network, the image feature vector is processed to obtain high-frequency features; wherein, the high-resolution features, context features and high-frequency features form image semantic features.

[0057] Among them, the proportional branch subnetwork, namely the proportional (P) branch, is used to parse and save the high-resolution features of the image feature vector; the integral branch subnetwork, namely the integral (I) branch, is used to aggregate the local or global features of the image feature vector to obtain contextual features; the differential branch subnetwork, namely the differential (D) branch, is used to extract the high-frequency features of the image feature vector for predicting the boundary area.

[0058] In order to further expand the disturbance space, the image segmentation module also includes a feature disturbance network; accordingly, the specific implementation method of S230 includes but is not limited to the following methods: based on the feature perturbation network in the image segmentation module, the feature vector of the first disturbance image contained in the image feature vector is subjected to feature perturbation processing to obtain the disturbance feature vector; based on the semantic segmentation network in the image segmentation module, the disturbance feature vector and the image feature vector are subjected to semantic segmentation processing to obtain the image semantic feature.

[0059] Among them, the feature perturbation network can specifically be a regularization network, which is used to perform regularization (Dropout) processing on the feature vector of the first perturbation image contained in the image feature vector to achieve the feature perturbation effect and obtain the perturbation feature vector.

[0060] Specifically, the electronic device inputs the acquired disturbance feature vector, the feature vector of the first disturbance image contained in the image feature vector, and the feature vector of the second disturbance image contained in the image feature vector into the semantic segmentation network in the image segmentation module for semantic segmentation processing to obtain the image semantic feature.

[0061] S240, based on the feature fusion network in the image segmentation module of the semantic segmentation model, performing boundary detection processing on the image semantic features to obtain the tooth area and the root canal orifice area in the tooth microscopic image.

[0062] In this embodiment, the electronic device inputs the image semantic features consisting of high-resolution features, context features and high-frequency features into a feature fusion network for feature fusion, so as to segment the tooth area and the root canal orifice area from the tooth microscopic image.

[0063] The feature fusion network refers to a network layer that fuses high-resolution features, context features, and high-frequency features. Optionally, the feature fusion network includes but is not limited to a Bag module.

[0064] In this way, the image segmentation module performs semantic segmentation and boundary detection on the image feature vector in order to accurately determine the tooth area and root canal area in the tooth microscopic image. In addition, by adding a feature perturbation network to the image segmentation module, the features of the perturbation space can be expanded, further improving the accuracy of the tooth area and root canal area.

[0065] In yet another embodiment of the present disclosure, the training process of the semantic segmentation model is explained in detail.

[0066] Figure 3 A flowchart of a semantic segmentation model training method provided by an embodiment of the present disclosure is shown.

[0067] like Figure 3 As shown, the semantic segmentation model training method may include the following steps.

[0068] S310, obtaining a first reference perturbation image, a second reference perturbation image, and a region label of the first reference perturbation image of a tooth microscopic reference image.

[0069] It is understandable that, in practice, the amount of data of the dental microscopic reference image is too large, and it is impossible to fully annotate the reference perturbation image converted from the dental microscopic image manually. Therefore, this embodiment obtains the first reference perturbation image and its partial regional labels, and the second reference perturbation image, and adopts a semi-supervised approach to use the first reference perturbation image and its corresponding regional labels, and the second reference perturbation image for model training, so as to reduce the difficulty of manual annotation.

[0070] The first reference perturbation image refers to a weak perturbation reference image, some of which are pre-labeled with region labels, and the remaining first reference perturbation images do not have pre-labeled region labels. The second reference perturbation image refers to a strong perturbation reference image, and different second reference perturbation images do not have pre-labeled region labels.

[0071] In some embodiments, obtaining a first reference disturbed image and a second reference disturbed image includes: performing perturbation processing on geometric parameters of a tooth microscopic reference image to obtain a first reference disturbed image; and performing perturbation processing on image data of the tooth microscopic reference image to obtain a second reference disturbed image.

[0072] In other embodiments, obtaining a first reference perturbed image and a second reference perturbed image includes: perturbing geometric parameters of a tooth microscopic reference image to obtain a first reference perturbed image, wherein the first reference perturbed image includes a reference perturbed image pre-labeled with region labels and a reference perturbed image without region labels; and perturbing image data of the first reference perturbed image to obtain a second perturbed image.

[0073] Specifically, the electronic device performs weak disturbance processing such as scaling, cropping, and flipping on the tooth microscopic reference image to adjust the geometric parameters of the tooth microscopic reference image to obtain a first reference perturbation image. At the same time, the tooth microscopic reference image is subjected to color dithering, blurring, grayscale conversion, and CutMix data enhancement processing to adjust the image data of the tooth microscopic reference image to obtain a second reference perturbation image.

[0074] S320: Based on the feature extraction module in the initial neural network model, perform feature extraction processing on the first reference perturbation image and the second reference perturbation image respectively to obtain a reference feature vector of the first reference perturbation image and a reference feature vector of the second reference perturbation image.

[0075] The initial neural network model may be an initial semantic segmentation model (such as PIDNet).

[0076] The feature extraction module in the initial neural network model performs preliminary feature extraction on the first reference perturbation image and the second reference perturbation image to obtain a reference feature vector of the first reference perturbation image and a reference feature vector of the second reference perturbation image, and the two reference feature vectors can form an image reference feature vector. Optionally, the feature extraction module includes but is not limited to a Stem Layer.

[0077] For easier understanding, see Figure 4 The schematic diagram of the model structure of the initial neural network model is shown in FIG. Figure 4 As shown, the electronic device converts the first reference disturbance image x w and the second reference perturbation image x s Input the initial neural network model to use the feature extraction module in the initial neural network model to extract x w and x s Perform feature extraction processing to obtain the first reference disturbance image x w The reference eigenvector t w and the reference feature vector t of the second reference perturbation image s .

[0078] S330. Based on the image segmentation module in the initial neural network model, segment the reference feature vector of the first reference perturbed image and the reference feature vector of the second reference perturbed image to obtain a predicted segmentation region of the first reference perturbed image and a predicted segmentation region of the second reference perturbed image.

[0079] In some embodiments, the image segmentation module in the initial neural network model includes a semantic segmentation network and a feature fusion network; accordingly, the specific implementation method of S330 includes but is not limited to the following methods: based on the semantic segmentation network, semantic segmentation processing is performed on the reference feature vector of the first reference perturbation image and the reference feature vector of the second reference perturbation image to obtain the first semantic feature and the second semantic feature; based on the feature fusion network, boundary detection processing is performed on the first semantic feature and the second semantic feature to obtain the predicted segmentation area of ​​the first reference perturbation image and the predicted segmentation area of ​​the second reference perturbation image.

[0080] Wherein, based on the semantic segmentation network, the reference feature vector of the first reference perturbation image and the reference feature vector of the second reference perturbation image are semantically segmented to obtain the first semantic feature and the second semantic feature, including: based on the ratio branch subnetwork in the semantic segmentation network, the reference feature vector of the first reference perturbation image and the reference feature vector of the second reference perturbation image are processed to obtain the high-resolution feature of the first reference perturbation image and the high-resolution feature of the second reference perturbation image; based on the integral branch subnetwork in the semantic segmentation network, the reference feature vector of the first reference perturbation image and the reference feature vector of the second reference perturbation image are processed to obtain the context feature of the first reference perturbation image. The method comprises the steps of: processing a reference feature vector of the first reference perturbed image and a contextual feature of the second reference perturbed image based on a differential branch subnetwork in the semantic segmentation network to obtain high-frequency features of the first reference perturbed image and high-frequency features of the second reference perturbed image; wherein the high-resolution features of the first reference perturbed image, the contextual features of the first reference perturbed image and the high-frequency features of the first reference perturbed image form a first semantic feature of the first reference perturbed image; and the high-resolution features of the second reference perturbed image, the contextual features of the second reference perturbed image and the high-frequency features of the second reference perturbed image form a second semantic feature of the second reference perturbed image.

[0081] In some other embodiments, the image segmentation module in the initial neural network model includes a feature perturbation network, a semantic segmentation network, and a feature fusion network; accordingly, the specific implementation method of S330 includes but is not limited to the following methods: based on the feature perturbation network, the reference feature vector of the first reference perturbation image is subjected to feature perturbation processing to obtain the perturbation feature vector; based on the semantic segmentation network, the reference feature vector of the first reference perturbation image, the reference feature vector of the second reference perturbation image, and the perturbation feature vector are subjected to semantic segmentation processing to obtain the first semantic feature, the second semantic feature, and the perturbation semantic feature; based on the feature fusion network, the first semantic feature, the second semantic feature, and the perturbation semantic feature are subjected to boundary detection processing to obtain the predicted segmentation area of ​​the first reference perturbation image, the predicted segmentation area of ​​the second reference perturbation image, and the predicted segmentation area corresponding to the perturbation feature vector.

[0082] Among them, the specific method of obtaining the first semantic feature, the second semantic feature and the disturbance semantic feature can be found in the above description and will not be repeated here.

[0083] For ease of understanding, see Figure 4 , the predicted segmentation area of ​​the first reference perturbation image is denoted as p w , the predicted segmentation area of ​​the second reference perturbation image is p s , the perturbation semantic feature is recorded as the perturbation feature vector t fp The corresponding predicted segmentation area p fp .

[0084] S340. When the probability value of the predicted segmented area of ​​the first reference perturbed image is greater than a preset probability threshold, the total loss value of the initial neural network model is calculated based on the predicted segmented area of ​​the first reference perturbed image, the area label of the first reference perturbed image, and the predicted segmented area of ​​the second reference perturbed image.

[0085] It should be noted that in order to improve the generalization ability and reliability of semi-supervised learning, after the electronic device determines the probability value of the predicted segmentation area of ​​the first reference perturbation image, it is necessary to judge whether the probability value of the predicted segmentation area of ​​the first reference perturbation image is greater than a preset probability threshold. If it is greater, the predicted segmentation area of ​​the first reference perturbation image is used as a pseudo-label to assist in calculating the total loss value of the initial neural network model.

[0086] In some embodiments, when the image segmentation module in the initial neural network model includes a semantic segmentation network and a feature fusion network, the specific implementation method of S340 includes the following manner: calculating a first loss value based on the predicted segmentation area of ​​the first reference perturbation image and the area label of the first reference perturbation image; calculating a second loss value based on the predicted segmentation area of ​​the first reference perturbation image and the predicted segmentation area of ​​the second reference perturbation image; determining the total loss value of the initial neural network model based on the first loss value and the second loss value.

[0087] Specifically, the total loss value of the initial neural network model in this case may be the sum of the first loss value and the second loss value.

[0088] In other embodiments, when the image segmentation module in the initial neural network model includes a feature perturbation network, a semantic segmentation network, and a feature fusion network, the specific implementation method of S340 includes the following methods: calculating a first loss value based on the predicted segmentation area of ​​the first reference perturbation image and the area label of the first reference perturbation image; calculating a second loss value based on the predicted segmentation area of ​​the first reference perturbation image and the predicted segmentation area of ​​the second reference perturbation image; calculating a third loss value based on the predicted segmentation area of ​​the first reference perturbation image and the predicted segmentation area corresponding to the perturbation feature vector; and determining the total loss value of the initial neural network model based on the first loss value, the second loss value, and the third loss value.

[0089] Specifically, the total loss value of the initial neural network model in this case may be the sum of the first loss value, the second loss value, and the third loss value.

[0090] For ease of understanding, the first loss value is recorded as L s , the second loss value is recorded as L us , the third loss value is recorded as L ufp , then the total loss value of the initial neural network model can be determined as follows:

[0091] L=0.5*L s +0.25*L us +0.25*L ufp

[0092] S350, iteratively adjusting the network parameters of each network layer in the initial neural network model based on the total loss value of the initial neural network model, until the total loss value under the current number of adjustments is less than the preset loss value threshold, and obtaining a semantic segmentation model.

[0093] In this embodiment, the specific method of obtaining the semantic segmentation model includes but is not limited to the following methods: when the total loss value at the current adjustment times is less than a preset loss value threshold, obtaining the neural network model at the current adjustment times; performing format conversion on the neural network model at the current adjustment times to obtain a model in an open neural network exchange format; converting the model in the open neural network exchange format into a quantized model with a precision less than a preset precision as a semantic segmentation model.

[0094] Among them, the model in the Open Neural Network Exchange (ONNX) format is a cross-platform, cross-framework model. The semantic segmentation model is a quantized model (TensorRT model). The preset precision is a 32-bit floating point format (FP32) model. Since the semantic segmentation model is a 16-bit floating point format (FP16) model, compared to the 32-bit floating point format (FP32) model, the precision is lower, while maintaining the model performance, it can significantly reduce the amount of calculation and content occupancy.

[0095] In this way, by adopting a semi-supervised model training method and taking the predicted segmented area of ​​the first reference perturbation image as a pseudo-label, the cost of manual annotation can be reduced while improving the training accuracy of the model. In addition, by adding a feature perturbation network to the initial neural network, the features of the perturbation space can be expanded, further improving the semi-supervised learning ability.

[0096] In some embodiments, a tooth microscopic image is input into a pre-trained semantic segmentation model, and a probability matrix of size 3×1024×1920 is output, where 3 represents three channels, corresponding to the three categories of background, tooth, and root canal orifice. Each element value in the matrix represents the probability that the pixel at that position is predicted to be the corresponding category.

[0097] The present disclosure also provides a tooth microscopic image segmentation device for implementing the above-mentioned tooth microscopic image segmentation method. Figure 5 In the disclosed embodiment, the tooth microscopic image segmentation device may be an electronic device, which may include a tablet computer, a desktop computer, a laptop computer, or other device with a communication function, or a device simulated by a virtual machine or a simulator.

[0098] Figure 5 A schematic structural diagram of a tooth microscopic image segmentation device provided in an embodiment of the present disclosure is shown.

[0099] like Figure 5 As shown, the tooth microscopic image segmentation device 500 may include:

[0100] An image acquisition module 510, for acquiring a first disturbed image and a second disturbed image of a tooth microscopic image;

[0101] A feature extraction module 520 is used to perform feature extraction processing on the first perturbed image and the second perturbed image respectively based on a feature extraction module in a pre-trained semantic segmentation model to obtain an image feature vector, wherein the semantic segmentation model is a quantization model with a precision less than a preset precision;

[0102] The segmentation module 530 is used to perform segmentation processing on the image feature vector based on the image segmentation module in the semantic segmentation model to obtain the tooth area and the root canal orifice area in the tooth microscopic image.

[0103] A tooth microscopic image segmentation device according to an embodiment of the present disclosure obtains a first disturbed image and a second disturbed image of a tooth microscopic image; based on a feature extraction module in a pre-trained semantic segmentation model, feature extraction is performed on the first disturbed image and the second disturbed image respectively to obtain an image feature vector, wherein the semantic segmentation model is a quantization model with a precision less than a preset precision; based on the image segmentation module in the semantic segmentation model, segmentation is performed on the image feature vector to obtain a tooth region and a root canal orifice region in the tooth microscopic image. In this way, the semantic segmentation model is used to perform feature extraction and segmentation on the two types of disturbed images of the tooth microscopic image in turn to achieve accurate segmentation of the tooth microscopic image and determine the tooth region and the root canal orifice region. At the same time, since the semantic segmentation model is a quantization model with a precision less than a preset precision, while maintaining the model performance, the amount of calculation and memory usage are significantly reduced, thereby improving the real-time performance of the segmentation.

[0104] In some embodiments of the present disclosure, the image segmentation module includes a semantic segmentation network and a feature fusion network; accordingly, the segmentation module 530 includes:

[0105] A semantic segmentation unit, configured to perform semantic segmentation processing on the image feature vector based on the semantic segmentation network in the image segmentation module to obtain image semantic features;

[0106] The boundary detection unit is used to perform boundary detection processing on the image semantic features based on the feature fusion network in the image segmentation module to obtain the tooth area and the root canal orifice area in the tooth microscopic image.

[0107] In some embodiments of the present disclosure, the image segmentation module further includes: a feature perturbation network; and correspondingly, a semantic segmentation unit, specifically configured to:

[0108] Based on the feature perturbation network in the image segmentation module, feature perturbation processing is performed on the feature vector of the first perturbation image contained in the image feature vector to obtain a perturbation feature vector;

[0109] Based on the semantic segmentation network in the image segmentation module, semantic segmentation processing is performed on the disturbance feature vector and the image feature vector to obtain the image semantic features.

[0110] In some embodiments of the present disclosure, the semantic segmentation network includes: a proportional branch subnetwork, an integral branch subnetwork and a differential branch subnetwork; accordingly, the semantic segmentation unit is specifically used to:

[0111] Based on the proportional branch subnetwork in the semantic segmentation network, the image feature vector is processed to obtain high-resolution features;

[0112] Based on the integral branch subnetwork in the semantic segmentation network, the image feature vector is processed to obtain context features;

[0113] Based on the differential branch subnetwork in the semantic segmentation network, the image feature vector is processed to obtain high-frequency features;

[0114] The high-resolution features, the context features and the high-frequency features form the image semantic features.

[0115] In some embodiments of the present disclosure, the device further includes:

[0116] A sample acquisition module, used to acquire a first reference perturbation image, a second reference perturbation image and a region label of the first reference perturbation image of a tooth microscopic reference image;

[0117] A reference feature vector extraction module, configured to perform feature extraction processing on the first reference perturbed image and the second reference perturbed image respectively based on the feature extraction module in the initial neural network model, and obtain a reference feature vector of the first reference perturbed image and a reference feature vector of the second reference perturbed image;

[0118] A segmentation region acquisition module, configured to perform segmentation processing on the reference feature vector of the first reference perturbed image and the reference feature vector of the second reference perturbed image based on the image segmentation module in the initial neural network model, and acquire a predicted segmentation region of the first reference perturbed image and a predicted segmentation region of the second reference perturbed image;

[0119] a loss calculation module, configured to calculate a total loss value of the initial neural network model based on the predicted segmented region of the first reference perturbed image, the region label of the first reference perturbed image, and the predicted segmented region of the second reference perturbed image when the probability value of the predicted segmented region of the first reference perturbed image is greater than a preset probability threshold;

[0120] An iterative training module is used to iteratively adjust the network parameters of each network layer in the initial neural network model based on the total loss value of the initial neural network model until the total loss value under the current number of adjustments is less than a preset loss value threshold, thereby obtaining the semantic segmentation model.

[0121] In some embodiments of the present disclosure, the loss calculation module is specifically used to:

[0122] Calculating a first loss value based on the predicted segmented region of the first reference perturbation image and the region label of the first reference perturbation image;

[0123] Calculating a second loss value based on the predicted segmented region of the first reference perturbed image and the predicted segmented region of the second reference perturbed image;

[0124] Based on the first loss value and the second loss value, a total loss value of the initial neural network model is determined.

[0125] In some embodiments of the present disclosure, the iterative training module is specifically used to:

[0126] When the total loss value under the current number of adjustments is less than the preset loss value threshold, obtaining the neural network model under the current number of adjustments;

[0127] Performing format conversion on the neural network model under the current adjustment times to obtain a model in an open neural network exchange format;

[0128] The model in the open neural network exchange format is converted into a quantized model with a precision less than a preset precision as the semantic segmentation model.

[0129] It should be noted that Figure 5 The tooth microscopic image segmentation device 500 shown can perform Figures 1 to 4 The various steps in the method embodiment shown in the figure are implemented Figures 1 to 4 The various processes and effects in the method embodiment shown are not described in detail here.

[0130] Figure 6 A schematic structural diagram of an electronic device provided by an embodiment of the present disclosure is shown.

[0131] like Figure 6 As shown, the electronic device may include a processor 601 and a memory 602 storing computer program instructions.

[0132] Specifically, the processor 601 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0133] The memory 602 may include a large capacity memory for information or instructions. By way of example and not limitation, the memory 602 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 602 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 602 may be inside or outside the integrated gateway device. In a particular embodiment, the memory 602 is a non-volatile solid-state memory. In a particular embodiment, the memory 602 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (Electrically Erasable Programmable ROM, EEPROM), an electrically rewritable ROM (EAROM) or a flash memory, or a combination of two or more of these.

[0134] The processor 601 reads and executes the computer program instructions stored in the memory 602 to perform the steps of the tooth microscopic image segmentation method provided in the embodiment of the present disclosure.

[0135] In one example, the electronic device may further include a transceiver 603 and a bus 604. Figure 6 As shown, the processor 601, the memory 602 and the transceiver 603 are connected via a bus 604 and communicate with each other.

[0136] The bus 604 includes hardware, software, or both. For example, but not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, the bus 604 may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.

[0137] The following is an embodiment of a computer-readable storage medium provided in an embodiment of the present disclosure. The computer-readable storage medium and the tooth microscopic image segmentation method of the above-mentioned embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the computer-readable storage medium, reference can be made to the embodiment of the above-mentioned tooth microscopic image segmentation method.

[0138] This embodiment provides a storage medium containing computer executable instructions. When the computer executable instructions are executed by a computer processor, they are used to perform a tooth microscopic image segmentation method. The method includes:

[0139] Acquire a first disturbed image and a second disturbed image of a tooth microscopic image;

[0140] Based on a feature extraction module in a pre-trained semantic segmentation model, feature extraction processing is performed on the first perturbed image and the second perturbed image respectively to obtain image feature vectors, wherein the semantic segmentation model is a quantization model with a precision less than a preset precision;

[0141] Based on the image segmentation module in the semantic segmentation model, the image feature vector is segmented to obtain the tooth area and the root canal orifice area in the tooth microscopic image.

[0142] Of course, the computer executable instructions of a storage medium containing computer executable instructions provided in an embodiment of the present disclosure are not limited to the above method operations, and can also execute related operations in the tooth microscopic image segmentation method provided in any embodiment of the present disclosure.

[0143] Through the above description of the implementation methods, technicians in the relevant field can clearly understand that the present disclosure can be implemented with the help of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present disclosure can essentially or contribute to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer cloud platform (which can be a personal computer, server, or network cloud platform, etc.) to execute the tooth microscopic image segmentation method provided by each embodiment of the present disclosure.

[0144] Note that the above are only preferred embodiments of the present disclosure and the technical principles used. Those skilled in the art will understand that the present disclosure is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present disclosure. Therefore, although the present disclosure is described in more detail through the above embodiments, the present disclosure is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present disclosure, and the scope of the present disclosure is determined by the scope of the appended claims.

Claims

1. A tooth microscopic image segmentation method, characterized in that: include: Acquire a first reference perturbation image, a second reference perturbation image, and a region label of the first reference perturbation image of a tooth microscopic reference image; Based on the feature extraction module in the initial neural network model, feature extraction processing is performed on the first reference perturbation image and the second reference perturbation image respectively to obtain a reference feature vector of the first reference perturbation image and a reference feature vector of the second reference perturbation image; Based on the image segmentation module in the initial neural network model, segment the reference feature vector of the first reference perturbed image and the reference feature vector of the second reference perturbed image to obtain a predicted segmentation region of the first reference perturbed image and a predicted segmentation region of the second reference perturbed image; When the probability value of the predicted segmented region of the first reference perturbed image is greater than a preset probability threshold, calculating a total loss value of the initial neural network model based on the predicted segmented region of the first reference perturbed image, the region label of the first reference perturbed image, and the predicted segmented region of the second reference perturbed image; Iteratively adjusting the network parameters of each network layer in the initial neural network model based on the total loss value of the initial neural network model until the total loss value under the current number of adjustments is less than a preset loss value threshold, thereby obtaining a semantic segmentation model; Acquire a first disturbed image and a second disturbed image of a tooth microscopic image; Based on a feature extraction module in a pre-trained semantic segmentation model, feature extraction processing is performed on the first perturbed image and the second perturbed image respectively to obtain image feature vectors, wherein the semantic segmentation model is a quantization model with a precision less than a preset precision; Based on the image segmentation module in the semantic segmentation model, the image feature vector is segmented to obtain the tooth area and the root canal orifice area in the tooth microscopic image.

2. The method according to claim 1, characterized in that: The image segmentation module includes a semantic segmentation network and a feature fusion network; Accordingly, the image segmentation module in the semantic segmentation model performs segmentation processing on the image feature vector to obtain the tooth area and the root canal orifice area in the tooth microscopic image, including: Based on the semantic segmentation network in the image segmentation module, performing semantic segmentation processing on the image feature vector to obtain image semantic features; Based on the feature fusion network in the image segmentation module, the image semantic features are subjected to boundary detection processing to obtain the tooth area and the root canal orifice area in the tooth microscopic image.

3. The method according to claim 2, characterized in that The image segmentation module also includes: a feature perturbation network; Correspondingly, the semantic segmentation network in the image segmentation module performs semantic segmentation processing on the image feature vector to obtain the image semantic features, including: Based on the feature perturbation network in the image segmentation module, feature perturbation processing is performed on the feature vector of the first perturbation image contained in the image feature vector to obtain a perturbation feature vector; Based on the semantic segmentation network in the image segmentation module, semantic segmentation processing is performed on the disturbance feature vector and the image feature vector to obtain the image semantic features.

4. The method according to claim 2, characterized in that: The semantic segmentation network includes: a proportional branch sub-network, an integral branch sub-network and a differential branch sub-network; Correspondingly, the semantic segmentation network in the image segmentation module performs semantic segmentation processing on the image feature vector to obtain the image semantic features, including: Based on the proportional branch subnetwork in the semantic segmentation network, the image feature vector is processed to obtain high-resolution features; Based on the integral branch subnetwork in the semantic segmentation network, the image feature vector is processed to obtain context features; Based on the differential branch subnetwork in the semantic segmentation network, the image feature vector is processed to obtain high-frequency features; The high-resolution features, the context features and the high-frequency features form the image semantic features.

5. The method according to claim 1, characterized in that The calculating the total loss value of the initial neural network model based on the predicted segmented region of the first reference perturbation image, the region label of the first reference perturbation image, and the predicted segmented region of the second reference perturbation image includes: Calculating a first loss value based on the predicted segmented region of the first reference perturbation image and the region label of the first reference perturbation image; Calculating a second loss value based on the predicted segmented region of the first reference perturbed image and the predicted segmented region of the second reference perturbed image; Based on the first loss value and the second loss value, a total loss value of the initial neural network model is determined.

6. The method according to claim 1, characterized in that The obtaining of the semantic segmentation model comprises: When the total loss value under the current number of adjustments is less than the preset loss value threshold, obtaining the neural network model under the current number of adjustments; Performing format conversion on the neural network model under the current adjustment times to obtain a model in an open neural network exchange format; The model in the open neural network exchange format is converted into a quantized model with a precision less than a preset precision as the semantic segmentation model.

7. A tooth microscopic image segmentation device, characterized in that: include: A sample acquisition module, used to acquire a first reference perturbation image, a second reference perturbation image and a region label of the first reference perturbation image of a tooth microscopic reference image; A reference feature vector extraction module, configured to perform feature extraction processing on the first reference perturbed image and the second reference perturbed image respectively based on the feature extraction module in the initial neural network model, and obtain a reference feature vector of the first reference perturbed image and a reference feature vector of the second reference perturbed image; A segmentation region acquisition module, configured to perform segmentation processing on the reference feature vector of the first reference perturbed image and the reference feature vector of the second reference perturbed image based on the image segmentation module in the initial neural network model, and acquire a predicted segmentation region of the first reference perturbed image and a predicted segmentation region of the second reference perturbed image; a loss calculation module, configured to calculate a total loss value of the initial neural network model based on the predicted segmented region of the first reference perturbed image, the region label of the first reference perturbed image, and the predicted segmented region of the second reference perturbed image when the probability value of the predicted segmented region of the first reference perturbed image is greater than a preset probability threshold; An iterative training module, used for iteratively adjusting the network parameters of each network layer in the initial neural network model based on the total loss value of the initial neural network model, until the total loss value under the current number of adjustments is less than a preset loss value threshold, thereby obtaining a semantic segmentation model; An image acquisition module, used for acquiring a first disturbed image and a second disturbed image of a tooth microscopic image; A feature extraction module, configured to perform feature extraction processing on the first perturbed image and the second perturbed image respectively based on a feature extraction module in a pre-trained semantic segmentation model to obtain an image feature vector, wherein the semantic segmentation model is a quantization model with a precision less than a preset precision; The segmentation module is used to perform segmentation processing on the image feature vector based on the image segmentation module in the semantic segmentation model to obtain the tooth area and the root canal orifice area in the tooth microscopic image.

8. An electronic device, characterized in that: include: processor; A memory for storing executable instructions; The processor is used to read the executable instructions from the memory and execute the executable instructions to implement the method described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the method according to any one of claims 1 to 6.

Images