Method and device for detecting positional relationship between molar root and neural tube

Abstract

The invention discloses a method and a device for detecting a position relationship between a molar root and a neural tube. The method comprises the following steps: acquiring a curved surface body layer image; inputting the curved surface body layer image into the trained detection model to obtain a detection result; wherein the detection model comprises a feature extraction network, a feature fusion network and a prediction network, and the feature extraction network comprises a lightweight cross-stage module and a space-channel mixed attention module; the step of obtaining the detection result comprises the following steps: inputting a curved surface body layer image into a feature extraction network, processing the curved surface body layer image through a lightweight cross-stage module and a space-channel mixed attention module to obtain a plurality of sub-feature maps, inputting the plurality of sub-feature maps into a feature fusion network for processing to obtain a plurality of output feature maps, and outputting the plurality of output feature maps. And inputting the plurality of output feature maps into a prediction network to obtain a predicted bounding box and target confidence. According to the invention, the recognition precision of the curved surface body layer image can be improved, and the network calculation complexity and parameter quantity can be reduced.

Description

Technical Field

[0001] The present application relates to the field of computer vision prediction technology, and in particular to a method, device, storage medium and electronic device for detecting the positional relationship between a molar root and a neural tube. Background Art

[0002] Mandibular third molar extraction surgery is a common surgery in the oral and maxillofacial surgery clinic. Failure of extraction surgery will cause permanent physical damage to the nerves in the adjacent mandibular canal, ultimately affecting the patient's normal speech, chewing and other physiological functions. Therefore, in order to avoid nerve damage in the mandibular canal caused by failure of extraction surgery, it is necessary to determine the positional relationship between the molar root and the neural canal.

[0003] As deep learning has shown good performance in tasks such as image target detection and recognition, some scholars have applied deep learning models to the field of oral medical image processing and analysis. At present, mandibular third molars are generally classified and detected through deep learning models such as VGG 16 and YOLOv4. However, when dealing with complex anatomical structures and variable image quality, the recognition accuracy of these models needs to be improved, and incorrect classification and detection may occur, which may lead to misjudgment of clinical decisions. In addition, the processing speed of existing models is still not ideal for large-scale data sets and real-time clinical applications. The long training and inference time limits the application of these models in actual clinical environments. Summary of the invention

[0004] The embodiments of the present application provide a method, device, storage medium and electronic device for detecting the positional relationship between a molar root and a neural tube, which can improve the recognition accuracy of curved body layer images and reduce the complexity of network calculations and the amount of parameters.

[0005] The present application embodiment provides a method for detecting the positional relationship between a molar root and a neural tube, comprising: Acquiring curved tomographic images; Inputting the curved body layer image into a trained detection model to obtain a detection result; wherein the detection model includes a feature extraction network, a feature fusion network and a prediction network, and the feature extraction network includes a lightweight cross-stage module and a space-channel hybrid attention module; The step of inputting the curved volume layer image into the trained detection model to obtain the detection result includes: inputting the curved volume layer image into the feature extraction network, obtaining a plurality of sub-feature maps after processing by the lightweight cross-stage module and the spatial-channel hybrid attention module, inputting the plurality of sub-feature maps into the feature fusion network for processing to obtain a plurality of output feature maps, and inputting the plurality of output feature maps into the prediction network to obtain a predicted bounding box and target confidence.

[0006] Furthermore, in the above-mentioned method for detecting the positional relationship between the molar root and the neural tube, the feature extraction network includes a convolution-batch normalization-activation function combination layer, a first type of lightweight cross-stage module, a second type of lightweight cross-stage module, a space-channel conversion hybrid attention module, and a space pyramid pooling module; The curved body layer image is input into the feature extraction network, and processed by the lightweight cross-stage module and the spatial-channel hybrid attention module to obtain multiple sub-feature maps, including: Input the curved volume layer image into the convolution-batch normalization-activation function combination layer for two feature extractions, and then sequentially input the image into a first type of lightweight cross-stage module, a space-channel conversion hybrid attention module, and a second type of lightweight cross-stage module, and input the output of the second type of lightweight cross-stage module into a space-channel conversion hybrid attention module again to obtain a first sub-feature map; Inputting the first sub-characteristic map into a first-type lightweight cross-stage module and a second-type lightweight cross-stage module in sequence to obtain a second sub-characteristic map; The second sub-feature map is sequentially input into a convolution-batch normalization-activation function combination layer, a first-class lightweight cross-stage module, a spatial-channel conversion hybrid attention module and a spatial pyramid pooling module to obtain the third sub-feature map.

[0007] Furthermore, in the above-mentioned method for detecting the positional relationship between the molar root and the neural tube, in the space-channel hybrid attention module, the processing of the original feature map input into the space-channel hybrid attention module includes: Performing group normalization on the original feature map; Obtaining a normalized scaling factor based on the trainable parameters generated during the group normalization process; Mapping the scaling factor and obtaining a first attention weight and a second attention weight through a gating unit; Multiplying the first attention weight and the second attention weight by the original feature map respectively to obtain a first weighted feature map and a second weighted feature map; Pooling the first weighted feature map and the second weighted feature map respectively to obtain a first space-channel information descriptor and a second space-channel information descriptor; Calculating a first feature weight vector and a second feature weight vector based on the first space-channel information descriptor and the second space-channel information descriptor; The output of the spatial-channel hybrid attention module is obtained based on the first feature weight vector, the second feature weight vector, the first weighted feature map and the second weighted feature map.

[0008] Furthermore, in the above-mentioned method for detecting the positional relationship between the molar root and the neural tube, the first type of lightweight cross-stage modules includes a convolution-batch normalization-activation function combination layer and a lightweight convolution-batch normalization-activation function combination layer; In the first type of lightweight cross-stage module, processing the feature map input to the first type of lightweight cross-stage module includes: Inputting the feature map input to the first type of lightweight cross-stage module into a convolution-batch normalization-activation function combination layer to obtain a first processed feature map; Inputting the first processed feature map into two lightweight convolution-batch normalization-activation function combination layers in sequence to obtain a second processed feature map; Inputting the first processed feature map into a convolution-batch normalization-activation function combination layer for processing to obtain a third processed feature map; Performing an element-by-element addition operation on the features in the first processing feature map and the features in the second processing feature map, and concatenating the result of the element-by-element addition with the third processing feature map; The result of the splicing operation is input into a convolution-batch normalization-activation function combination layer to obtain the output of the first type of lightweight cross-stage module.

[0009] Furthermore, in the above-mentioned method for detecting the positional relationship between the molar root and the neural tube, the second type of lightweight cross-stage module includes a convolution-batch normalization-activation function combination layer, a lightweight convolution-batch normalization-activation function combination layer and a depth-separable convolution layer; In the second type of lightweight cross-stage module, the processing of the feature map input to the second type of lightweight cross-stage module includes: Inputting the feature map input to the second type of lightweight cross-stage module into a convolution-batch normalization-activation function combination layer to obtain a fourth processed feature map; Inputting the fourth processed feature map into a lightweight convolution-batch normalization-activation function combination layer, a depth-separable convolution and another lightweight convolution-batch normalization-activation function combination layer in sequence to obtain a fifth processed feature map; Inputting the fourth processed feature map into a lightweight convolution-batch normalization-activation function combination layer and a convolution-batch normalization-activation function combination layer in sequence to obtain a sixth processed feature map; Inputting the fourth processed feature map into a convolution-batch normalization-activation function combination layer for processing to obtain a seventh processed feature map; Performing an element-by-element addition operation on the features in the fifth processing feature map and the features in the sixth processing feature map, and performing a splicing operation on the result of the element-by-element addition and the seventh processing feature map; The result of the concatenation operation is input into a convolution-batch normalization-activation function combination layer to obtain the output of the second type of lightweight cross-stage module.

[0010] Furthermore, in the above-mentioned method for detecting the positional relationship between the molar root and the neural tube, the feature fusion network includes a convolution-batch normalization-activation function combination layer, an upsampling module, a splicing module and a second type of lightweight cross-stage module; The inputting the multiple sub-feature maps into the feature fusion network for processing to obtain multiple output feature maps includes: Inputting the third sub-feature map into a convolution-batch normalization-activation function combination layer to obtain a fourth sub-feature map; Upsampling the fourth sub-feature map and concatenating it with the second sub-feature map to obtain an eighth processed feature map, and sequentially inputting the eighth processed feature map into a second type of lightweight cross-stage module and a convolution-batch normalization-activation function combination layer to obtain a fifth sub-feature map; The fifth sub-feature map is upsampled, concatenated with the first sub-feature map, and then passed through a second type lightweight cross-stage module to obtain a first output feature map; Passing the first output feature map through a convolution-batch normalization-activation function combination layer, and then concatenating it with the fourth sub-feature map, and inputting the concatenated features into a second type of lightweight cross-stage module to obtain a second output feature map; The second output feature map is concatenated with the fourth sub-feature map after passing through a convolution-batch normalization-activation function combination layer, and the concatenated features are input into a second-type lightweight cross-stage module to obtain a third output feature map.

[0011] Furthermore, the above-mentioned method for detecting the positional relationship between the molar root and the neural tube, wherein the inputting of the multiple output feature maps into the prediction network to obtain the predicted bounding box and target confidence comprises: Inputting the first output feature map, the second output feature map, and the third output feature map into different convolution-batch normalization-activation function combination layers respectively to obtain a first multi-scale feature map, a second multi-scale feature map, and a third multi-scale feature map; Prediction is performed based on the first multi-scale feature map, the second multi-scale feature map, and the third multi-scale feature map to obtain a bounding box and an object confidence.

[0012] The embodiment of the present application also provides a device for detecting the position relationship between a molar root and a neural tube, comprising: An acquisition module, used for acquiring curved body layer images; A detection module, used for inputting the curved surface body layer image into a trained detection model to obtain a detection result; wherein the detection model includes a feature extraction network, a feature fusion network and a prediction network, and the feature extraction network includes a lightweight cross-stage module and a space-channel hybrid attention module; The step of inputting the curved volume layer image into the trained detection model to obtain the detection result includes: inputting the curved volume layer image into the feature extraction network, obtaining a plurality of sub-feature maps after processing by the lightweight cross-stage module and the spatial-channel hybrid attention module, inputting the plurality of sub-feature maps into the feature fusion network for processing to obtain a plurality of output feature maps, and inputting the plurality of output feature maps into the prediction network to obtain a predicted bounding box and target confidence.

[0013] An embodiment of the present application also provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions are suitable for being loaded by a processor to execute any of the above-mentioned methods for detecting the positional relationship between a molar root and a neural tube.

[0014] An embodiment of the present application also provides an electronic device, including a processor and a memory, wherein the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used for the steps in the method for detecting the positional relationship between the molar root and the neural tube as described in any of the above items.

[0015] The present application provides a method, device, storage medium and electronic device for detecting the positional relationship between the molar root and the neural tube. The present application sets up a lightweight cross-stage module. The lightweight cross-stage module adds a lightweight convolution-batch normalization-activation function combination layer to maintain the overall low computational complexity and parameter amount of the network, thereby enhancing the detection accuracy and speed of the positional relationship between the mandibular third molar root and the neural tube. The present application also sets up a space-channel conversion hybrid attention module. The space-channel conversion hybrid attention module extracts features from the channel and space dimensions respectively through the network, and assigns different weights to these features to achieve a sequential mixed conversion process. The module dynamically adjusts the feature weights in both the channel and space aspects, effectively enhancing the network's expression ability, enabling it to suppress redundant information in both the channel and space dimensions, and improve recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The technical solution and other beneficial effects of the present application will be made apparent by describing in detail the specific implementation methods of the present application in conjunction with the accompanying drawings.

[0017] Figure 1A flow chart of a method for detecting the positional relationship between a molar root and a neural canal provided in an embodiment of the present application.

[0018] Figure 2 Another flow chart of the method for detecting the positional relationship between the molar root and the neural canal provided in an embodiment of the present application.

[0019] Figure 3 A schematic diagram of the structure of each module in the feature extraction network provided in an embodiment of the present application.

[0020] Figure 4 A flowchart of the processing of the original feature map in the spatial-channel mixed attention module provided in an embodiment of the present application.

[0021] Figure 5 A bounding box image provided in an embodiment of the present application.

[0022] Figure 6 Flowchart of the training process of the detection model provided in the embodiment of the present application Figure 7 A schematic diagram of the structure of a device for detecting the positional relationship between a molar root and a neural tube provided in an embodiment of the present application.

[0023] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0024] Fig. 9 Another schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0026] The embodiments of the present application provide a method, device, storage medium and electronic device for detecting the positional relationship between a molar root and a neural tube. The embodiments of the present application provide a device for detecting the positional relationship between a molar root and a neural tube, which can be integrated into an electronic device, which can be a terminal, a server or other device, wherein the terminal can include a tablet computer, a laptop computer, a personal computer (PC), a microprocessor box, or other devices.

[0027] See also Figure 1 , Figure 1A flow chart of a method for detecting the positional relationship between a molar root and a neural tube provided in an embodiment of the present application, which is applied to an electronic device, and the method for detecting the positional relationship between a molar root and a neural tube comprises the following steps: S1, obtaining a curved tomographic image.

[0028] S2, input the curved body layer image into the trained detection model to obtain the detection result; wherein, the detection model includes a feature extraction network, a feature fusion network and a prediction network, and the feature extraction network includes a lightweight cross-stage module and a space-channel hybrid attention module.

[0029] The curved body layer image is input into the trained detection model to obtain the detection results, including: inputting the curved body layer image into the feature extraction network, obtaining multiple sub-feature maps after processing through the lightweight cross-stage module and the space-channel hybrid attention module, inputting the multiple sub-feature maps into the feature fusion network for processing to obtain multiple output feature maps, and inputting the multiple output feature maps into the prediction network to obtain the predicted bounding box and target confidence.

[0030] Figure 2 Another flow chart of the method for detecting the position relationship between the molar root and the neural tube provided in the embodiment of the present application, as shown in FIG. Figure 2 As shown, the feature extraction network includes multiple convolution-batch normalization-activation function combination layers (CBS), multiple first-type lightweight cross-stage modules (C3Ghost_1), multiple second-type lightweight cross-stage modules (C3Ghost_2), multiple spatial-channel conversion hybrid attention modules (SCH) and a spatial pyramid pooling module (SPPF).

[0031] In step S2, the curved body layer image is input into the feature extraction network, and multiple sub-feature maps are obtained after being processed by the lightweight cross-stage module and the spatial-channel hybrid attention module, including the following steps: S21, inputting the curved volume layer image into a convolution-batch normalization-activation function combination layer for two feature extractions, and then sequentially inputting the image into a first type of lightweight cross-stage module, a space-channel conversion hybrid attention module, and a second type of lightweight cross-stage module, and inputting the output of the second type of lightweight cross-stage module into a space-channel conversion hybrid attention module again to obtain a first sub-feature map; S22, inputting the first sub-feature map into a first-type lightweight cross-stage module and a second-type lightweight cross-stage module in sequence to obtain a second sub-feature map.

[0032] S23, the second sub-feature map is sequentially input into a convolution-batch normalization-activation function combination layer, a first-class lightweight cross-stage module, a spatial-channel conversion hybrid attention module and a spatial pyramid pooling module to obtain a third sub-feature map.

[0033] Figure 3 A schematic diagram of the structure of each module in the feature extraction network provided in the embodiment of the present application, such as Figure 3 As shown, the first type of lightweight cross-stage modules includes convolution-batch normalization-activation function combination layers and lightweight convolution-batch normalization-activation function combination layers.

[0034] In the first type of lightweight cross-stage module, the processing of the feature map input to the first type of lightweight cross-stage module includes: inputting the feature map input to the first type of lightweight cross-stage module into a convolution-batch normalization-activation function combination layer to obtain a first processed feature map; inputting the first processed feature map into two lightweight convolution-batch normalization-activation function combination layers in sequence to obtain a second processed feature map; inputting the first processed feature map into a convolution-batch normalization-activation function combination layer for processing to obtain a third processed feature map; performing an element-by-element addition operation on the features in the first processed feature map and the features in the second processed feature map, and performing a splicing operation on the result of the element-by-element addition and the third processed feature map; and inputting the result of the splicing operation into a convolution-batch normalization-activation function combination layer to obtain the output of the first type of lightweight cross-stage module.

[0035] The second type of lightweight cross-stage modules includes convolution-batch normalization-activation function combination layer, lightweight convolution-batch normalization-activation function combination layer (GhostConv) and depth-separable convolution layer (DWConv). In the second type of lightweight cross-stage module, the processing of the feature map input to the second type of lightweight cross-stage module includes: inputting the feature map input to the second type of lightweight cross-stage module into a convolution-batch normalization-activation function combination layer to obtain a fourth processed feature map; inputting the fourth processed feature map into a lightweight convolution-batch normalization-activation function combination layer, a depth-separable convolution and another lightweight convolution-batch normalization-activation function combination layer in sequence to obtain a fifth processed feature map; inputting the fourth processed feature map into a lightweight convolution-batch normalization-activation function combination layer and a convolution-batch normalization-activation function combination layer in sequence to obtain a sixth processed feature map; inputting the fourth processed feature map into a convolution-batch normalization-activation function combination layer for processing to obtain a seventh processed feature map; performing an element-by-element addition operation on the features in the fifth processed feature map and the features in the sixth processed feature map, and performing a splicing operation on the result of the element-by-element addition and the seventh processed feature map; and inputting the result of the splicing operation into a convolution-batch normalization-activation function combination layer to obtain the output of the second type of lightweight cross-stage module.

[0036] The lightweight convolution-batch normalization-activation function combination layer includes a convolution-batch normalization-activation function combination layer and a two-dimensional convolution layer (Conv2d).

[0037] In the lightweight convolution-batch normalization-activation function combination layer, the processing of the feature map input to the lightweight convolution-batch normalization-activation function combination layer includes: inputting the feature map input to the lightweight convolution-batch normalization-activation function combination layer into the convolution-batch normalization-activation function combination layer to obtain a ninth processed feature map, inputting the ninth processed feature map into the two-dimensional convolution layer to obtain a tenth processed feature map, and splicing the ninth processed feature map with the tenth processed feature map to obtain the output of the lightweight convolution-batch normalization-activation function combination layer.

[0038] The convolution-batch normalization-activation function combination layer includes multiple two-dimensional convolution layers, and the processing of the feature map input to the convolution-batch normalization-activation function combination layer includes: performing at least three two-dimensional convolution processes on the input feature map.

[0039] The spatial pyramid pooling module includes multiple convolution-batch normalization-activation function combination layers and multiple pooling layers. The processing of the feature map input to the spatial pyramid pooling module includes: inputting the input feature map into a convolution-batch normalization-activation function combination layer to obtain feature map A, inputting feature map A into pooling layer 1 to obtain feature map B, inputting feature map B into pooling layer 2 to obtain feature map C, inputting feature map C into pooling layer 3 to obtain feature map D, and concatenating feature map A, feature map B, feature map C and feature map D and inputting them into a convolution-batch normalization-activation function combination layer to obtain the output of the spatial pyramid pooling module.

[0040] The spatial-channel conversion hybrid attention module improves the accuracy and efficiency of feature extraction by performing conversion, mixing and attention in the spatial and channel dimensions. It achieves adaptive weight allocation and redundant information filtering in the multi-dimensional feature space through fine feature mixing. Figure 4 The processing flow chart of the original feature map provided in the embodiment of the present application in the space-channel mixed attention module is as follows: Figure 4 As shown, in the spatial-channel mixed attention module, the processing of the original feature map input into the spatial-channel mixed attention module includes: S231, performing group normalization on the original feature map.

[0041] The input feature map X of size N×C×H×W is group normalized; the group normalization process is as follows:

[0042] Among them, μ and σ are the mean and standard deviation of feature X, ε is a very small constant added to ensure the stability of division, and γ and β are trainable variables.

[0043] S232, obtaining a normalized scaling factor based on the trainable parameters generated in the group normalization process.

[0044] The trainable parameter γ in the group normalization process is used as a method to measure the spatial pixel variance of each batch and channel, and the normalized scaling factor W is obtained by the scaling factor normalization operation. γ , the normalization process is as follows:

[0045] S233, mapping the scaling factor and obtaining a first attention weight and a second attention weight through a gating unit.

[0046] The weight value re-weighted by Wγ is mapped to the range of 0 to 1 through the Sigmoid function and passed through the gating unit, where the gating unit obtains two sets of attention weights Z1 and Z2 through a trainable threshold. The process of obtaining the entire attention weight is as follows:

[0047] Among them, Gate represents the gate operation, which sets the weight above the trainable threshold as the attention weight Z 1 , the weight below the threshold is set as the attention weight Z 2 .

[0048] S234, multiplying the first attention weight and the second attention weight by the original feature map respectively to obtain a first weighted feature map and a second weighted feature map.

[0049] Specifically, the attention weights are multiplied by the original input features to obtain the first weighted feature maps X of the two branches. 1 and the second weighted feature map X 2 .

[0050] S235 , pooling the first weighted feature map and the second weighted feature map respectively to obtain a first space-channel information descriptor and a second space-channel information descriptor.

[0051] Perform a conversion mixing operation: perform pooling operations on the first weighted feature map and the second weighted feature map respectively to obtain a first space-channel information descriptor G1 and a second space-channel information descriptor G2. The operation process is as follows:

[0052] S236 , calculating a first feature weight vector and a second feature weight vector based on the first space-channel information descriptor and the second space-channel information descriptor.

[0053] G1 and G2 are passed through the SoftMax function to obtain the first feature weight vector 𝛿1 and the second feature weight vector 𝛿2. The specific process is as follows:

[0054] S237, obtain the output of the spatial-channel hybrid attention module based on the first feature weight vector, the second feature weight vector, the first weighted feature map and the second weighted feature map.

[0055] The output is obtained by channel merging based on the feature weight vector. The specific process is:

[0056] The spatial-channel conversion hybrid attention module extracts features from the channel and space dimensions through the network, and assigns different weights to these features to achieve a sequential hybrid conversion process. This module dynamically adjusts the feature weights in both the channel and space dimensions, effectively enhancing the network's expressiveness, enabling it to suppress redundant information in both the channel and space dimensions. The lightweight cross-stage module enhances the detection accuracy and speed of the positional relationship between the root and the neural canal of the mandibular third molar by adding a lightweight convolution-batch normalization-activation function combination layer to maintain the overall low computational complexity and parameter amount of the network. In the test phase, the newly taken curved surface layer image is input into the trained spatial-channel conversion hybrid attention network. Only one scan of the entire curved surface layer image is required to quickly and automatically identify the positional relationship between the root and the neural canal in the image.

[0057] Please continue reading Figure 2,The feature fusion network includes a convolution-batch normalization-activation function combination layer, an upsampling module, a splicing module and a second type of lightweight cross-stage module.

[0058] Inputting multiple sub-feature maps into a feature fusion network for processing to obtain multiple output feature maps, including: inputting the third sub-feature map into a convolution-batch normalization-activation function combination layer to obtain a fourth sub-feature map; upsampling the fourth sub-feature map and splicing it with the second sub-feature map to obtain an eighth processed feature map, and sequentially inputting the eighth processed feature map into a second type lightweight cross-stage module and a convolution-batch normalization-activation function combination layer to obtain a fifth sub-feature map; upsampling the fifth sub-feature map and splicing it with the first sub-feature map and then passing it through a second type lightweight cross-stage module to obtain a first output feature map; passing the first output feature map through a convolution-batch normalization-activation function combination layer and splicing it with the fourth sub-feature map, and inputting the spliced ​​features into a second type lightweight cross-stage module to obtain a second output feature map; passing the second output feature map through a convolution-batch normalization-activation function combination layer and splicing it with the fourth sub-feature map, and inputting the spliced ​​features into a second type lightweight cross-stage module to obtain a third output feature map.

[0059] The prediction network includes multiple convolution-batch normalization-activation function combination layers. In step S2, multiple output feature maps are input into the prediction network to obtain the predicted bounding box and target confidence, including: The first output feature map, the second output feature map and the third output feature map are respectively input into different convolution-batch normalization-activation function combination layers to obtain the first multi-scale feature map, the second multi-scale feature map and the third multi-scale feature map; prediction is performed based on the first multi-scale feature map, the second multi-scale feature map and the third multi-scale feature map to obtain the bounding box and the target confidence.

[0060] The bounding box contains the specific positions of the molar root and the neural tube (expressed by the horizontal axis coordinates, vertical axis coordinates, and width and height of the bounding box center at different scales), as well as the category number of whether the molar root is in contact with the neural tube. Figure 5 The bounding box images provided in the embodiments of the present application, the rectangular box in each image is the area where the position relationship between the tooth root and the neural tube in the curved tomographic image is detected using the method of the present invention, that is, the bounding box of the area where the position relationship between the tooth root and the neural tube is predicted by the method of the present invention.

[0061] The first number above the bounding box indicates whether the tooth root and the nerve tube are in contact in the curved tomographic image detected by the method of the present invention. If the number is equal to 1, it indicates that the tooth root and the nerve tube are in contact in the curved tomographic image detected by the method of the present invention; if the number is equal to 0, it indicates that the tooth root and the nerve tube are not in contact in the curved tomographic image detected by the method of the present invention. The second number above the bounding box in each figure indicates the confidence level of whether the tooth root and the nerve tube are in contact or not in contact in the curved tomographic image detected by the method of the present invention.

[0062] The above describes the application process of the detection model. Figure 6 A flowchart of the training process of the detection model provided in the embodiment of the present application, such as Figure 6 The training process of the detection model shown is as follows: (1) Data collection and processing.

[0063] The surface tomographic images and the corresponding three-dimensional cone-beam CT images were collected, and the root position annotation information of the mandibular third molar was extracted from the surface tomographic images, and the category annotation information of the position relationship between the root and the neural canal was extracted from the three-dimensional cone-beam CT images. The surface tomographic images were data enhanced to generate diversified training data to improve the generalization ability of the model. The processed image data were integrated to form a training set for model training. The training set used here includes surface tomographic images and three-dimensional cone-beam CT images of 984 research cases.

[0064] Data collection and processing includes the following three parts: a. Obtain the position annotation information of tooth root and nerve canal from the curved surface tomographic image. The specific steps are as follows: When annotating, the tooth root and nerve canal part of the curved surface tomographic image is taken as the region of interest, and R is recorded. i is the minimum area rectangular frame containing the i-th region of interest of the curved tomographic image, and all R i The center point coordinates (x i ,y i ) and width and height (w i ,h i ) as positioning annotation information. Among them, both the tooth root and the nerve tube contact and non-contact need to be included, for example, the tooth root and the nerve tube contact 851 cases, and the tooth root and the nerve tube do not contact 678 cases.

[0065] b. Obtain the category annotation information of the positional relationship between the tooth root and the neural tube from the three-dimensional cone beam CT image. The specific steps are: observe the relative position of the tooth root and the neural tube from the cross-section, sagittal and coronal planes of the three-dimensional cone beam CT image, and obtain the category annotation information of the positional relationship between the tooth root and the neural tube. If the tooth root does not contact the neural tube, it is marked as negative, and if the tooth root contacts the neural tube, it is marked as positive.

[0066] c. Use data enhancement technology on the curved tomographic images used for training. The specific steps are as follows: By using various processing methods including random cropping, scaling, rotation, color adjustment, affine transformation, etc. on the curved tomographic images, the model can better adapt to different brightness, angles and background changes. Mosaic data enhancement technology is used to stitch multiple curved tomographic images into one for training, further enriching the diversity of training data and improving the robustness and performance of the detection method.

[0067] (2) Set the initial values ​​of the detection model hyperparameters, the initial learning rate is set to 0.001, the optimizer is SGD, and 300 rounds of hyperparameter evolution are used to train the model for 100 epochs each round to obtain the optimal hyperparameter combination. Then use the optimal hyperparameter combination to train the model for 600 epochs to obtain the training weights.

[0068] The model training adopts a multi-task learning strategy and optimizes the rectangular box loss at the same time. l box , confidence loss l obj and classification loss l cls The rectangular box loss function uses the Complete Intersection OverUnion (CIOU) loss, confidence loss l obj and classification loss l cls Using Binary CrossEntropy (BCE) loss, the above loss function expression is as follows:

[0069]

[0070]

[0071] in, p The distance between the center points of box A and box B, c is the diagonal length of the minimum enclosing rectangle of box A and box B, v is the similarity of the aspect ratio of frame A and frame B, a for v Impact factor;

[0072] in, N is the number of samples in a mini-batch, y ∈[0,1] represents the input label data, ∈[0,1] is a sample j Confidence of the classification; The total loss function is the weighted sum of the above three losses to balance the learning of positioning and classification tasks, and the parameters of the detection model are adjusted based on the total loss function.

[0073] (3) Repeat step (2) as the training phase of the detection model to obtain the final model weights.

[0074] (4) Use the detection model trained in step (3) to test the newly acquired curved surface layer images. Input the curved surface layer images to be tested into the detection model. The network model outputs: the horizontal axis coordinate of the center of the bounding box containing the position of the molar root and the neural tube, the vertical axis coordinate, and the width and height of the bounding box. These values ​​are all relative to the image width and height ratio (ranging from 0 to 1); the category number of whether the molar root is in contact with the neural tube and the corresponding confidence. The final detection results are then screened by the non-maximum suppression (NMS) algorithm, and the position relationship category of the root and the neural tube is determined according to the preset threshold.

[0075] This application inputs the curved surface tomographic image into a trained test model. During the test, only one scan of the entire curved surface tomographic image is required to quickly and automatically identify the positional relationship between the molar root and the neural tube in the image. Secondly, the present invention utilizes a space-channel conversion hybrid attention module and a lightweight cross-stage module to suppress redundant features in the channel and space dimensions respectively while reducing network complexity and parameter quantity. The space-channel conversion hybrid attention improves the network's sensitivity to different feature channels, significantly improving the recognition accuracy of the model.

[0076] According to the method described in the above embodiment, this embodiment will be further described from the perspective of a detection device for the position relationship between the molar root and the neural tube. The detection device for the position relationship between the molar root and the neural tube can be implemented as an independent entity or integrated in an electronic device. The electronic device can be a terminal, a server or other device, wherein the terminal can include a tablet computer, a laptop computer, a personal computer (PC), a micro processing box, or other devices.

[0077] See also Figure 7 , Figure 7 The present invention specifically describes a device for detecting the position relationship between a molar root and a neural tube provided in an embodiment of the present invention, which is applied to an electronic device. The device for detecting the position relationship between a molar root and a neural tube may include: An acquisition module, used for acquiring curved body layer images; A detection module is used to input the curved body layer image into the trained detection model to obtain the detection result; wherein the detection model includes a feature extraction network, a feature fusion network and a prediction network, and the feature extraction network includes a lightweight cross-stage module and a space-channel hybrid attention module; The curved body layer image is input into the trained detection model to obtain the detection results, including: inputting the curved body layer image into the feature extraction network, obtaining multiple sub-feature maps after processing through the lightweight cross-stage module and the space-channel hybrid attention module, inputting the multiple sub-feature maps into the feature fusion network for processing to obtain multiple output feature maps, and inputting the multiple output feature maps into the prediction network to obtain the predicted bounding box and target confidence.

[0078] During specific implementation, the above modules and / or units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above modules and / or units can refer to the previous method embodiments. The specific beneficial effects that can be achieved can also refer to the beneficial effects in the previous method embodiments, which will not be repeated here.

[0079] In addition, the embodiment of the present application also provides an electronic device, which may be a computer, a tablet computer, or other device. Figure 8 As shown, the electronic device 400 includes a processor 401 and a memory 402. The processor 401 is electrically connected to the memory 402.

[0080] The processor 401 is the control center of the electronic device 400. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device and processes data by running or loading applications stored in the memory 402 and calling data stored in the memory 402, thereby monitoring the electronic device as a whole.

[0081] In this embodiment, the processor 401 in the electronic device 400 loads instructions corresponding to the processes of one or more application programs into the memory 402 according to the following steps, and the processor 401 runs the application programs stored in the memory 402 to implement various functions: Acquiring curved tomographic images; Inputting the curved body layer image into a trained detection model to obtain a detection result; wherein the detection model includes a feature extraction network, a feature fusion network and a prediction network, and the feature extraction network includes a lightweight cross-stage module and a space-channel hybrid attention module; The step of inputting the curved volume layer image into the trained detection model to obtain the detection result includes: inputting the curved volume layer image into the feature extraction network, obtaining a plurality of sub-feature maps after processing by the lightweight cross-stage module and the spatial-channel hybrid attention module, inputting the plurality of sub-feature maps into the feature fusion network for processing to obtain a plurality of output feature maps, and inputting the plurality of output feature maps into the prediction network to obtain a predicted bounding box and target confidence.

[0082] The electronic device can implement the steps in any embodiment of the method for detecting the positional relationship between the molar root and the neural tube provided in the embodiments of the present application. Therefore, it can achieve the beneficial effects that can be achieved by any method for detecting the positional relationship between the molar root and the neural tube provided in the embodiments of the present application. Please see the previous embodiments for details and will not be repeated here.

[0083] Fig. 9 The specific structural block diagram of the electronic device provided in the embodiment of the present invention is shown, and the electronic device can be used to implement the detection method of the position relationship between the molar root and the nerve tube provided in the above embodiment. The electronic device 500 can be a terminal, a server, etc., wherein the terminal can include a tablet computer, a notebook computer, a personal computer (PC, Personal Computer), a micro processing box, or other devices.

[0084] The RF circuit 510 is used to receive and send electromagnetic waves, realize the mutual conversion between electromagnetic waves and electrical signals, and thus communicate with a communication network or other devices. The RF circuit 510 may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, user identity module (SIM) cards, memories, etc. The RF circuit 510 can communicate with various networks such as the Internet, corporate intranets, wireless networks, or communicate with other devices through wireless networks. The above-mentioned wireless networks may include cellular telephone networks, wireless local area networks, or metropolitan area networks. The above-mentioned wireless networks may use various communication standards, protocols and technologies, including but not limited to Global System for Mobile Communication (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), Code Division Access (CDMA), Time Division Multiple Access (TDMA), Wireless Fidelity (Wi-Fi) (such as Institute of Electrical and Electronics Engineers standards IEEE 802.11a, IEEE 802.11b, IEEE802.11g and / or IEEE802.11n), Voice over Internet Protocol (VoIP), Worldwide Interoperability for Microwave Access (Wi-Max), other protocols for email, instant messaging and short messages, and any other suitable communication protocols, even those that have not yet been developed.

[0085] The memory 520 can be used to store software programs and modules, such as the corresponding program instructions / modules in the above embodiments. The processor 580 executes various functional applications and data processing by running the software programs and modules stored in the memory 520, that is, realizing the functions of taking pictures with the front camera, processing the captured images, and switching the display color of the display content on the display screen. The memory 520 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 520 may further include a memory remotely arranged relative to the processor 580, and these remote memories may be connected to the electronic device 500 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0086] The input unit 530 may be used to receive input digital or character information, and generate a keyboard and a mouse related to user settings and function control. The display unit 540 may be used to display information input by the user or information provided to the user and various graphical user interfaces, which may be composed of graphics, text, icons, videos, and any combination thereof. The display unit 540 may include a display panel 541, which may be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), or the like.

[0087] The audio circuit 560, the speaker 561, and the microphone 562 can provide an audio interface between the user and the electronic device 500. The audio circuit 560 can transmit the electrical signal converted from the received audio data to the speaker 561, which is converted into a sound signal for output; on the other hand, the microphone 562 converts the collected sound signal into an electrical signal, which is received by the audio circuit 560 and converted into audio data, and then the audio data is output to the processor 580 for processing, and then sent to another terminal through the RF circuit 510, or the audio data is output to the memory 520 for further processing. The audio circuit 560 may also include an earplug jack to provide communication between an external headset and the electronic device 500.

[0088] The electronic device 500 can help the user receive requests, send information, etc. through the transmission module 570 (such as a Wi-Fi module), which provides the user with wireless broadband Internet access. Although the transmission module 570 is shown in the figure, it can be understood that it is not a necessary component of the electronic device 500 and can be omitted as needed without changing the essence of the invention.

[0089] The processor 580 is the control center of the electronic device 500. It uses various interfaces and lines to connect various parts of the entire mobile phone. By running or executing software programs and / or modules stored in the memory 520, and calling data stored in the memory 520, it executes various functions of the electronic device 500 and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 580 may include one or more processing cores; in some embodiments, the processor 580 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communications. It can be understood that the above-mentioned modem processor may not be integrated into the processor 580.

[0090] The electronic device 500 also includes a power supply 590 (such as a battery) for supplying power to various components. In some embodiments, the power supply can be logically connected to the processor 580 through a power management system, so that the power management system can manage charging, discharging, and power consumption management. The power supply 590 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0091] Although not shown, the electronic device 500 also includes a camera (such as a front camera, a rear camera), a Bluetooth module, etc., which will not be described in detail here. Specifically in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal also includes a memory, and one or more programs, wherein one or more programs are stored in the memory, and are configured to be executed by one or more processors. One or more programs include instructions for performing the following operations: Acquiring curved tomographic images; Inputting the curved body layer image into a trained detection model to obtain a detection result; wherein the detection model includes a feature extraction network, a feature fusion network and a prediction network, and the feature extraction network includes a lightweight cross-stage module and a space-channel hybrid attention module; The step of inputting the curved volume layer image into the trained detection model to obtain the detection result includes: inputting the curved volume layer image into the feature extraction network, obtaining multiple sub-feature maps after processing by the lightweight cross-stage module and the spatial-channel hybrid attention module, inputting the multiple sub-feature maps into the feature fusion network for processing to obtain multiple output feature maps, and inputting the multiple output feature maps into the prediction network to obtain the predicted bounding box and target confidence. In specific implementation, the above modules can be implemented as independent entities, or they can be arbitrarily combined and implemented as the same or several entities. The specific implementation of the above modules can refer to the previous method embodiments, which will not be repeated here.

[0092] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by controlling related hardware through instructions, and the instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. To this end, an embodiment of the present invention provides a storage medium, which stores a plurality of instructions, and the instructions can be loaded by a processor to execute the steps of any embodiment of the method for detecting the positional relationship between the molar root and the neural tube provided in the embodiment of the present invention.

[0093] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0094] Since the instructions stored in the storage medium can execute the steps in any embodiment of the method for detecting the positional relationship between the molar root and the neural tube provided in the embodiments of the present invention, the beneficial effects that can be achieved by any method for detecting the positional relationship between the molar root and the neural tube provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0095] The above is a detailed introduction to the detection method, device, storage medium and electronic device for the positional relationship between the molar root and the neural tube provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for detecting the positional relationship between a molar root and a neural tube, characterized in that: The method comprises: Acquiring curved tomographic images; Inputting the curved body layer image into a trained detection model to obtain a detection result; wherein the detection model includes a feature extraction network, a feature fusion network and a prediction network, and the feature extraction network includes a lightweight cross-stage module and a space-channel hybrid attention module; The step of inputting the curved volume layer image into the trained detection model to obtain the detection result includes: inputting the curved volume layer image into the feature extraction network, obtaining a plurality of sub-feature maps after processing by the lightweight cross-stage module and the spatial-channel hybrid attention module, inputting the plurality of sub-feature maps into the feature fusion network for processing to obtain a plurality of output feature maps, and inputting the plurality of output feature maps into the prediction network to obtain a predicted bounding box and target confidence.

2. The method for detecting the positional relationship between molar roots and neural tubes according to claim 1, characterized in that: The feature extraction network includes a convolution-batch normalization-activation function combination layer, a first type of lightweight cross-stage module, a second type of lightweight cross-stage module, a space-channel conversion hybrid attention module, and a space pyramid pooling module; The curved body layer image is input into the feature extraction network, and processed by the lightweight cross-stage module and the spatial-channel hybrid attention module to obtain multiple sub-feature maps, including: Input the curved volume layer image into the convolution-batch normalization-activation function combination layer for two feature extractions, and then sequentially input the image into a first type of lightweight cross-stage module, a space-channel conversion hybrid attention module, and a second type of lightweight cross-stage module, and input the output of the second type of lightweight cross-stage module into a space-channel conversion hybrid attention module again to obtain a first sub-feature map; Inputting the first sub-characteristic map into a first-type lightweight cross-stage module and a second-type lightweight cross-stage module in sequence to obtain a second sub-characteristic map; The second sub-feature map is sequentially input into a convolution-batch normalization-activation function combination layer, a first-class lightweight cross-stage module, a spatial-channel conversion hybrid attention module and a spatial pyramid pooling module to obtain the third sub-feature map.

3. The method for detecting the positional relationship between molar roots and neural tubes according to claim 2, characterized in that: In the spatial-channel hybrid attention module, processing of the original feature map input into the spatial-channel hybrid attention module includes: Performing group normalization on the original feature map; Obtaining a normalized scaling factor based on the trainable parameters generated during the group normalization process; Mapping the scaling factor and obtaining a first attention weight and a second attention weight through a gating unit; Multiplying the first attention weight and the second attention weight by the original feature map respectively to obtain a first weighted feature map and a second weighted feature map; Pooling the first weighted feature map and the second weighted feature map respectively to obtain a first space-channel information descriptor and a second space-channel information descriptor; Calculating a first feature weight vector and a second feature weight vector based on the first space-channel information descriptor and the second space-channel information descriptor; The output of the spatial-channel hybrid attention module is obtained based on the first feature weight vector, the second feature weight vector, the first weighted feature map and the second weighted feature map.

4. The method for detecting the positional relationship between molar roots and neural tubes according to claim 2, characterized in that: The first type of lightweight cross-stage modules includes a convolution-batch normalization-activation function combination layer and a lightweight convolution-batch normalization-activation function combination layer; In the first type of lightweight cross-stage module, processing the feature map input to the first type of lightweight cross-stage module includes: Inputting the feature map input to the first type of lightweight cross-stage module into a convolution-batch normalization-activation function combination layer to obtain a first processed feature map; Inputting the first processed feature map into two lightweight convolution-batch normalization-activation function combination layers in sequence to obtain a second processed feature map; Inputting the first processed feature map into a convolution-batch normalization-activation function combination layer for processing to obtain a third processed feature map; Performing an element-by-element addition operation on the features in the first processing feature map and the features in the second processing feature map, and concatenating the result of the element-by-element addition with the third processing feature map; The result of the splicing operation is input into a convolution-batch normalization-activation function combination layer to obtain the output of the first type of lightweight cross-stage module.

5. The method for detecting the positional relationship between molar roots and neural tubes according to claim 2, characterized in that: The second type of lightweight cross-stage modules includes a convolution-batch normalization-activation function combination layer, a lightweight convolution-batch normalization-activation function combination layer and a depth-separable convolution layer; In the second type of lightweight cross-stage module, the processing of the feature map input to the second type of lightweight cross-stage module includes: Inputting the feature map input to the second type of lightweight cross-stage module into a convolution-batch normalization-activation function combination layer to obtain a fourth processed feature map; Inputting the fourth processed feature map into a lightweight convolution-batch normalization-activation function combination layer, a depth-separable convolution and another lightweight convolution-batch normalization-activation function combination layer in sequence to obtain a fifth processed feature map; Inputting the fourth processed feature map into a lightweight convolution-batch normalization-activation function combination layer and a convolution-batch normalization-activation function combination layer in sequence to obtain a sixth processed feature map; Inputting the fourth processed feature map into a convolution-batch normalization-activation function combination layer for processing to obtain a seventh processed feature map; Performing an element-by-element addition operation on the features in the fifth processing feature map and the features in the sixth processing feature map, and performing a splicing operation on the result of the element-by-element addition and the seventh processing feature map; The result of the concatenation operation is input into a convolution-batch normalization-activation function combination layer to obtain the output of the second type of lightweight cross-stage module.

6. The method for detecting the positional relationship between molar roots and neural tubes according to claim 2, characterized in that: The feature fusion network includes a convolution-batch normalization-activation function combination layer, an upsampling module, a splicing module and a second type of lightweight cross-stage module; The inputting the multiple sub-feature maps into the feature fusion network for processing to obtain multiple output feature maps includes: Inputting the third sub-feature map into a convolution-batch normalization-activation function combination layer to obtain a fourth sub-feature map; Upsampling the fourth sub-feature map and concatenating it with the second sub-feature map to obtain an eighth processed feature map, and sequentially inputting the eighth processed feature map into a second type of lightweight cross-stage module and a convolution-batch normalization-activation function combination layer to obtain a fifth sub-feature map; The fifth sub-feature map is upsampled, concatenated with the first sub-feature map, and then passed through a second type lightweight cross-stage module to obtain a first output feature map; Passing the first output feature map through a convolution-batch normalization-activation function combination layer, and then concatenating it with the fourth sub-feature map, and inputting the concatenated features into a second type of lightweight cross-stage module to obtain a second output feature map; The second output feature map is passed through a convolution-batch normalization-activation function combination layer and then concatenated with the fourth sub-feature map, and the concatenated features are input into a second-type lightweight cross-stage module to obtain a third output feature map.

7. The method for detecting the positional relationship between molar roots and neural tubes according to claim 6, characterized in that: The inputting the plurality of output feature maps into the prediction network to obtain a predicted bounding box and target confidence comprises: Inputting the first output feature map, the second output feature map, and the third output feature map into different convolution-batch normalization-activation function combination layers respectively to obtain a first multi-scale feature map, a second multi-scale feature map, and a third multi-scale feature map; Prediction is performed based on the first multi-scale feature map, the second multi-scale feature map, and the third multi-scale feature map to obtain a bounding box and an object confidence.

8. A device for detecting the positional relationship between molar roots and neural tubes, characterized in that: include: An acquisition module, used for acquiring curved body layer images; A detection module, used for inputting the curved surface body layer image into a trained detection model to obtain a detection result; wherein the detection model includes a feature extraction network, a feature fusion network and a prediction network, and the feature extraction network includes a lightweight cross-stage module and a space-channel hybrid attention module; The step of inputting the curved volume layer image into the trained detection model to obtain the detection result includes: inputting the curved volume layer image into the feature extraction network, obtaining a plurality of sub-feature maps after processing by the lightweight cross-stage module and the spatial-channel hybrid attention module, inputting the plurality of sub-feature maps into the feature fusion network for processing to obtain a plurality of output feature maps, and inputting the plurality of output feature maps into the prediction network to obtain a predicted bounding box and target confidence.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the method for detecting the positional relationship between a molar root and a neural canal as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes a processor and a memory, the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used to execute the steps in the method for detecting the position relationship between the molar root and the neural tube as described in any one of claims 1 to 7.

Images