An intelligent sensing and recognition method and system for the contact strength of tooth proximal surfaces

Abstract

The present invention discloses an intelligent sensing and recognition method and system for the contact strength of tooth adjacent surfaces, belonging to the technical field of oral health detection. The present invention uses a sound signal classification model to identify whether the contact strength of tooth adjacent surfaces is normal. The sound signal classification model includes a pointwise depth convolutional network, two residual networks connected in sequence, and also includes a lightweight attention network placed between the two residual networks. At the same time, the second 3*3 convolutional block in the BasicBlock of the residual network is replaced with a split branch block. The present invention can effectively learn the ripple feature information of the spectrogram with lower parameters, promote the extraction of effective information in the downsampling process, and effectively fuse the feature information generated by different filters, improve the feature expression ability and the model accuracy. Finally, the model can be used to detect and determine whether the adjacency strength is normal.

Description

Technical Field

[0001] The present invention relates to the field of oral health detection technology, and more particularly to a method and system for intelligently sensing and identifying tooth proximal contact strength. Background Art

[0002] The interproximal contact zone refers to the area where the teeth are in close contact with adjacent teeth in the same dental arch. A well-contoured, correctly positioned, and firm interproximal contact zone is essential to maintain the integrity of the dental arch and the health of the supporting structure. In clinical practice, indicators such as interproximal strength, area, position, and shape are often used to quantitatively study and evaluate the interproximal zone. Low interproximal strength may lead to common oral diseases such as food impaction, caries, and periodontal disease; high interproximal strength may cause problems such as poor tooth movement and periodontal tissue damage. Similarly, clinical diseases such as food impaction or caries may also destroy the tooth interproximal area, resulting in abnormal interproximal strength, thereby aggravating food impaction or tooth caries.

[0003] At present, there are mainly the following methods for detecting adjacency strength:

[0004] ① Dental floss: It is the most commonly used method for checking the interproximal strength in clinical practice. It has high repeatability in determining whether there is interproximal contact and is a qualitative test.

[0005] ② Different thicknesses of occlusal paper or metal film: The thickness of the occlusal paper or metal film that initially has resistance entering the proximal contact area of ​​adjacent teeth is equated with the proximal strength, which is a semi-quantitative test.

[0006] ③ Tooth gap measuring devices (feeler gauges) or vernier calipers of different thicknesses: The size that can smoothly enter the adjacent area is defined as the adjacent strength. It is mainly aimed at adjacent areas with gaps and is a semi-quantitative test.

[0007] ④ Thin metal strip: The maximum friction generated by removing the metal strip located in the adjacent area from the buccal and lingual direction or from The maximum friction force generated by the metal strip that extends outward into the adjacent area is the adjacent strength, which is a quantitative test.

[0008] ⑤ Charge-coupled device or stereoscopic photography: The instrument is complex, the measurement is extremely cumbersome, the clinical application is limited, and it is a quantitative test.

[0009] Among the above methods, the first three methods are insufficient in accuracy, effectiveness, and repeatability, and cannot accurately and quantitatively analyze the adjacency strength; method ④ passively separates the adjacency area during measurement, which is not in line with physiological conditions, and the detection equipment is complex; the equipment in method ⑤ is cumbersome, and the clinical application is obviously limited. Therefore, there is currently a lack of a portable, objective, and stable system method and device for detecting whether the adjacency strength is normal.

[0010] Therefore, it is an urgent problem for those skilled in the art to propose a portable intelligent sensing and recognition method and system that can objectively and stably detect the contact strength of tooth proximal surfaces. Summary of the Invention

[0011] In view of this, the present invention provides an intelligent sensing and recognition method and system for tooth proximal contact strength, constructs a sound signal classification model, can effectively learn the ripple feature information of the spectrogram with low parameters, promotes the extraction of effective information in the downsampling process, and effectively fuses the feature information generated from different filters, improves the feature expression ability and the model accuracy rate, and finally uses this model to detect and determine whether the adjacent strength is normal.

[0012] In order to achieve the above object, the present invention adopts the following technical solutions:

[0013] On the one hand, the present invention discloses an intelligent sensing and recognition method for tooth proximal contact strength, including the following steps:

[0014] Obtain the sound data of tooth proximal contact strength, and process the sound data to obtain a spectrogram;

[0015] Construct a sound signal classification model; the sound signal classification model includes a pointwise depth convolutional network, two residual networks connected in sequence, and also includes a lightweight attention network, the lightweight attention network is placed between the two residual networks, wherein, the second 3*3 convolutional block in the BasicBlock of the residual network is replaced with a split branch block;

[0016] Input the spectrogram into the trained sound signal classification model, and output the recognition result of tooth proximal contact strength.

[0017] Preferably, processing the sound data to obtain a spectrogram includes:

[0018] Perform pre-emphasis on the sound data to obtain a pre-emphasized audio waveform;

[0019] Perform frame addition, windowing and short-time Fourier transform on the pre-emphasized audio waveform to obtain a spectral matrix;

[0020] Convert the spectral matrix into the spectrogram.

[0021] Preferably, the lightweight attention network includes a channel attention network and a spatial attention network;

[0022] Group the feature maps along the channel dimension, and each group respectively generates a channel attention map and a spatial attention map through the channel attention network and the spatial attention network;

[0023] Merge the channel attention map and the spatial attention map and perform a channel shuffle operation before inputting into the second residual network.

[0024] Preferably, the channel attention network is improved by global average pooling, scaling, and activation functions, with the formula as follows:

[0025]

[0026] X' i1 = σ(W 1 s + b 1 ).X i1 ;

[0027] where X' i1 is the output of the channel attention network. The feature map X is divided into G groups, X ∈ R C*W*H , and C, H, and W represent the channel encoding, width, and height of the feature map respectively. X i1 is the first branch of the i-th group of the feature map X. f gp (·) is the global pooling operation, s is the feature map generated after the global pooling operation, W 1 is the first weight parameter, b 1 is the first bias parameter, and σ represents the sigmoid activation function.

[0028] Preferably, the calculation formula of the spatial attention network is as follows:

[0029] X' i2 = σ(W 2 .GN(X i2 + b 2 ).X i2 );

[0030] where X' i2 is the output of the spatial attention network. The feature map X is divided into G groups, X ∈ R C*W*H , and C, H, and W represent the channel encoding, width, and height of the feature map respectively. X i2 is the second branch of the i-th group of the feature map X. W 2 is the second weight parameter, b 2 is the second bias parameter, and σ represents the sigmoid activation function.

[0031] Preferably, the split branch block includes four branches as follows:

[0032] 1x1 convolution + BN layer;

[0033] 1x1 convolution + BN layer + KxK convolution + BN layer;

[0034] 1x1 convolution + BN layer + average pooling + BN layer;

[0035] 1x1 convolution + BN layer;

[0036] After the results of the four branches are added together, the output of the separation branch block is obtained through an activation function.

[0037] Preferably, the sound signal classification model further includes a global average pooling layer, a fully connected layer, and a softmax function connected in sequence. The global average pooling layer is connected to the second residual network.

[0038] On the other hand, the present invention also discloses an intelligent sensing and recognition system for the occlusal contact strength of teeth, which is used to implement the above-mentioned intelligent sensing and recognition method for the occlusal contact strength of teeth. The system includes:

[0039] A data acquisition module, which is used to acquire the sound data of the occlusal contact strength of teeth and process the sound data to obtain a spectrogram;

[0040] A model construction module, which is used to construct a sound signal classification model; the sound signal classification model includes a pointwise depth convolution network, two residual networks connected in sequence, and also includes a lightweight attention network. The lightweight attention network is placed between the two residual networks. Among them, the second 3*3 convolution block in the BasicBlock of the residual network is replaced by a separation branch block;

[0041] An output module, which is used to input the spectrogram into the trained sound signal classification model and output the recognition result of the occlusal contact strength of teeth.

[0042] It can be seen from the above technical solutions that the present invention discloses an intelligent sensing and recognition method and system for the occlusal contact strength of teeth, which has the following beneficial effects compared with the prior art:

[0043] 1. The present invention stacks two layers of residual networks (DBNet) as the backbone network, which can reduce the network training time, reduce the number of parameters, and achieve better training results within a shorter training time.

[0044] 2. The present invention proposes pointwise depth convolution (PDNet) to replace the downsampling module of the backbone network, effectively extracts feature information, improves the network downsampling ability, and further improves the network performance.

[0045] 3. Replace the 3*3 convolution module in the residual network basicblock with a separation branch block (DiverseBranch Block, DBB) to introduce different receptive fields, and significantly improve the network recognition performance under different complexity multi-branch structures.

[0046] 4. A lightweight attention network (Shuffle Attention, SA) is embedded between two residual networks to improve the expression ability of effective information between residual networks and enhance the recognition and classification performance of tooth adjacent strength. Description of the Drawings

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0048] Figure 1 It is a flowchart of the method of the present invention;

[0049] Figure 2 It is a schematic structural diagram of the sound signal classification model of the present invention;

[0050] Figure 3 It is a schematic structural diagram of the pointwise depth convolution network of the present invention;

[0051] Figure 4 It is a schematic structural diagram of the lightweight attention network of the present invention;

[0052] Figure 5 It is a schematic structural diagram of the separation branch block of the present invention;

[0053] Figure 6(a) is the Mel spectrogram with a normal recognition result, and Figure 6(b) is the Mel spectrogram with an abnormal recognition result. Detailed Embodiments

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0055] On the one hand, the embodiments of the present invention disclose an intelligent sensing and recognition method for tooth proximal contact strength, referring to Figure 1 , the method includes the following steps:

[0056] S1. Obtain the sound data of tooth proximal contact strength and process the sound data to obtain a spectrogram, including:

[0057] Perform pre-emphasis on the sound data to obtain a pre-emphasized audio waveform;

[0058] Frame the pre-emphasized audio waveform and apply a window function, then perform a short-time Fourier transform to obtain a spectral matrix;

[0059] Convert the spectral matrix into a spectrogram.

[0060] Here, the sound data is the audio signal of the tooth proximal contact intensity sound generated by dental floss, which varies greatly over time. The frequency-domain features contain the information of the time-domain signal and can simultaneously reflect the frequency characteristics of the audio. If the audio signal is directly subjected to a short-time Fourier transform (STFT), it will cause distortion of some audio waveforms. Therefore, pre-emphasis is performed on the audio before STFT. Pre-emphasis can not only reduce the influence of some amplitude-frequency distortion and frequency response changes, but also reduce the low-frequency noise in the audio signal. Frame the pre-emphasized audio waveform and perform STFT to obtain a spectral matrix. The process is as follows:

[0061]

[0062] y(t) is the time-domain signal, x is the frequency-domain signal, w(t - r) represents the Hamming window with the center position at r, f is the frequency, and r is the frame length.

[0063] The Mel filter is designed according to the characteristics of the human ear to improve the absorption of frequencies in the lower part of the sound. In this embodiment, 128 Mel filters are set per second in the sound audio segment. The formula of the Mel filter is as follows:

[0064]

[0065] where f mel is the calculated Mel-scale frequency, and f is the normal Hertz frequency.

[0066] The Mel filter bank imitates the human ear to filter speech. M triangular filters are set within the frequency range of an audio segment. In this embodiment, it is set to 512. The filter layout is from dense to sparse; as the Hertz frequency increases, the filter width increases from small to large; there is 50% overlap between each filter to avoid information loss; these filters are shown as having equal width on the Mel scale. Finally, convert the output matrix into a spectrogram.

[0067] S2. Construct a sound signal classification model; the sound signal classification model includes a pointwise depth convolutional network (PDNet), two residual networks (DBBDB) connected in sequence, and also includes a lightweight attention network (Shuffle Attention, SA). The lightweight attention network is placed between the two residual networks. Among them, the second 3*3 convolutional block in the BasicBlock of the residual network is replaced with a split branch block (DBB). Refer toFigure 2 。

[0068] Specifically, the PDNet structure in this embodiment can refer to Figure 3 , Figure 3 In the first to third parts, it is a pointwise convolution process, and the subsequent part is a depthwise separable convolution process.

[0069] Specifically, the Mel spectrogram generated by feature extraction is passed through a pointwise convolution network to generate a feature map with 16 channels to increase the number of spectrogram channels. On the premise of dimensionality increase, it can also well integrate the information between channels separated by depth convolution and retain the effective feature information of the spectrogram. Then, the feature map is passed through a depthwise separable convolution network to generate a thumbnail feature map with 64 channels, reducing the number of model parameters on the premise of effectively improving network performance and alleviating the problem of parameter increase. Depthwise separable convolution divides standard convolution into depth convolution and pointwise convolution, mapping the feature channels and spatial dimensions simultaneously, effectively retaining the ripple information of the spectrogram and removing the environmental noise information.

[0070] During the process of the backbone network identifying the tooth adjacency strength sound samples, the interference of environmental sounds in the samples will cause certain misjudgments to the recognition results. To improve the inheritance of effective information between the two residual networks of the backbone network and reduce the interference brought by environmental sounds during the network recognition process, a lightweight attention network SA is embedded in the two residual networks of the backbone network in this embodiment. As Figure 4 shown, the lightweight attention network effectively combines two types of attention mechanisms using Shuffle units.

[0071] Specifically, SA first groups the channel dimensions into multiple sub-features, and then processes these sub-features in parallel. For each sub-feature, SA uses a Shuffle unit to describe the feature dependencies in the spatial and channel dimensions. Finally, all sub-features are aggregated, and the "channel shuffle" operator is used to achieve information communication between different sub-features.

[0072] Functional grouping: Given the input feature map X ∈ R C*W*H , where C, H, and W represent the channel encoding, width, and height of the feature map respectively. For the first time, the feature map X is divided into G groups along the channel dimension, that is Then each group is divided into two branches along the channel direction, One branch is the channel attention network, which uses the mutual relationship between channels to generate a channel attention map, and the other branch is the spatial attention network, which uses the mutual relationship between features to generate a spatial attention map. In this embodiment, G is set to 16.

[0073] The channel attention network is improved through a combination of global average pooling, scaling, and activation functions. The specific calculation formula is as follows:

[0074]

[0075] X' i1 = σ(f c (s)).X i1 = σ(w 1 s + b 1 ).X i1 ;

[0076] are two parameters that can be continuously trained through the network. σ represents the sigmoid activation function, f gp (·) is the global pooling operation, and s is the feature map generated after the global average pooling operation.

[0077] Spatial attention network: Spatial attention can be regarded as a complement to channel attention and uses group normalization (GN) operations.

[0078] The specific formula is as follows:

[0079] X' i2 = σ(W 2 ·GN(X i2 + b 2 ).X i2 );

[0080] are also two parameters that are continuously trained through the network. σ represents the sigmoid activation function.

[0081] Aggregation: After completing the learning of the two types of attention and recalibrating the features, it is necessary to splice and aggregate the two branches, in order to aggregate all sub-features and perform a channel shuffle operation.

[0082] In the process of identifying the sound perception samples of the adjacent surface contact intensity of teeth, in order to increase the expression ability of the model, different receptive fields are introduced. In the basicblock of the two residual networks in the backbone network, the second 3*3 Conv module is replaced with DBB, which can enrich the feature space by combining branches with different scales and complexities, including convolutional sequences, multi-scale convolutions, and average pooling, so as to enhance the representation ability of a single convolution. DBB adopts a complex "microstructure" during training while keeping the macrostructure unchanged. This "heterogeneity" can make the model have higher complexity and performance during training and can return to the original structure for inference.

[0083] Assume that the number of input channels is C, the number of output channels is D, and the size of C is K×K, then the Conv kernel is F ∈ R D×C×K×K, the optional deviation parameter is \(b\in\mathbb{R}\) D . For subsequent merging convenience, the deviation parameter is defined as \(REP(b)\in\mathbb{R}\) D×H×W , the convolution is defined as follows:

[0084] \(O = I\times F+REP(B)\);

[0085] The calculation method of the value at \((h, w)\) of the \(j\)-th output channel is:

[0086] \(O\) j，h，w \(=\sum\) c＝1 \(\sum\) u＝1 \(\sum\) v＝1 \(F\) j,c，，u,v \(X\) (c,h,w)u,v \(+b\) j ;

[0087] where \(X(c, h, w)\in\mathbb{R}\) K×K represents the sliding window. From the above formula, the linear properties of the convolution can be deduced, including homogeneity and additivity. A typical example of using DBB in this embodiment is as Figure 5 shown, using equal quantity combinations to enhance the traditional Conv. For the \(1\times1\) and \(K\times K\) branches, set the number of intermediate channels equal to the number of input channels, and initialize the \(1\times1\) Conv as the identity matrix, and other branches are also initialized in the traditional way. Add a Batch Normalization (BN) layer after each convolution to provide non-linearity during the training process, which is necessary for improving performance.

[0088] Specifically, from Figure 5 it can be seen that the DBB proposed in this embodiment includes four branches as follows:

[0089] \(1\times1\) convolution + BN layer;

[0090] \(1\times1\) convolution + BN layer + \(K\times K\) convolution + BN layer;

[0091] \(1\times1\) convolution + BN layer + average pooling + BN layer;

[0092] \(1\times1\) convolution + BN layer;

[0093] After adding the results of the four branches, the output of the separated branch block is obtained through the activation function.

[0094] According to Figure 2 , the sound signal classification model also includes a global average pooling layer, a fully connected layer, and a softmax function connected in sequence, and the global average pooling layer is connected to the second residual network.

[0095] S3. Input the spectrogram into the trained sound signal classification model, and output the recognition result of the occlusal contact intensity of the teeth, and the results are normal and abnormal.

[0096] In this embodiment, the intensity sounds of OralB dental floss passing through the adjacent areas of normal teeth and abnormal teeth (the unified tooth positions in this example are 36 and 37) are collected according to certain standards to establish a sample data set.

[0097] The collected samples of the normal and abnormal adjacent intensity data sets are converted into Mel spectrograms through Mel filters, and the results are shown in Figures 6(a) and 6(b): The difference between the normal and abnormal visualization results is relatively large. The normal (Figure 6(a)) has obvious ripple features, while the abnormal (Figure 6(b)) has relatively blurred ripples.

[0098] There is a certain amount of additive and multiplicative noise in the spectrogram. In this embodiment, an attention mechanism is added to the sound signal classification model to improve the ripple information of the spectrogram, and the recognition performance is effectively improved in the backbone network with relatively low parameters.

[0099] On the other hand, the present invention also discloses an intelligent sensing and recognition system for the adjacent surface contact intensity of teeth, which is used to implement the above-mentioned intelligent sensing and recognition method for the adjacent surface contact intensity of teeth. The system includes:

[0100] A data acquisition module, which is used to acquire the sound data of the adjacent surface contact intensity of teeth and process the sound data to obtain a spectrogram;

[0101] A model construction module, which is used to construct a sound signal classification model; the sound signal classification model includes a pointwise depth convolutional network, two residual networks connected in sequence, and also includes a lightweight attention network. The lightweight attention network is placed between the two residual networks. Among them, the second 3*3 convolutional block in the BasicBlock of the residual network is replaced with a split branch block;

[0102] An output module, which is used to input the spectrogram into the trained sound signal classification model and output the recognition result of the adjacent surface contact intensity of teeth.

[0103] In this specification, each embodiment is described in a progressive manner. The key points of each embodiment are the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0104] The foregoing description of the disclosed embodiments enables those skilled in the art to practice or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for intelligent perception and recognition of tooth interproximal contact strength, characterized in that: The following steps are involved: Acquiring sound data of tooth interproximal contact strength, and processing the sound data to obtain a frequency spectrum; Constructing a sound signal classification model; the sound signal classification model includes a point-by-point deep convolutional network and two residual networks connected in sequence, and also includes a lightweight attention network, wherein the lightweight attention network is placed between the two residual networks, wherein the second 3*3 convolutional block in the BasicBlock of the residual network is replaced with a separation branch block; The spectrum graph is input into a trained sound signal classification model to output a tooth proximal contact strength recognition result.

2. The method for intelligently sensing and identifying the contact strength of tooth interproximal surfaces according to claim 1, characterized in that: The sound data is processed to obtain a spectrum diagram, including: Pre-emphasize the sound data to obtain a pre-emphasized audio waveform; Performing frame windowing and short-time Fourier transform on the pre-emphasized audio waveform to obtain a spectrum matrix; The spectrum matrix is ​​converted into the spectrum map.

3. The method for intelligently sensing and identifying the contact strength of tooth proximal surfaces according to claim 1, characterized in that: The lightweight attention network includes a channel attention network and a spatial attention network; The feature maps are grouped along the channel dimension, and each group generates a channel attention map and a spatial attention map through the channel attention network and the spatial attention network respectively; The channel attention map and the spatial attention map are aggregated and a channel shuffle operation is performed before inputting into the second residual network.

4. The method for intelligently sensing and identifying the contact strength of tooth interproximal surfaces according to claim 3, characterized in that: The channel attention network is improved by global average pooling, scaling and activation functions, and the formula is as follows: X′ i1 =σ(W1s+b1).X i1 ; Among them, X' i1 is the output of the channel attention network, dividing the feature map X into G groups, X∈R C*W*H , C, H, W represent the channel encoding, width and height of the feature map respectively, X i1 is the first branch of the i-th group of feature graph X, f gp (·) is the global pooling operation, s is the feature map generated after the global pooling operation, W1 is the first weight parameter, b1 is the first bias parameter, and σ represents the sigmoid activation function.

5. The method for intelligently sensing and identifying the contact strength of tooth interproximal surfaces according to claim 3, characterized in that: The calculation formula of the spatial attention network is as follows: X′ i2 =σ(W2.GN(X i2 +b2).X i2 ); Among them, X' i2 is the output of the spatial attention network, dividing the feature map X into G groups, X∈R C*W*H , C, H, W represent the channel encoding, width and height of the feature map respectively, X i2 is the second branch of the i-th group of feature graph X, W2 is the second weight parameter, b2 is the second bias parameter, and σ represents the sigmoid activation function.

6. The method for intelligently sensing and identifying the contact strength of tooth interproximal surfaces according to claim 1, characterized in that: The separation branch block includes four branches as follows: 1x1 convolution + BN layer; 1x1 convolution + BN layer + KxK convolution + BN layer; 1x1 convolution + BN layer + average pooling + BN layer; 1x1 convolution + BN layer; After the results of the four branches are added together, the output of the separation branch block is obtained through an activation function.

7. The method for intelligently sensing and identifying the contact strength of tooth interproximal surfaces according to claim 1, characterized in that: The sound signal classification model also includes a global average pooling layer, a fully connected layer and a softmax function connected in sequence, and the global average pooling layer is connected to the second residual network.

8. An intelligent perception and recognition system for tooth interproximal contact strength, characterized in that: include: A data acquisition module, used to acquire sound data of tooth interproximal contact strength, and process the sound data to obtain a frequency spectrum; A model building module, used to build a sound signal classification model; the sound signal classification model includes a point-by-point deep convolutional network and two residual networks connected in sequence, and also includes a lightweight attention network, the lightweight attention network is placed between the two residual networks, wherein the second 3*3 convolutional block in the BasicBlock of the residual network is replaced with a separation branch block; The output module is used to input the spectrum graph into a trained sound signal classification model and output a tooth proximal contact strength recognition result.

Images