Intelligent perception and identification method and system for tooth interproximal contact strength

Abstract

An intelligent perception and identification method and system for tooth interproximal contact strength, relating to the technical field of oral health test. A sound signal classification model is used to identify whether the tooth interproximal contact strength is normal or not, wherein the sound signal classification model comprises a pointwise and depthwise convolutional network and two residual networks which are connected in sequence, and further comprises a lightweight attention network arranged between the two residual networks, and a second 3*3 convolutional block in a BasicBlock of each residual network is replaced with a diverse branch block. Ripple feature information of a spectrogram can be effectively learned under a relatively low parameter, a down-sampling process is promoted to extract effective information, feature information generated by different filters is effectively fused, the feature expression capability and the model accuracy are improved, and ultimately, whether the interproximal contact strength is normal or not can be detected and determined by utilizing a model.

Description

A method and system for intelligent perception and recognition of tooth interproximal contact strength Technical Field

[0001] The present invention relates to the field of oral health detection technology, and more particularly to a method and system for intelligently sensing and identifying tooth interproximal contact strength. Background Art

[0002] The interproximal contact zone, or proximal area, refers to the area where teeth are in close contact with adjacent teeth in the same dental arch. A well-contoured, correctly positioned, and firm proximal contact zone is essential for maintaining the integrity of the dental arch and the health of the supporting structure. In clinical practice, indicators such as proximal strength, area, position, and shape are often used to quantitatively study and evaluate the proximal area. Low proximal strength may lead to common oral diseases such as food impaction, caries, and periodontal disease; high proximal strength may cause problems such as poor tooth movement and periodontal tissue damage. Similarly, clinical diseases such as food impaction or caries may also destroy the proximal areas of teeth, resulting in abnormal proximal strength, thereby aggravating food impaction or tooth caries.

[0003] Currently, there are several methods for detecting adjacency strength:

[0004] ① Dental floss: The most commonly used method for examining the interproximal strength in clinical practice. It has high repeatability in determining whether there is interproximal contact and is a qualitative test.

[0005] ② Different thicknesses of articulating paper or metal film: The thickness of the articulating paper or metal film that initially has resistance entering the proximal contact area of ​​adjacent teeth is equated with the proximity strength, which is a semi-quantitative test.

[0006] ③ Interdental space gauges (feeler gauges) or vernier calipers of varying thickness: The dimension that can smoothly penetrate the interproximal area is defined as the interproximal strength. This is primarily used for interproximal areas with gaps and is a semi-quantitative test.

[0007] ④ Thin metal strip: The maximum friction generated by removing the metal strip located in the adjacent area from the buccal and lingual direction or from The maximum friction force generated by the metal strip extending outward into the adjacent area is the adjacent strength, which is a quantitative test.

[0008] ⑤ Charge-coupled device or stereoscopic photography: The instrument is complex, the measurement is extremely cumbersome, the clinical application is limited, and it is a quantitative test.

[0009] Of the aforementioned methods, the first three lack accuracy, effectiveness, and repeatability, making it impossible to accurately quantify adjacency strength. Method 4 passively separates the adjacency zone during measurement, which is inconsistent with physiological conditions and requires complex testing equipment. Method 5 also requires cumbersome equipment, significantly limiting its clinical application. Therefore, a portable, objective, and stable method and device for determining the normality of adjacency strength is currently lacking in clinical practice.

[0010] Therefore, it is an urgent problem to be solved by those skilled in the art to propose a portable intelligent sensing and recognition method and system that can objectively and stably detect the contact strength of tooth interproximal surfaces.

[0011] Summary of the Invention

[0012] In view of this, the present invention provides an intelligent perception and recognition method and system for tooth interproximal contact strength, constructs a sound signal classification model, can effectively learn the ripple feature information of the spectrum graph within lower parameters, promote the downsampling process to extract effective information, and effectively fuse the feature information generated by different filters, thereby improving the feature expression ability and model accuracy. Finally, the model can be used to detect and determine whether the adjacent strength is normal.

[0013] In order to achieve the above object, the present invention adopts the following technical solutions:

[0014] In one aspect, the present invention discloses a method for intelligently sensing and identifying the contact strength of tooth interproximal surfaces, comprising the following steps:

[0015] Acquiring tooth interproximal contact strength sound data, and processing the sound data to obtain a frequency spectrum;

[0016] Constructing a sound signal classification model; the sound signal classification model includes a sequentially connected point-by-point deep convolutional network, two residual networks, and a lightweight attention network, wherein the lightweight attention network is placed between the two residual networks, wherein the second 3*3 convolutional block in the BasicBlock of the residual network is replaced with a separation branch block;

[0017] The spectrum graph is input into a trained sound signal classification model to output a tooth interproximal contact strength recognition result.

[0018] Preferably, processing the sound data to obtain a spectrogram includes:

[0019] Pre-emphasize the sound data to obtain a pre-emphasized audio waveform;

[0020] Performing frame windowing and short-time Fourier transform on the pre-emphasized audio waveform to obtain a spectrum matrix;

[0021] The spectrum matrix is ​​converted into the spectrum map.

[0022] Preferably, the lightweight attention network includes a channel attention network and a spatial attention network;

[0023] The feature maps are grouped along the channel dimension, and each group generates a channel attention map and a spatial attention map through the channel attention network and the spatial attention network respectively;

[0024] The channel attention map and the spatial attention map are aggregated and subjected to a channel shuffling operation before being input into the second residual network.

[0025] Preferably, the channel attention network is improved by global average pooling, scaling and activation functions, and the formula is as follows: X′ i1 =σ(W1s+b1).X i1 ;

[0026] Among them, X' i1 is the output of the channel attention network, dividing the feature map X into G groups, X∈R C*W*H , C, H, W represent the channel coding, width and height of the feature map respectively, X i1 is the first branch of the i-th group of feature graph X, f gp (·) is the global pooling operation, s is the feature map generated after the global pooling operation, W1 is the first weight parameter, b1 is the first bias parameter, and σ represents the sigmoid activation function.

[0027] Preferably, the calculation formula of the spatial attention network is as follows: i2 =σ(W2.GN(X i2 +b2).X i2 );

[0028] Among them, X' i2 is the output of the spatial attention network, dividing the feature map X into G groups, X∈R C*W*H , C, H, W represent the channel coding, width and height of the feature map respectively, X i2 is the second branch of the i-th group of feature graph X, W2 is the second weight parameter, b2 is the second bias parameter, and σ represents the sigmoid activation function.

[0029] Preferably, the separation branch block includes four branches as follows:

[0030] 1x1 convolution + BN layer;

[0031] 1x1 convolution + BN layer + KxK convolution + BN layer;

[0032] 1x1 convolution + BN layer + average pooling + BN layer;

[0033] 1x1 convolution + BN layer;

[0034] After the results of the four branches are added together, the output of the separation branch block is obtained through an activation function.

[0035] Preferably, the sound signal classification model further includes a global average pooling layer, a fully connected layer and a softmax function connected in sequence, and the global average pooling layer is connected to the second residual network.

[0036] On the other hand, the present invention further discloses a system for intelligently sensing and identifying the contact strength of proximal surfaces of teeth, which is used to implement the above-mentioned method for intelligently sensing and identifying the contact strength of proximal surfaces of teeth. The system comprises:

[0037] A data acquisition module is used to acquire sound data of the contact strength of the tooth interproximal surfaces and process the sound data to obtain a spectrum diagram;

[0038] A model construction module for constructing a sound signal classification model; the sound signal classification model includes a sequentially connected point-by-point deep convolutional network and two residual networks, and also includes a lightweight attention network, wherein the lightweight attention network is placed between the two residual networks, wherein the second 3*3 convolution block in the BasicBlock of the residual network is replaced with a separation branch block;

[0039] The output module is used to input the spectrum graph into the trained sound signal classification model and output the tooth proximal contact strength recognition result.

[0040] As can be seen from the above technical solutions, the present invention discloses a method and system for intelligent perception and recognition of interproximal contact strength of teeth, which has the following beneficial effects compared with the prior art:

[0041] 1. The present invention stacks two layers of residual networks (DBNet) as the backbone network, which can reduce network training time, reduce the number of parameters, and achieve better training results in a shorter training time.

[0042] 2. This paper proposes point-by-point deep convolution (PDNet) to replace the downsampling module of the backbone network, effectively extracting feature information, improving the network downsampling capability, and further improving network performance.

[0043] 3. The 3*3 convolution module in the residual network basicblock is replaced by a Diverse Branch Block (DBB) to introduce different receptive fields, significantly improving the network recognition performance under multi-branch structures of different complexities.

[0044] 4. A lightweight attention network (Shuffle Attention, SA) is embedded between the two residual networks to improve the ability to express effective information between the residual networks and improve the performance of tooth adjacency strength recognition and classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0046] FIG1 is a flow chart of the method of the present invention;

[0047] FIG2 is a schematic diagram of the structure of a sound signal classification model according to the present invention;

[0048] FIG3 is a schematic diagram of the point-by-point deep convolutional network structure of the present invention;

[0049] FIG4 is a schematic diagram of the lightweight attention network structure of the present invention;

[0050] FIG5 is a schematic structural diagram of a separation branch block according to the present invention;

[0051] Figure 6(a) shows the Mel spectrum diagram when the recognition result is normal, and Figure 6(b) shows the Mel spectrum diagram when the recognition result is abnormal. DETAILED DESCRIPTION

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0053] In one aspect, an embodiment of the present invention discloses a method for intelligently sensing and identifying interproximal contact strength of teeth. Referring to FIG1 , the method includes the following steps:

[0054] S1. Acquire sound data of interproximal contact strength of teeth and process the sound data to obtain a spectrum, including:

[0055] Pre-emphasize the sound data to obtain a pre-emphasized audio waveform;

[0056] Perform frame windowing and short-time Fourier transform on the pre-emphasized audio waveform to obtain a spectrum matrix;

[0057] Convert the spectrum matrix into a spectrogram.

[0058] The sound data here is the audio signal of the interproximal contact strength sound of the teeth generated with the help of dental floss, which varies greatly over time. The frequency domain features contain time domain signal information and can simultaneously reflect the frequency characteristics of the audio. If the audio signal is directly subjected to short-time Fourier transform (STFT), part of the audio waveform will be distorted, so the audio will be pre-emphasized before STFT. Pre-emphasis can not only reduce the impact of some amplitude-frequency distortion and frequency response changes, but also reduce low-frequency noise in the audio signal. The pre-emphasized audio waveform is framed, windowed, and STFTed to obtain a spectrum matrix. The process is as follows:

[0059] y(t) is the time domain signal, x is the frequency domain signal, w(tr) represents the Hamming window with the center position at r, f is the frequency, and r is the frame length.

[0060] The Mel filter is designed based on the characteristics of the human ear to improve the absorption of the lower frequencies of the sound. In this embodiment, the number of Mel filters per second in the sound audio clip is set to 128. The formula of the Mel filter is expressed as follows:

[0061] where f mel is the calculated Mel-scale frequency, and f is the normal Hertz frequency.

[0062] The Mel filter bank mimics the human ear in filtering speech. M triangular filters are set within the frequency range of an audio segment. In this example, 512 filters are set. The filter layout is denser and sparser. As the hertz frequency increases, the filter width increases from small to large. Each filter has a 50% overlap to prevent information loss. On the Mel scale, these filters appear to have equal widths. Finally, the output matrix is ​​converted into a spectrogram.

[0063] S2. Construct a sound signal classification model; the sound signal classification model includes a point-by-point deep convolutional network (PDNet) and two residual networks (DBBDB) connected in sequence, as well as a lightweight attention network (Shuffle Attention, SA). The lightweight attention network is placed between the two residual networks. Among them, the second 3*3 convolution block in the BasicBlock of the residual network is replaced with a separation branch block (DBB), refer to Figure 2.

[0064] Specifically, the PDNet structure in this embodiment can refer to Figure 3. The first to third parts in Figure 3 are the point-by-point convolution process, and the subsequent parts are the depth-separable convolution process.

[0065] Specifically, the mel-spectrogram generated by feature extraction is passed through a point-by-point convolutional network to generate a 16-channel feature map, increasing the number of channels in the spectrogram. While increasing dimensionality, it also effectively integrates the inter-channel information removed by the deep convolution, preserving the effective feature information of the spectrogram. The feature map is then passed through a depthwise separable convolutional network to generate a 64-channel thumbnail feature map. This reduces the number of model parameters while effectively improving network performance and alleviating the parameter increase problem. The depthwise separable convolution splits the standard convolution into depthwise and point-by-point convolutions, simultaneously mapping feature channels and spatial dimensions, effectively preserving the ripple information of the spectrogram and removing environmental noise.

[0066] When the backbone network recognizes tooth contact strength sound samples, the interference of ambient sound in the samples can cause certain misjudgments in the recognition results. To improve the inheritance of effective information between the two residual networks of the backbone network and reduce the interference caused by ambient sound during the network recognition process, this embodiment embeds a lightweight attention network SA between the two residual networks of the backbone network. As shown in Figure 4, this lightweight attention network uses a shuffle unit to effectively combine two types of attention mechanisms.

[0067] Specifically, SA first groups channel dimensions into multiple sub-features, then processes these sub-features in parallel. For each sub-feature, SA utilizes a shuffle unit to describe feature dependencies in both spatial and channel dimensions. Finally, all sub-features are aggregated, and a "channel shuffle" operator is used to facilitate information communication between different sub-features.

[0068] Feature grouping: Given an input feature map X∈R C*W*H , where C, H, and W represent the channel code, width, and height of the feature map, respectively. The feature map X is first divided into G groups along the channel dimension, i.e. Each group is then divided into two branches along the channel direction. One branch is a channel attention network, which uses the relationship between channels to generate a channel attention map. The other branch is a spatial attention network, which uses the relationship between features to generate a spatial attention map. In this embodiment, G is set to 16.

[0069] The channel attention network is improved by combining global average pooling + scaling + activation function. The specific calculation formula is as follows:

[0070] are two parameters that can be trained continuously through the network, σ represents the sigmoid activation function, f gp(·) is the global pooling operation, and s is the feature map generated after the global average pooling operation.

[0071] Spatial Attention Network: Spatial attention can be viewed as a complement to channel attention, using a group normalization (GN) operation.

[0072] The specific formula is as follows: X′ i2 =σ(W2.GN(X i2 +b2).X i2 );

[0073] There are also two parameters that are continuously trained through the network, and σ represents the sigmoid activation function.

[0074] Aggregation: After completing the two attention learning and recalibrating the features, the two branches need to be spliced ​​and aggregated. to aggregate all sub-features and perform channel shuffling operations.

[0075] In the recognition process of sound perception samples of interproximal contact strength, different receptive fields were introduced to increase the model's expressive power. In the basic blocks of the two residual networks in the backbone network, the second 3*3 Conv module was replaced with DBB. DBB enriches the feature space by combining branches of different scales and complexities, including convolution sequences, multi-scale convolutions, and average pooling to enhance the representational power of a single convolution. DBB adopts a complex "microstructure" during training while keeping the macrostructure unchanged. This "heterogeneity" enables the model to have higher complexity and performance during training, and can return to the original structure for inference.

[0076] Assuming the number of input channels is C, the number of output channels is D, and the size of C is K×K, then the Conv kernel is F∈R D×C×K×K , the optional bias parameter is b∈R D To facilitate subsequent merging, the deviation parameter is set to REP(b)∈R D×H×W ,Convolution is defined as follows: O=I×F+REP(B);

[0077] The calculation method of the value of the j-th output channel at (h,w) is: j,h,w =∑ c＝1 ∑ u＝1 ∑ v＝1 F j,c,,u,v X (c,h,w)u,v +b j ;

[0078] where X(c,h,w)∈R K×KRepresents a sliding window. The linear properties of convolution, including uniformity and additivity, can be derived from the above formula. This embodiment uses a typical example of DBB, as shown in Figure 5, to enhance traditional Conv using equal combinations. For the 1×1 and K×K branches, the number of intermediate channels is set equal to the number of input channels, the initialized 1×1Conv is used as the unit matrix, and the other branches are also initialized in the traditional way. A batch normalization (BN) layer is added after each convolution to provide nonlinearity during training, which is necessary to improve performance.

[0079] Specifically, as shown in FIG5 , the DBB proposed in this embodiment includes four branches as follows:

[0080] 1x1 convolution + BN layer;

[0081] 1x1 convolution + BN layer + KxK convolution + BN layer;

[0082] 1x1 convolution + BN layer + average pooling + BN layer;

[0083] 1x1 convolution + BN layer;

[0084] After the results of the four branches are added together, the output of the separation branch block is obtained through the activation function.

[0085] According to Figure 2, the sound signal classification model also includes a global average pooling layer, a fully connected layer and a softmax function connected in sequence, and the global average pooling layer is connected to the second residual network.

[0086] S3. Input the spectrum graph into the trained sound signal classification model and output the tooth interproximal contact strength identification results, which are normal and abnormal.

[0087] In this embodiment, the intensity sound of Oral B dental floss passing through the adjacent area of ​​normal teeth and abnormal teeth (the unified teeth positions are 36 and 37 in this example) is collected according to certain standards to establish a sample data set.

[0088] The collected samples of the normal and abnormal adjacency strength dataset were converted into Mel spectrograms using a Mel filter. The results are shown in Figures 6(a) and 6(b). The visualization results of the normal and abnormal images are quite different. The ripple characteristics of the normal image (Figure 6(a)) are obvious, while the ripples of the abnormal image (Figure 6(b)) are more blurred.

[0089] The spectrogram has certain additive and multiplicative noise. This embodiment adds an attention mechanism to the sound signal classification model to improve the ripple information of the spectrogram, effectively improving the recognition performance in the backbone network with lower parameters.

[0090] On the other hand, the present invention further discloses a system for intelligently sensing and identifying the contact strength of proximal surfaces of teeth, which is used to implement the above-mentioned method for intelligently sensing and identifying the contact strength of proximal surfaces of teeth. The system comprises:

[0091] A data acquisition module is used to acquire sound data of the interproximal contact strength of teeth and process the sound data to obtain a spectrum diagram;

[0092] A model construction module for constructing a sound signal classification model; the sound signal classification model includes a sequentially connected point-by-point deep convolutional network, two residual networks, and a lightweight attention network placed between the two residual networks. The lightweight attention network is replaced by a separation branch block in the second 3*3 convolution block in the BasicBlock of the residual network;

[0093] The output module is used to input the spectrum graph into the trained sound signal classification model and output the tooth interproximal contact strength recognition result.

[0094] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0095] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for intelligent perception and recognition of tooth interproximal contact strength, characterized in that: The following steps are involved: Acquiring tooth interproximal contact strength sound data, and processing the sound data to obtain a frequency spectrum; Constructing a sound signal classification model; the sound signal classification model includes a sequentially connected point-by-point deep convolutional network, two residual networks, and a lightweight attention network, wherein the lightweight attention network is placed between the two residual networks, wherein the second 3*3 convolutional block in the BasicBlock of the residual network is replaced with a separation branch block; The spectrum graph is input into a trained sound signal classification model to output a tooth interproximal contact strength recognition result.

2. The method for intelligent perception and recognition of tooth interproximal contact strength according to claim 1, characterized in that: The sound data is processed to obtain a spectrogram, including: Pre-emphasize the sound data to obtain a pre-emphasized audio waveform; Performing frame windowing and short-time Fourier transform on the pre-emphasized audio waveform to obtain a spectrum matrix; The spectrum matrix is ​​converted into the spectrum map.

3. The method for intelligent perception and recognition of tooth interproximal contact strength according to claim 1, characterized in that: The lightweight attention network includes a channel attention network and a spatial attention network; The feature maps are grouped along the channel dimension, and each group generates a channel attention map and a spatial attention map through the channel attention network and the spatial attention network respectively; The channel attention map and the spatial attention map are aggregated and subjected to a channel shuffling operation before being input into the second residual network.

4. The method for intelligent perception and recognition of tooth interproximal contact strength according to claim 3, characterized in that: The channel attention network is improved by global average pooling, scaling and activation function, and the formula is as follows: X′ i1 =σ(W1s+b1).X i1 ; Among them, X' i1 is the output of the channel attention network, dividing the feature map X into G groups, X∈R C*W*H , C, H, W represent the channel coding, width and height of the feature map respectively, X i1 is the first branch of the i-th group of feature graph X, f gp (·) is the global pooling operation, s is the feature map generated after the global pooling operation, W1 is the first weight parameter, b1 is the first bias parameter, and σ represents the sigmoid activation function.

5. The method for intelligent perception and recognition of tooth interproximal contact strength according to claim 3, characterized in that: The calculation formula of the spatial attention network is as follows: i2 =σ(W2.GN(X i2 +b2).X i2 ); Among them, X' i2 is the output of the spatial attention network, dividing the feature map X into G groups, X∈R C*W*H , C, H, W represent the channel coding, width and height of the feature map respectively, X i2 is the second branch of the i-th group of feature graph X, W2 is the second weight parameter, b2 is the second bias parameter, and σ represents the sigmoid activation function.

6. The method for intelligent perception and recognition of tooth interproximal contact strength according to claim 1, characterized in that: The separation branch block includes four branches as follows: 1x1 convolution + BN layer; 1x1 convolution + BN layer + KxK convolution + BN layer; 1x1 convolution + BN layer + average pooling + BN layer; 1x1 convolution + BN layer; After the results of the four branches are added together, the output of the separation branch block is obtained through an activation function.

7. The method for intelligent perception and recognition of tooth interproximal contact strength according to claim 1, characterized in that: The sound signal classification model also includes a global average pooling layer, a fully connected layer and a softmax function connected in sequence, and the global average pooling layer is connected to the second residual network.

8. An intelligent perception and recognition system for tooth interproximal contact strength, characterized in that: include: A data acquisition module is used to acquire sound data of the contact strength of the tooth interproximal surfaces and process the sound data to obtain a spectrum diagram; A model construction module for constructing a sound signal classification model; the sound signal classification model includes a sequentially connected point-by-point deep convolutional network and two residual networks, and also includes a lightweight attention network, wherein the lightweight attention network is placed between the two residual networks, wherein the second 3*3 convolution block in the BasicBlock of the residual network is replaced with a separation branch block; The output module is used to input the spectrum graph into the trained sound signal classification model and output the tooth proximal contact strength recognition result.

Images