An acoustic-based tooth brushing activity recognition method and system

Abstract

The application discloses a tooth brushing activity recognition method and system based on acoustics, relates to the sound recognition technical field, and is used for solving the technical problem that, in the process of recognizing a tooth brushing area, the change of a user's head action leads to misjudgment of the tooth brushing area and the accurate recognition of the tooth brushing activity cannot be realized in the prior art. The tooth brushing activity recognition method comprises the following steps: dividing the tooth surface of a human body into a plurality of tooth brushing activity areas; training a convolutional neural network by collecting sound signals of an electric toothbrush in a tooth brushing process to obtain a training model; collecting sound data of the electric toothbrush in the tooth brushing process; inputting the sound data into the training model, and outputting, by the training model, a tooth brushing activity area where a brush head of the electric toothbrush is located at any moment, so as to realize the recognition of the tooth brushing activity; and the technical scheme of the application is used for providing a tooth brushing activity recognition method and system based on acoustics.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of sound recognition, and more particularly, to an acoustic-based tooth brushing activity recognition method and system. BACKGROUND

[0002] Oral health is closely related to overall health, and correct tooth brushing is the key to maintaining good oral hygiene and preventing dental caries and periodontal diseases. However, many people have improper posture, insufficient or excessive brushing time, and other problems during daily brushing, resulting in poor brushing effect and causing oral health problems.

[0003] With the development of intelligent tooth brushing technology, a tooth brushing area recognition method has emerged. The tooth brushing area recognition method refers to a technology in which a tooth brushing device automatically recognizes a tooth brushing area. In the prior art, tooth brushing area recognition is mainly achieved by acquiring posture information of the tooth brushing device through a sensor on the tooth brushing device, and then recognizing the tooth brushing area of the user according to the posture information. However, if the user's head movement changes during brushing, it will cause misjudgment of the tooth brushing area, and accurate recognition of the tooth brushing activity cannot be achieved. SUMMARY

[0004] The present application aims to provide an acoustic-based tooth brushing activity recognition method and system to solve the technical problem that the existing tooth brushing area recognition method causes misjudgment of the tooth brushing area due to changes in the user's head movement during the recognition process, and cannot accurately recognize the tooth brushing activity. Therefore, the present application is implemented by the following scheme.

[0005] In a first aspect, the present application provides an acoustic-based tooth brushing activity recognition method, comprising:

[0006] dividing the tooth surface of the human body into a plurality of tooth brushing activity areas;

[0007] training a convolutional neural network by collecting sound signals of an electric toothbrush during tooth brushing to obtain a trained model;

[0008] collecting sound data of the electric toothbrush during tooth brushing;

[0009] inputting the sound data into the trained model, and outputting the tooth brushing activity area where the toothbrush head of the electric toothbrush is located at any time, to realize recognition of the tooth brushing activity;

[0010] wherein the convolutional neural network takes the sound signal as input and connects each tooth brushing activity area corresponding to the sound signal through a fully connected layer.

[0011] Compared with the prior art, the acoustic-based tooth brushing activity recognition method of the present application is used for recognizing tooth brushing activity, in particular, for recognizing a tooth brushing activity zone, which is suitable for a commercially available electric toothbrush. In the tooth brushing activity recognition method, the tooth surface of a human body is first divided into a plurality of tooth brushing activity zones, a convolutional neural network is trained by collecting sound signals of the electric toothbrush during tooth brushing, and a training model is obtained, wherein the convolutional neural network takes the sound signals as input and each tooth brushing activity zone corresponding to the sound signals is determined through a fully connected layer. In the training model, the sound signals during tooth brushing correspond to the tooth brushing activity zones, that is, the tooth brushing activity zone corresponding to the sound signals at any time can be output by inputting the sound signals into the training model; then the application of the training model is applied, specifically, the collected sound data is input into the training model, and the tooth brushing activity zone where the brush head of the electric toothbrush is located at any time can be output by the training model, thereby realizing the recognition of tooth brushing activity. It can be seen that the technical solution of the present application is based on sound data (or sound signals) for tooth brushing activity recognition, and the recognition accuracy is basically not affected by the change of the user's head movement (based on sound signals rather than movement), which can realize accurate recognition of tooth brushing activity, and does not need to additionally increase or purchase wearable devices, thereby saving costs. Through the above technical solution of the present application, the technical problem that the existing tooth brushing area recognition method cannot realize accurate recognition of tooth brushing activity due to misjudgment of the tooth brushing area caused by the change of the user's head movement during recognition of the tooth brushing area is solved.

[0012] Further, in the acoustic-based tooth brushing activity recognition method of the present application, the sound data of the electric toothbrush during tooth brushing is collected by a sound collecting device, and the sound collecting device has a first microphone and a second microphone. When collecting the sound data, the first microphone is above and the second microphone is below.

[0013] Further, in the acoustic-based tooth brushing activity recognition method of the present application, after collecting the sound data of the electric toothbrush during tooth brushing, the method further comprises:

[0014] obtaining the difference between the distances from the first microphone and the second microphone to the brush head of the electric toothbrush;

[0015] determining whether the brush head is on the upper jaw or the lower jaw according to the difference between the distances.

[0016] Further, in the acoustic-based tooth brushing activity recognition method of the present application, the tooth brushing activity zone includes a left upper cheek surface, a left lower cheek surface, a right upper cheek surface, a right lower cheek surface, a middle upper lip surface, a middle lower lip surface, a left upper occlusal surface, a left lower occlusal surface, a right upper occlusal surface, a right lower occlusal surface, and an idle.

[0017] Further, in the tooth brushing activity recognition method based on acoustics, the convolutional neural network comprises three convolutional layers and two fully connected layers.

[0018] One of the two fully connected layers has 1024 neurons, and the other has 512 neurons.

[0019] Further, in the tooth brushing activity recognition method based on acoustics, after the sound data of the electric toothbrush during tooth brushing is collected, the method further comprises:

[0020] The sound data is converted from a time domain signal to a frequency spectrum signal, and the frequency spectrum signal is frequency spectrum data, which is used as input to the training model.

[0021] Further, in the tooth brushing activity recognition method based on acoustics, the sound data is converted from a time domain signal to a frequency spectrum signal, which comprises:

[0022] Step 1: set the sampling frequency of the sound data to 22.5 kHz, and the size of the frame to 2 periods of sampling points; the sampling points are the ratio of the sampling frequency to the rotation speed of the electric toothbrush;

[0023] Step 2: add a Hanning window to the frame data of the sound data;

[0024] Step 3: perform discrete Fourier transform on the sound data to obtain frequency domain data;

[0025] Step 4: take the absolute value and square of the frequency domain data to obtain frequency spectrum data.

[0026] Further, in the tooth brushing activity recognition method based on acoustics, after the frequency spectrum data is obtained, the method further comprises:

[0027] Step 5: perform log amplitude transformation on the frequency spectrum data to obtain log frequency spectrum data, which is used as input to the training model.

[0028] Further, in the tooth brushing activity recognition method based on acoustics, in the process of adding a Hanning window to the frame data of the sound data, the overlap rate of the Hanning window is 50%.

[0029] In a second aspect, the application further provides a tooth brushing activity recognition system based on acoustics for executing the tooth brushing activity recognition method based on acoustics, which comprises:

[0030] A tooth brushing activity area division module for dividing the tooth surface of a human body into a plurality of tooth brushing activity areas.

[0031] A neural network training module is configured to train a convolutional neural network by collecting sound signals of the electric toothbrush during tooth brushing, and obtain a training model;

[0032] A data collection module is configured to collect sound data of the electric toothbrush during tooth brushing;

[0033] An identification module is configured to input the sound data into the training model, and the training model outputs a tooth brushing activity area where the brush head of the electric toothbrush is located at any moment;

[0034] A sound source positioning module is configured to obtain a difference between distances from the first microphone and the second microphone to the brush head of the electric toothbrush, and determine whether the brush head is located on the upper teeth or the lower teeth according to the difference between the distances.

[0035] Compared with the prior art, the beneficial effects of the tooth brushing activity recognition system based on acoustics are the same as those of the tooth brushing activity recognition method based on acoustics, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0036] The accompanying drawings, which are included to provide a further understanding of the application, illustrate embodiments of the application and together with the description serve to explain the application. The illustrative embodiments of the application and the description thereof are not meant to limit the application in any way.

[0037] Figure 1 FIG. 1 is a flowchart of a tooth brushing activity recognition method based on acoustics according to an embodiment of the present application;

[0038] Figure 2 FIG. 2 is a flowchart of another tooth brushing activity recognition method based on acoustics according to an embodiment of the present application;

[0039] Figure 3 FIG. 3 is a composition diagram of a tooth brushing activity recognition system based on acoustics according to an embodiment of the present application;

[0040] Figure 4 FIG. 4 is a diagram of sound data collection according to an embodiment of the present application;

[0041] Figure 5 FIG. 5 is a diagram of sound data processing according to an embodiment of the present application;

[0042] Figure 6 FIG. 6 is a diagram of frame length comparison of electric toothbrushes with different vibration frequencies according to an embodiment of the present application;

[0043] Figure 7 FIG. 7 is a diagram of displaying each tooth brushing activity area after dividing the tooth surface according to an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the technical problems to be solved by the present application, technical solutions and beneficial effects clearer, the present application will be further described in detail below in conjunction with the drawings and examples. It should be understood that the specific examples described herein are only intended to explain the present application and not to limit the present application.

[0045] It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it can be directly on the other element or indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element.

[0046] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise explicitly and specifically limited. The meaning of "several" is one or more, unless otherwise explicitly and specifically limited.

[0047] With the development of intelligent tooth brushing technology, a tooth brushing area recognition method has appeared. The tooth brushing area recognition method refers to a technology that automatically recognizes the tooth brushing area by the tooth brushing device. In the prior art, the recognition of the tooth brushing area is mainly through the sensor on the tooth brushing device to obtain the posture information of the tooth brushing device, so as to recognize the tooth brushing area of the user according to the posture information. However, if the head movement changes during the tooth brushing process of the user, it will cause misjudgment of the tooth brushing area, and the accurate recognition of the tooth brushing activity cannot be realized.

[0048] Please refer to Figure 1 , in order to solve the above technical problems, in a first aspect, the present application provides a tooth brushing activity recognition method based on acoustics, comprising:

[0049] S100, dividing the tooth surface of the human body into a plurality of tooth brushing activity areas;

[0050] S200, training a convolutional neural network by collecting sound signals of the electric toothbrush during tooth brushing to obtain a training model;

[0051] S300, collecting sound data of the electric toothbrush during tooth brushing;

[0052] S400, inputting the sound data into the training model, and the training model outputs the tooth brushing activity area where the toothbrush head of the electric toothbrush is located at any time, realizing the recognition of the tooth brushing activity;

[0053] The convolutional neural network takes the sound signal as the input, and each tooth brushing activity area corresponding to the sound signal is obtained through the fully connected layer.

[0054] With the above technical solution, the tooth brushing activity recognition method based on acoustics of the present application is used to recognize tooth brushing activity, specifically, to recognize tooth brushing activity area, which is suitable for commercially available electric toothbrushes. In the tooth brushing activity recognition method, the tooth surface of the human body is first divided into a plurality of tooth brushing activity areas, a convolutional neural network is trained by collecting sound signals of the electric toothbrush during tooth brushing, and a training model is obtained, wherein the convolutional neural network takes the sound signal as input and corresponds to each tooth brushing activity area through a fully connected layer. In the training model, the sound signal during tooth brushing corresponds to the tooth brushing activity area, that is, the tooth brushing activity area corresponding to the sound signal at any time can be output by inputting the sound signal into the training model; then the application of the training model is applied, specifically, the collected sound data is input into the training model, and the training model can output the tooth brushing activity area where the brush head of the electric toothbrush is located at any time, thereby realizing the recognition of tooth brushing activity. It can be seen that the technical solution of the present application is based on sound data (or sound signal) for tooth brushing activity recognition, and the recognition accuracy is basically not affected by the change of the user's head movement (based on sound signal instead of movement), which can realize accurate recognition of tooth brushing activity, and does not need to increase or purchase additional wearable devices, saving costs. Through the above technical solution of the present application, the technical problem that the existing tooth brushing area recognition method causes misjudgment of the tooth brushing area due to the change of the user's head movement during the recognition of the tooth brushing area, and cannot realize accurate recognition of tooth brushing activity is solved.

[0055] Further, referring to Figure 2 Another tooth brushing activity recognition method based on acoustics of the present application comprises:

[0056] Step 1, dividing the tooth surface of the human body into a plurality of tooth brushing activity areas;

[0057] Step 2, training a convolutional neural network by collecting sound signals of the electric toothbrush during tooth brushing to obtain a training model;

[0058] Step 3, collecting sound data of the electric toothbrush during tooth brushing;

[0059] Step 4, inputting the sound data into the training model, and the training model outputting the tooth brushing activity area where the brush head of the electric toothbrush is located at any time, thereby realizing the recognition of tooth brushing activity;

[0060] Step 5, determining the brush head in the upper jaw or lower jaw according to the sound source positioning.

[0061] In the technical solution, the sound data of the electric toothbrush during tooth brushing is collected by a sound collecting device, the sound collecting device has a first microphone and a second microphone, and when the sound data is collected, the first microphone is above and the second microphone is below. After the sound data of the electric toothbrush during tooth brushing is collected, the difference between the distances from the first microphone and the second microphone to the brush head of the electric toothbrush is obtained, and the brush head is determined to be on the upper jaw or the lower jaw according to the distance difference, that is, the brush head is determined to be on the upper jaw or the lower jaw according to sound source positioning (step 5 in Figure 2 ), which further improves the accuracy of tooth brushing activity recognition.

[0062] In a second aspect, the application further provides an acoustic-based tooth brushing activity recognition system for executing the acoustic-based tooth brushing activity recognition method described above, please refer to Figure 3 , the tooth brushing activity recognition system comprises:

[0063] a tooth brushing activity area division module for dividing the tooth surface of a human body into a plurality of tooth brushing activity areas;

[0064] a neural network training module for training a convolutional neural network by collecting sound signals of an electric toothbrush during tooth brushing to obtain a training model;

[0065] a data collection module for collecting sound data of an electric toothbrush during tooth brushing;

[0066] a recognition module for inputting the sound data into the training model, and the training model outputs the tooth brushing activity area where the brush head of the electric toothbrush is located at any time;

[0067] a sound source positioning module for obtaining the difference between the distances from the first microphone and the second microphone to the brush head of the electric toothbrush, and determining the brush head to be on the upper jaw or the lower jaw according to the distance difference.

[0068] In the acoustic-based tooth brushing activity recognition system of the application, the above acoustic-based tooth brushing activity recognition method can be efficiently executed through the cooperation of the modules.

[0069] In order to better understand the application, the application will be further illustrated below in conjunction with specific examples, but the application is not limited to the following examples.

[0070] Example 1

[0071] The embodiment provides an acoustic-based tooth brushing activity recognition method, comprising:

[0072] Step 1, dividing the tooth surface of a human body into a plurality of tooth brushing activity areas;

[0073] Step 2, training a convolutional neural network by collecting sound signals of the electric toothbrush during tooth brushing to obtain a trained model; wherein the convolutional neural network takes the sound signals as input and each tooth brushing activity area corresponding to the sound signals is output through a fully connected layer;

[0074] Step 3, collecting sound data of the electric toothbrush during tooth brushing;

[0075] Step 4, inputting the sound data into the trained model, and the trained model outputs the tooth brushing activity area where the toothbrush head of the electric toothbrush is located at any time, thereby realizing the recognition of tooth brushing activities.

[0076] Embodiment 2

[0077] The embodiment provides an acoustic-based tooth brushing activity recognition method, which comprises:

[0078] S100, dividing the tooth surface of a human body into a plurality of tooth brushing activity areas; please refer to Figure 7 In this embodiment, the tooth brushing activity areas are divided into the following 11 categories, specifically, the tooth surface is divided into: left upper buccal surface (LUB, Left Upper Buccal), left lower buccal surface (LLB, Left Lower Buccal), right upper buccal surface (RUB, Right Upper Buccal), right lower buccal surface (RLB, Right Lower Buccal), mid upper labial surface (MUL, Mid Upper Labial), mid lower labial surface (MLL, Mid Lower Labial), left upper occlusal surface (LUO, Left Upper Occlusal), left lower occlusal surface (LLO, Left Lower Occlusal), right upper occlusal surface (RUO, Right Upper Occlusal), right lower occlusal surface (RLO, Right Lower Occlusal), and idling (IDL, Idling) which is a tooth brushing activity, a total of 11 tooth brushing activity areas;

[0079] S200, training a convolutional neural network by collecting sound signals of the electric toothbrush during tooth brushing to obtain a trained model; wherein the convolutional neural network takes the sound signals as input and each tooth brushing activity area corresponding to the sound signals is output through a fully connected layer;

[0080] The convolutional neural network comprises three convolutional layers and two fully connected layers; wherein one of the two fully connected layers has 1024 neurons, and the other has 512 neurons;

[0081] S300, using a sound acquisition device to collect sound data during the brushing process of an electric toothbrush, the sound acquisition device having a first microphone and a second microphone, wherein when the sound acquisition device collects the sound data, the first microphone is on top and the second microphone is on the bottom;

[0082] Further, step S300 includes:

[0083] S301, Obtain the difference between the distances from the first microphone and the second microphone to the brush head of the electric toothbrush;

[0084] S302, determine whether the brush head is on the maxillary or mandibular teeth based on the difference in the distance;

[0085] S310, the sound data is converted from a time-domain signal to a spectral signal, which is the spectral data, and the spectral data is used to input into the training model;

[0086] Further, step S310 includes:

[0087] S311, Set the sampling frequency of the sound data to 22.5kHz, and the frame size to the number of sampling points in two cycles; the number of sampling points is the ratio of the sampling frequency to the rotation speed of the electric toothbrush.

[0088] S312, Add a Hanning window to the frame data of the audio data, with the overlap rate of the Hanning window being 50%;

[0089] S313, Perform a discrete Fourier transform on the sound data to obtain frequency domain data;

[0090] S314, take the absolute value of the frequency domain data and square it to obtain the spectrum data;

[0091] S315, Perform a log amplitude transformation on the spectral data to obtain logarithmic spectral data, which is then input into the training model;

[0092] S400, the sound data (logarithmic spectrum data) is input into the training model, and the training model outputs the brushing activity area where the electric toothbrush head is located at any time, thereby realizing the recognition of brushing activity.

[0093] Example 3

[0094] Firstly, this embodiment provides an acoustic-based method for recognizing brushing activities, including:

[0095] S100 divides the surface of human teeth into multiple brushing zones; please refer to [link / reference]. Figure 7The present embodiment divides the tooth brushing activity area into the following 11 categories, and specifically divides the tooth surface into: left upper buccal (LUB, Left Upper Buccal), left lower buccal (LLB, Left Lower Buccal), right upper buccal (RUB, Right Upper Buccal), right lower buccal (RLB, Right Lower Buccal), mid upper labial (MUL, Mid Upper Labial), mid lower labial (MLL, Mid Lower Labial), left upper occlusal (LUO, Left Upper Occlusal), left lower occlusal (LLO, Left Lower Occlusal), right upper occlusal (RUO, Right Upper Occlusal), right lower occlusal (RLO, Right Lower Occlusal) 10 tooth surfaces, plus idling (IDL, Idling) which is the tooth brushing activity, a total of 11 tooth brushing activity areas;

[0096] S200, please refer to Figure 5 The training model is obtained by training a convolutional neural network through the sound signals collected during the tooth brushing process of the electric toothbrush; specifically comprising:

[0097] S201, data collection for each tooth brushing activity area, that is, in a quiet environment, the sound data of the electric toothbrush during the tooth brushing process is collected through a smart phone, and the tooth brushing process of the electric toothbrush lasts for 180 seconds; wherein each process covers the tooth brushing activities of 10 tooth surfaces and the idling tooth brushing activity, the collection time of the tooth surface is 15 seconds, and the collection time of the idling is 5 seconds;

[0098] S202, data processing, that is, a 15s audio (sound data) contains the whole process of brushing a tooth surface by the user, and the electric toothbrush will move slowly during the whole process, the obtained audio is divided into segments every 0.21 seconds, and each segment is saved as a new audio file, and the audio segments are ensured not to overlap in time;

[0099] S203, training the convolutional neural network, that is, the audio file in step S202 is input into the convolutional neural network for training, and a classification model is obtained after training, which is the training model obtained above; the convolutional neural network comprises three convolutional layers and two fully connected layers; wherein one of the two fully connected layers has 1024 neurons, and the other has 512 neurons;

[0100] S300, please refer to Figure 4, the sound data of the electric toothbrush during the tooth brushing process is collected by using a smart phone, the smart phone has a first microphone (Mic1) and a second microphone (Mic2), when collecting the sound data, the first microphone (Mic1) is above, and the second microphone (Mic2) is below; Specifically, when the user brushes teeth normally, the smart phone is placed in front of the user (such as in front of a mirror) at a distance of 60 cm, the user holds the electric toothbrush, and the first microphone (Mic1, upper part) and the second microphone (Mic2, lower part) in the smart phone collect the sound data (or audio data) generated during the tooth brushing in real time, and in the process, the distances from the first microphone and the second microphone of the smart phone to the center of the oral cavity are equal;

[0101] Further, the step S300 comprises:

[0102] S301, obtaining the difference between the distances from the first microphone (Mic1) and the second microphone (Mic2) to the brush head of the electric toothbrush, specifically, as shown in Figure 4 , the distance from the first microphone (Mic1) to the brush head is r1 , the distance from the second microphone (Mic2) to the brush head is r2 , and the difference between the distances is obtained by the following formula:

[0103] (1)

[0104] wherein, is the difference between the distances, r1 is the distance from the first microphone (Mic1) to the brush head, r2 is the distance from the second microphone (Mic2) to the brush head, is the difference between the time taken by the sound signal emitted by the brush head to reach the first microphone and the time taken by the sound signal emitted by the brush head to reach the second microphone, is the propagation speed of sound, and in the embodiment ;

[0105] S302, determining whether the brush head is on the upper jaw or the lower jaw according to the difference between the distances;

[0106] For example, in this step, the sound signal emitted by the brush head can be transmitted to the first microphone and the second microphone, for example, the sound signal reaches the first microphone first and then reaches the second microphone, and it can be considered that the brush head is on the upper jaw, and for example, the sound signal reaches the second microphone first and then reaches the first microphone, and it can be considered that the brush head is on the lower jaw;

[0107] S310, converting the sound data from a time domain signal into a frequency spectrum signal, the frequency spectrum signal being frequency spectrum data, and the frequency spectrum data is used as input into the training model;

[0108] Further, step S310 comprises:

[0109] S311, refer to Figure 6 , set the sampling frequency of the sound data to 22.5 kHz, in order to be compatible with the audio data of various electric toothbrushes, assuming that the rotating speed of a certain electric toothbrush is , representing the number of brush movements of the brush head of the electric toothbrush in 1 minute, then the number of brush movements in 1 second is ; assuming that the sampling frequency is sr , the sampling frequency represents the number of samples in 1 second, then the number of sampling points contained in each brush movement of the brush head of the electric toothbrush is N , then:

[0110] (2)

[0111] wherein, N is the number of sampling points, sr is the sampling frequency, represents the rotating speed of the electric toothbrush;

[0112] Further, each brush movement of the brush head of the electric toothbrush represents half a motor cycle in the vibrating toothbrush, and then the number of sampling points contained in one cycle is , which is represented in time as: ;

[0113] wherein, represents the number of sampling points contained in one cycle, sr is the sampling frequency, represents the rotating speed of the electric toothbrush;

[0114] Further, in order to ensure that at least one motor cycle is contained in one frame, the size of the frame should be at least the number of sampling points of 2 cycles, that is, it satisfies ; the rotating speed of the existing vibrating electric toothbrush is between , and the number of sampling points of one cycle is 160.36~55.125, then:

[0115] (3)

[0116] wherein, represents the size of one audio frame; represents the maximum value of the number of sampling points of one cycle of the vibrating electric toothbrush with the rotating speed of 16500 rpm~48000 rpm;

[0117] Since the minimum number greater than 320.72 and being a power of 2 is 512, 512 is selected as the frame length, which can adapt to the vibrating electric toothbrush with the rotating speed greater than ;

[0118] S312, a Hanning window is added to the frame data of the sound data, and the overlap rate of the Hanning window is 50%; the addition of the Hanning window is realized by the following formula:

[0119] (4)

[0120] wherein, represents the window function value obtained by the Hanning window, is the natural constant, is the total number of sample points in the frame, and in the embodiment, n = 256 , is the sample point in the frame,

[0121] S313, the frequency domain data is obtained by performing a discrete Fourier transform on the formula (4); the frequency domain data is represented by the following formula:

[0122] (5)

[0123] wherein, represents the frequency domain data of the sound signal, is the time domain signal of the sample point in any frame, is the natural constant, is the natural constant, is the natural constant, is the frequency, is the time domain time; is the number of sample points in any frame; is the imaginary unit;

[0124] S314, the frequency domain data of the step S313 is taken as an absolute value and squared to obtain the frequency spectrum data;

[0125] S315, the frequency spectrum data is subjected to a log amplitude transformation to obtain the logarithmic frequency spectrum data, and the logarithmic frequency spectrum data is used as an input into the training model;

[0126] S400, the sound data (i.e. the logarithmic frequency spectrum data) is input into the training model, and the training model outputs the brushing activity area where the brush head of the electric toothbrush is located at any time, thereby realizing the recognition of the brushing activity.

[0127] ​Further, in the step S400 of the above embodiment 3, in the process of outputting the brushing activity area where the brush head of the electric toothbrush is located at any time by the trained model, the obtained is a probability value, that is, the sound data at any time may contain information of multiple brushing activity areas, the trained model gives different probability values to all brushing activity areas related to the time and outputs, and the brushing activity area to which the sound data at the time finally belongs can be determined according to the principle of maximum probability value, so as to realize accurate identification of the brushing activity; further, in the step S300 of the above embodiment 3, it is determined by the sound source positioning that the sound data at the time belongs to the upper jaw or the lower jaw, and the accuracy of the brushing activity identification is further improved by the coordination of the trained model and the sound source positioning.

[0128] It should be noted that, in order to improve the accuracy of the brushing activity identification, the present application further has the following requirements for the brushing state of the user and the position of the sound collecting device in the data collection process (such as the step S300 of the above embodiment 3): (1) the user's lips are open during brushing, and the brushing action should avoid blocking or blocking the sound source transmission as much as possible; (2) the distance between the sound collecting device and the brush head of the electric toothbrush is kept at about 60 cm, which can be 55 cm, 57 cm, 60 cm, 62 cm or 65 cm; (3) the oral cavity is aligned with the center position of the sound collecting device as much as possible; (4) in addition to the sound of the electric toothbrush, the interference of other sounds is avoided as much as possible.

[0129] In a second aspect, the present embodiment provides an acoustic-based brushing activity identification system for executing the above acoustic-based brushing activity identification method, please refer to Figure 3 The brushing activity identification system comprises:

[0130] A brushing activity area division module is configured to divide the tooth surface of a human body into a plurality of brushing activity areas.

[0131] A neural network training module is configured to train a convolutional neural network by collecting sound signals of an electric toothbrush during brushing to obtain a trained model.

[0132] A data collection module is configured to collect sound data of the electric toothbrush during brushing.

[0133] An identification module is configured to input the sound data into the trained model, and the trained model outputs the brushing activity area where the brush head of the electric toothbrush is located at any time.

[0134] A sound source positioning module is configured to obtain the difference between the distances from the first microphone and the second microphone to the brush head of the electric toothbrush, and determine whether the brush head is in the upper jaw or the lower jaw according to the difference between the distances.

[0135] In the description of the above-mentioned embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0136] The above description is merely that of specific embodiments of the application, but the protection scope of the application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the application, which should be encompassed in the protection scope of the application. Therefore, the protection scope of the application should be subject to the protection scope of the claims.

Claims

1. An acoustic-based brushing activity recognition method, characterized in that, Comprising: S100, dividing the tooth surface of the human body into a plurality of brushing activity zones; the brushing activity zones include left upper buccal surface, left lower buccal surface, right upper buccal surface, right lower buccal surface, middle upper lip surface, middle lower lip surface, left upper occlusal surface, left lower occlusal surface, right upper occlusal surface, right lower occlusal surface, and idling; S200, training a convolutional neural network by collecting sound signals of the electric toothbrush during brushing to obtain a trained model; the convolutional neural network includes three convolutional layers and two fully connected layers; one of the two fully connected layers has 1024 neurons, and the other has 512 neurons; S300, collecting sound data of the electric toothbrush during brushing; converting the sound data from time domain signal to frequency spectrum signal, which is the frequency spectrum data; S400, inputting the frequency spectrum data into the trained model, and the trained model outputs the brushing activity zone where the brush head of the electric toothbrush is located at any time, realizing the recognition of brushing activity; Wherein, the convolutional neural network takes the sound signal as input, and each brushing activity zone of the sound signal is corresponded through the fully connected layer; In step S300, the sound data of the electric toothbrush during brushing is collected by using a sound collecting device, the sound collecting device has a first microphone and a second microphone, and when collecting the sound data, the first microphone is above and the second microphone is below; The difference between the distances from the first microphone and the second microphone to the brush head of the electric toothbrush is obtained; the brush head is determined to be on the upper jaw or the lower jaw according to the difference between the distances; the sound signal emitted by the brush head is transmitted to the first microphone and the second microphone, when the sound signal reaches the first microphone first and then reaches the second microphone, the brush head is on the upper jaw, and when the sound signal reaches the second microphone first and then reaches the first microphone, the brush head is on the lower jaw; during the collection of the sound data, the oral cavity is aligned with the center position of the sound collecting device, and the oral cavity is in an open state.

2. The acoustic-based brushing activity recognition method of claim 1, wherein, The conversion of the sound data from time domain signal to frequency spectrum signal includes: Step 1, setting the sampling frequency of the sound data to 22.5kHz, and the size of the frame to 2 periods of sampling points; the sampling point number is the ratio of the sampling frequency to the rotating speed of the electric toothbrush; Step 2, adding a Hanning window to the frame data of the sound data; Step 3, performing discrete Fourier transform on the sound data to obtain frequency domain data; Step 4, taking the absolute value and squaring the frequency domain data to obtain the frequency spectrum data.

3. The acoustic-based brushing activity recognition method of claim 2, wherein, After obtaining the frequency spectrum data, it further includes: Step 5, performing log amplitude transformation on the frequency spectrum data to obtain logarithmic frequency spectrum data, which is used for input into the trained model.

4. The acoustic-based brushing activity recognition method of claim 3, wherein, During the process of adding a Hanning window to the frame data of the sound data, the overlap rate of the Hanning window is 50%.

5. An acoustic-based brushing activity recognition system, characterized in that, A brushing activity recognition system for executing the acoustic-based brushing activity recognition method of any one of claims 1-4, comprising: a brushing activity zone division module for dividing the tooth surface of the human body into a plurality of brushing activity zones; A neural network training module is configured to train a convolutional neural network by collecting sound signals of the electric toothbrush during tooth brushing to obtain a training model; A data collection module is configured to collect sound data of the electric toothbrush during tooth brushing; An identification module is configured to input the sound data into the training model, and the training model outputs a tooth brushing activity area where the brush head of the electric toothbrush is located at any moment; A sound source positioning module is configured to obtain a difference between distances from the first microphone and the second microphone to the brush head of the electric toothbrush, and determine whether the brush head is located on the upper jaw or the lower jaw according to the difference between the distances.

Images