Tooth brushing activity identification method and system based on acoustics

Abstract

The invention discloses a tooth brushing activity recognition method and system based on acoustics, relates to the technical field of sound recognition, and is used for solving the technical problems that in the process of recognizing a tooth brushing area in the prior art, due to the change of the head action of a user, the tooth brushing area is misjudged, and the tooth brushing activity cannot be accurately recognized. The tooth brushing activity identification method comprises the following steps: dividing the tooth surface of a human body into a plurality of tooth brushing activity areas; the method comprises the following steps: training a convolutional neural network by collecting sound signals of an electric toothbrush in a tooth brushing process to obtain a training model; collecting sound data of the electric toothbrush in a tooth brushing process; the sound data are input into a training model, the training model outputs a tooth brushing activity area where the brush head of the electric toothbrush is located at any moment, and tooth brushing activity recognition is achieved; the technical scheme of the invention is used for providing a tooth brushing activity identification method and system based on acoustics.

Description

Technical Field

[0001] The present invention relates to the technical field of sound recognition, and more specifically, to an acoustic-based brushing activity recognition method and system. Background Art

[0002] Oral health is closely related to overall health. The correct way of brushing teeth is the key to maintaining good oral hygiene and preventing dental caries and periodontal diseases. However, many people have problems such as improper posture, insufficient or excessive brushing time during daily brushing, resulting in poor brushing effect and causing oral health problems.

[0003] With the development of intelligent toothbrushing technology, methods for identifying brushing areas have emerged. The method for identifying a brushing area refers to the technology in which a toothbrushing device automatically identifies the brushing area. In the prior art, the identification of the brushing area mainly obtains the posture information of the toothbrushing device through a sensor on the toothbrushing device, and then identifies the user's brushing area according to the posture information. However, during the toothbrushing process, if the user's head movement changes, it will cause misjudgment of the brushing area and cannot accurately identify the brushing activity. Summary of the Invention

[0004] The purpose of the present invention is to provide an acoustic-based brushing activity recognition method and system, which are used to solve the technical problem that in the process of identifying the brushing area by the existing brushing area recognition method, due to the change of the user's head movement, misjudgment of the brushing area is caused, and accurate recognition of the brushing activity cannot be achieved. In view of this, the present invention is realized through the following solutions.

[0005] In a first aspect, the present invention provides an acoustic-based brushing activity recognition method, including: Dividing the tooth surface of a human body into multiple brushing activity areas; Training a convolutional neural network by collecting sound signals of an electric toothbrush during the toothbrushing process to obtain a training model; Collecting sound data of the electric toothbrush during the toothbrushing process; Inputting the sound data into the training model, and the training model outputs the brushing activity area where the brush head of the electric toothbrush is located at any moment, realizing the recognition of the brushing activity; Wherein, the convolutional neural network takes the sound signal as an input and corresponds the sound signal to each brushing activity area through a fully connected layer.

[0006] Compared with the prior art, the acoustic-based toothbrushing activity recognition method of the present invention is used to recognize toothbrushing activities, specifically, to recognize toothbrushing activity areas, and is suitable for commercially available electric toothbrushes. In the toothbrushing activity recognition method, the surface of human teeth is first divided into multiple toothbrushing activity areas, and a convolutional neural network is trained by collecting sound signals of electric toothbrushes during the toothbrushing process to obtain a training model, wherein the convolutional neural network uses sound signals as input, and corresponds the sound signals to each toothbrushing activity area through a fully connected layer. In the training model, the sound signals during the toothbrushing process correspond to the toothbrushing activity areas, that is, by inputting sound signals into the training model, the toothbrushing activity area corresponding to the sound signal at any time can be output; followed by the application of the training model, specifically, the collected sound data is input into the training model, and the training model can output the toothbrushing activity area where the brush head of the electric toothbrush is located at any time, thereby realizing the recognition of toothbrushing activities. It can be seen that the technical solution of the present invention is based on sound data (or sound signals) to identify brushing activities, and its recognition accuracy is basically not affected by changes in the user's head movements (it is based on sound signals rather than movements), and can achieve accurate recognition of brushing activities without the need to add or purchase additional wearable devices, saving costs. The above technical solution of the present invention solves the technical problem that in the process of identifying the brushing area, the existing brushing area recognition method causes misjudgment of the brushing area due to changes in the user's head movement, and cannot achieve accurate recognition of the brushing activity.

[0007] Furthermore, in the acoustic-based toothbrushing activity recognition method of the present invention, a sound collection device is used to collect the sound data of the electric toothbrush during the brushing process. The sound collection device has a first microphone and a second microphone. When the sound collection device collects the sound data, the first microphone is on top and the second microphone is on the bottom.

[0008] Furthermore, in the method for recognizing toothbrushing activities based on acoustics of the present invention, after collecting the sound data of the electric toothbrush during the toothbrushing process, the method further comprises: Obtaining a difference in distance from the first microphone to the brush head of the electric toothbrush and from the second microphone to the brush head of the electric toothbrush; It is determined whether the brush head is on the maxillary teeth or the mandibular teeth based on the difference in the distances.

[0009] Furthermore, in the acoustic-based brushing activity identification method of the present invention, the brushing activity area includes the left upper cheek surface, the left lower cheek surface, the right upper cheek surface, the right lower cheek surface, the middle upper lip surface, the middle lower lip surface, the left upper occlusal surface, the left lower occlusal surface, the right upper occlusal surface, the right lower occlusal surface, and idling.

[0010] Furthermore, in the method for recognizing toothbrushing activities based on acoustics of the present invention, the convolutional neural network comprises three convolutional layers and two fully connected layers; Among them, one of the fully connected layers in the fully connected layer pair has 1024 neurons, and the other has 512 neurons.

[0011] Furthermore, in the acoustic-based brushing activity recognition method of the present invention, after collecting the sound data of the electric toothbrush during the brushing process, it further includes: Converting the sound data from a time-domain signal into a spectrum signal, which is the spectrum data, and the spectrum data is used to be input into the training model.

[0012] Furthermore, in the acoustic-based brushing activity recognition method of the present invention, the conversion of the sound data from a time-domain signal into a spectrum signal includes: Step 1, set the sampling frequency of the sound data to 22.5 kHz, and the frame size to the number of sampling points for 2 cycles; the number of sampling points is the ratio of the sampling frequency to the rotation speed of the electric toothbrush; Step 2, add a Hanning window to the frame data of the sound data; Step 3, perform a discrete Fourier transform on the sound data to obtain frequency-domain data; Step 4, take the absolute value and square the frequency-domain data to obtain spectrum data.

[0013] Furthermore, in the acoustic-based brushing activity recognition method of the present invention, after obtaining the spectrum data, it further includes: Step 5, perform a log amplitude transformation on the spectrum data to obtain logarithmic spectrum data, and the logarithmic spectrum data is used to be input into the training model.

[0014] Furthermore, in the acoustic-based brushing activity recognition method of the present invention, during the process of adding a Hanning window to the frame data of the sound data, the overlap rate of the Hanning window is 50%.

[0015] In a second aspect, the present invention also provides an acoustic-based brushing activity recognition system for performing the above-mentioned acoustic-based brushing activity recognition method. The brushing activity recognition system includes: A brushing activity area division module for dividing the tooth surface of a human body into multiple brushing activity areas; A neural network training module for training a convolutional neural network by collecting the sound signals of the electric toothbrush during the brushing process to obtain a training model; A data collection module for collecting the sound data of the electric toothbrush during the brushing process; An identification module for inputting the sound data into the training model, and the training model outputs the brushing activity area where the brush head of the electric toothbrush is located at any moment; A sound source localization module, configured to obtain the difference in the distances from the first microphone and the second microphone to the brush head of the electric toothbrush, and determine whether the brush head is on the upper teeth or the lower teeth according to the difference in the distances.

[0016] Compared with the prior art, the beneficial effects of the acoustic-based brushing activity recognition system of the present invention are the same as those of the acoustic-based brushing activity recognition method described in the above technical solution, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present invention, and constitute a part of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention, and do not constitute an improper limitation to the present invention. In the drawings: Figure 1 It is a schematic flow chart of an acoustic-based brushing activity recognition method of the present invention; Figure 2 It is a schematic flow chart of another acoustic-based brushing activity recognition method of the present invention; Figure 3 It is a schematic composition diagram of an acoustic-based brushing activity recognition system of the present invention; Figure 4 It is a schematic diagram of sound data acquisition in Embodiment 3 of the present invention; Figure 5 It is a schematic diagram of sound data processing in Embodiment 3 of the present invention; Figure 6 It is a schematic diagram of frame length comparison of electric toothbrushes with different vibration frequencies in the present invention; Figure 7 It is a schematic diagram showing each brushing activity area after dividing the tooth surface in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention clearer, the present invention will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0019] It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it can be directly on the other element or indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element.

[0020] In addition, the terms "first" and "second" are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, "a plurality of" means two or more, unless otherwise specifically defined. "Several" means one or more, unless otherwise specifically defined.

[0021] With the development of intelligent toothbrushing technology, methods for identifying the brushing area have emerged. The method for identifying the brushing area refers to the technology by which a toothbrushing device automatically identifies the brushing area. In the prior art, the identification of the brushing area mainly obtains the posture information of the toothbrushing device through a sensor on the toothbrushing device, and then identifies the user's brushing area based on the posture information. However, during the toothbrushing process, if the user's head movement changes, it will cause misjudgment of the brushing area and the accurate identification of the toothbrushing activity cannot be achieved.

[0022] Please refer to Figure 1 , to solve the above technical problems, in a first aspect, the present invention provides an acoustic-based method for identifying a toothbrushing activity, including: S100, dividing the tooth surface of a human body into a plurality of toothbrushing activity areas; S200, training a convolutional neural network by collecting sound signals of an electric toothbrush during the toothbrushing process to obtain a training model; S300, collecting sound data of the electric toothbrush during the toothbrushing process; S400, inputting the sound data into the training model, and the training model outputs the toothbrushing activity area where the brush head of the electric toothbrush is located at any moment, so as to realize the identification of the toothbrushing activity; wherein, the convolutional neural network takes the sound signal as an input and corresponds the sound signal to each toothbrushing activity area through a fully connected layer.

[0023] In the case of adopting the above technical solution, the acoustic-based brushing activity recognition method of the present invention is used to recognize the brushing activity. Specifically, it is used to recognize the brushing activity area, and it is applicable to commercially available electric toothbrushes. In this brushing activity recognition method, the tooth surface of the human body is first divided into multiple brushing activity areas, and a convolutional neural network is trained by collecting the sound signals of the electric toothbrush during the brushing process to obtain a training model. Among them, the convolutional neural network takes the sound signal as the input, and through the fully connected layer, the sound signal is corresponding to each brushing activity area. In this training model, the sound signal during the brushing process is corresponding to the brushing activity area. That is to say, when the sound signal is input into this training model, the brushing activity area corresponding to the sound signal at any moment can be output; then comes the application of this training model. Specifically, the collected sound data is input into the training model, and this training model can output the brushing activity area where the brush head of the electric toothbrush is located at any moment, so as to realize the recognition of the brushing activity. It can be seen that the technical solution of the present invention is based on sound data (or sound signals) to recognize the brushing activity, and its recognition accuracy is basically not affected by the change of the user's head movement (it is based on sound signals rather than movements), and it can accurately recognize the brushing activity. At the same time, there is no need to additionally add or purchase wearable devices, which saves costs. Through the above technical solution of the present invention, the technical problem that in the process of recognizing the brushing area by the existing brushing area recognition method, due to the change of the user's head movement, the brushing area is misjudged and the accurate recognition of the brushing activity cannot be realized is solved.

[0024] Further, please refer to Figure 2 , another acoustic-based brushing activity recognition method of the present invention includes: Step 1, divide the tooth surface of the human body into multiple brushing activity areas; Step 2, train a convolutional neural network by collecting the sound signals of the electric toothbrush during the brushing process to obtain a training model; Step 3, collect the sound data of the electric toothbrush during the brushing process; Step 4, input the sound data into the training model, and the training model outputs the brushing activity area where the brush head of the electric toothbrush is located at any moment to realize the recognition of the brushing activity; Step 5, determine whether the brush head is on the upper jaw teeth or the lower jaw teeth according to the sound source localization.

[0025] In the case of adopting the above technical solution, the present invention uses a sound acquisition device to collect the sound data of the electric toothbrush during the brushing process. The sound acquisition device has a first microphone and a second microphone. When collecting the sound data, the first microphone is above and the second microphone is below. After collecting the sound data of the electric toothbrush during the brushing process, obtain the difference in the distances from the first microphone and the second microphone to the brush head of the electric toothbrush, and determine whether the brush head is on the upper jaw teeth or the lower jaw teeth according to the difference in the distances, that is, determine whether the brush head is on the upper jaw teeth or the lower jaw teeth according to sound source localization ( Figure 2 Step 5 in

[0026] Secondly, the present invention also provides an acoustic-based brushing activity recognition system for performing the above acoustic-based brushing activity recognition method. Please refer to Figure 3 This brushing activity recognition system includes: A brushing activity area division module for dividing the tooth surface of the human body into multiple brushing activity areas; A neural network training module for training a convolutional neural network by collecting the sound signals of the electric toothbrush during the brushing process to obtain a training model; A data acquisition module for collecting the sound data of the electric toothbrush during the brushing process; An identification module for inputting the sound data into the training model, and the training model outputs the brushing activity area where the brush head of the electric toothbrush is located at any moment; A sound source localization module for obtaining the difference in the distances from the first microphone and the second microphone to the brush head of the electric toothbrush, and determining whether the brush head is on the upper jaw teeth or the lower jaw teeth according to the difference in the distances.

[0027] In the case of adopting the above technical solution, in the acoustic-based brushing activity recognition system of the present invention, the above acoustic-based brushing activity recognition method can be efficiently executed through the cooperation of each module.

[0028] To better understand the present invention, the content of the present invention will be further clarified below in conjunction with specific embodiments, but the content of the present invention is not limited to the following embodiments.

[0029] Embodiment 1 This embodiment provides an acoustic-based brushing activity recognition method, including: Step 1, divide the tooth surface of the human body into multiple brushing activity areas; Step 2, train a convolutional neural network by collecting the sound signals of the electric toothbrush during the brushing process to obtain a training model; wherein, the convolutional neural network takes the sound signal as input and corresponds the sound signal to each brushing activity area through a fully connected layer; Step 3, collect the sound data of the electric toothbrush during the brushing process; Step 4, input the sound data into the training model, and the training model outputs the brushing activity area where the brush head of the electric toothbrush is located at any moment, so as to realize the recognition of the brushing activity.

[0030] Embodiment 2 This embodiment provides an acoustic-based brushing activity recognition method, including: S100, divide the tooth surface of the human body into multiple brushing activity areas; please refer to Figure 7 , in this embodiment, the brushing activity areas are divided into the following 11 categories. Specifically, the tooth surface is divided into: Left Upper Buccal (LUB), Left Lower Buccal (LLB), Right Upper Buccal (RUB), Right Lower Buccal (RLB), Mid Upper Labial (MUL), Mid Lower Labial (MLL), Left Upper Occlusal (LUO), Left Lower Occlusal (LLO), Right Upper Occlusal (RUO), Right Lower Occlusal (RLO) 10 tooth surfaces, plus the brushing activity of idling (IDL), a total of 11 brushing activity areas; S200, train a convolutional neural network by collecting the sound signals of the electric toothbrush during the brushing process to obtain a training model; wherein, the convolutional neural network takes the sound signals as input and corresponds the sound signals to each brushing activity area through a fully connected layer; The convolutional neural network includes three convolutional layers and two fully connected layers; wherein, one of the two fully connected layers has 1024 neurons and the other has 512 neurons; S300, use a sound collection device to collect the sound data of the electric toothbrush during the brushing process. The sound collection device has a first microphone and a second microphone. When the sound collection device collects the sound data, the first microphone is on the top and the second microphone is on the bottom; Further, step S300 includes: S301, obtain the difference in the distances from the first microphone and the second microphone to the brush head of the electric toothbrush; S302. Determine whether the brush head is on the upper teeth or the lower teeth according to the difference in the distance. S310. Convert the sound data from a time-domain signal into a frequency-spectrum signal, which is the frequency-spectrum data and is used for input into the training model. Further, step S310 includes: S311. Set the sampling frequency of the sound data to 22.5 kHz, and the size of a frame to the number of sampling points in two periods; the number of sampling points is the ratio of the sampling frequency to the rotation speed of the electric toothbrush. S312. Add a Hanning window to the frame data of the sound data, and the overlap rate of the Hanning window is 50%. S313. Perform a discrete Fourier transform on the sound data to obtain frequency-domain data. S314. Take the absolute value of the frequency-domain data and square it to obtain frequency-spectrum data. S315. Perform a log amplitude transformation on the frequency-spectrum data to obtain logarithmic frequency-spectrum data, which is used for input into the training model. S400. Input the sound data (logarithmic frequency-spectrum data) into the training model, and the training model outputs the brushing activity area where the brush head of the electric toothbrush is located at any moment, realizing the recognition of the brushing activity.

[0031] Embodiment 3 In a first aspect, this embodiment provides an acoustic-based brushing activity recognition method, including: S100. Divide the tooth surface of a human body into multiple brushing activity areas; please refer to Figure 7 , in this embodiment, the brushing activity areas are divided into the following 11 categories. Specifically, the tooth surface is divided into 10 tooth surfaces: the left upper buccal surface (LUB, Left Upper Buccal), the left lower buccal surface (LLB, Left Lower Buccal), the right upper buccal surface (RUB, Right Upper Buccal), the right lower buccal surface (RLB, Right Lower Buccal), the mid-upper labial surface (MUL, Mid Upper Labial), the mid-lower labial surface (MLL, Mid Lower Labial), the left upper occlusal surface (LUO, Left Upper Occlusal), the left lower occlusal surface (LLO, Left Lower Occlusal), the right upper occlusal surface (RUO, Right Upper Occlusal), and the right lower occlusal surface (RLO, Right Lower Occlusal). Plus the brushing activity of idling (IDL, Idling) when not brushing, there are a total of 11 brushing activity areas. S200, please refer to Figure 5 , and train a convolutional neural network by collecting the sound signals of the electric toothbrush during the brushing process to obtain a training model; specifically including: S201, collect data for each brushing activity area, that is: in an environment as quiet as possible, collect the sound data of the electric toothbrush during the brushing process through a smartphone. The brushing process of the electric toothbrush lasts for 180 seconds; each process covers the brushing activities of 10 tooth surfaces and the idling brushing activity. The collection time for each tooth surface is 15 seconds, and the collection time for idling is 5 seconds; S202, data processing, that is: a 15s audio (sound data) contains the whole process of the user brushing one tooth surface. During the whole process, the electric toothbrush will move slowly. The obtained audio is segmented into segments every 0.21 seconds, and each segment is saved as a new audio file to ensure that there is no time overlap between audio segments; S203, train the convolutional neural network, that is: input the audio files in step S202 into the convolutional neural network for training, and obtain a classification model after training. This classification model is the above-mentioned obtained training model; the convolutional neural network includes three convolutional layers and two fully connected layers; among them, one of the two fully connected layers has 1024 neurons, and the other has 512 neurons; S300, please refer to Figure 4 , collect the above-mentioned sound data of the electric toothbrush during the brushing process through a smartphone. This smartphone has a first microphone (Mic1) and a second microphone (Mic2). When the smartphone collects sound data, the first microphone (Mic1) is on top and the second microphone (Mic2) is at the bottom; specifically, when the user is brushing teeth normally, place the phone 60cm in front of the face (such as in front of a mirror), hold the electric toothbrush by hand, and the first microphone (Mic1, upper part) and the second microphone (Mic2, lower part) built into the smartphone collect the sound data (or audio data) generated by brushing teeth in real time. During this process, set the distances from the first microphone and the second microphone of the smartphone to the center of the oral cavity to be equal; Furthermore, step S300 includes: S301, obtain the difference in the distances from the first microphone (Mic1) and the second microphone (Mic2) to the brush head of the electric toothbrush. Specifically, as Figure 4 shown, the distance from the first microphone (Mic1) to the brush head is r1 , and the distance from the second microphone (Mic2) to the brush head is r2 ; the difference in the distances is obtained through the following formula: (1) where is the difference in the distances,r1 is the distance from the first microphone (Mic1) to the brush head, r2 is the distance from the second microphone (Mic2) to the brush head, is the difference between the time taken for the sound signal emitted by the brush head to reach the first microphone and the time taken for the sound signal emitted by the brush head to reach the second microphone, is the speed of sound propagation. In this embodiment, ; S302. Determine whether the brush head is on the upper teeth or the lower teeth according to the difference in the distances; Exemplarily, in this step, the sound signal emitted by the brush head can reach the first microphone and the second microphone. If the sound signal reaches the first microphone first and then the second microphone, it can be considered that the brush head is on the upper teeth. If the sound signal reaches the second microphone first and then the first microphone, it can be considered that the brush head is on the lower teeth; S310. Convert the sound data from a time-domain signal to a frequency-spectrum signal, and this frequency-spectrum signal is the frequency-spectrum data, which is used to be input into the training model; Further, step S310 includes: S311. Refer to Figure 6 , set the sampling frequency of the sound data to 22.5 kHz. In order to be compatible with the audio data of various electric toothbrushes, assume that the rotational speed of a certain electric toothbrush is , which represents the number of times the brush head of the electric toothbrush brushes in one minute. Then the number of brush strokes per second is ; Let the sampling frequency be sr , and the sampling frequency represents the number of samples per second. Then the number of sampling points included in each brush stroke of the brush head of the electric toothbrush is N , then there is: (2) Among them, N is the number of sampling points, sr is the sampling frequency, represents the rotational speed of the electric toothbrush; Further, each brush stroke of the brush head of the electric toothbrush represents half a motor cycle in a vibrating toothbrush. Then the number of sampling points included in one cycle is , and in terms of time, it is expressed in microseconds as: ; Among them, represents the number of sampling points included in one cycle, sr is the sampling frequency, represents the rotational speed of the electric toothbrush; Further, in order to ensure that at least one motor cycle is included in one frame, the size of the frame should be at least the number of sampling points of 2 cycles, that is, it satisfies ; The rotation speed of the existing vibrating electric toothbrush is between Among them, the number of sampling points in one cycle ranges from 160.36 to 55.125, so there is: (3) Among them, represents the size of an audio frame; represents the maximum value of the number of sampling points in one cycle of a vibrating electric toothbrush with a rotation speed of 16500 rpm to 48000 rpm; Since the smallest number greater than 320.72 and being a power of 2 is 512, therefore, 512 is selected as the frame length, which can adapt to vibrating electric toothbrushes with a rotation speed greater than ; S312, Add a Hanning window to the frame data of the sound data, and the overlap rate of the Hanning window is 50%; The addition of the Hanning window is achieved through the following formula: (4) Among them, represents the window function value obtained by the Hanning window, is the pi, is the total number of sample points inside the frame. In this embodiment , is the th sample point inside the frame; S313, Perform a discrete Fourier transform on the above formula (4) to obtain frequency domain data; The frequency domain data is represented by the following formula: (5) Among them, represents the frequency domain data of the sound signal, the time domain signal of the th sample point in any frame, is the natural constant, is the pi, is the frequency, is the time domain time; is the number of sampling points in any frame; is the imaginary unit; S314, Take the absolute value and square the frequency domain data in step S313 to obtain spectrum data; S315, Perform a log amplitude transform on the spectrum data to obtain log spectrum data, and the log spectrum data is used to be input into the training model; S400, Input the sound data (i.e., log spectrum data) into the training model, and the training model outputs the brushing activity area where the brush head of the electric toothbrush is located at any moment to achieve the recognition of the brushing activity.

[0032] Furthermore, in the above step S400 of the present embodiment 3, when the training model outputs the brushing activity zone where the brush head of the electric toothbrush is located at any moment, a probability value is obtained. That is to say, the sound data at any moment may contain information of multiple brushing activity zones at the same time. The training model assigns different probability values ​​to all brushing activity zones related to the moment, and outputs them. According to the principle of maximum probability value, the brushing activity zone to which the sound data at the moment finally belongs can be determined, thereby realizing accurate recognition of brushing activities. Furthermore, in step S300 of the present embodiment 3, whether the sound data at the moment belongs to the maxillary teeth or the mandibular teeth is determined through sound source localization. The accuracy of brushing activity recognition is further improved through the coordinated cooperation of the training model and sound source localization.

[0033] It should be noted that in order to improve the accuracy of identifying brushing activities, the present invention further makes the following requirements on the user's brushing status and the position of the sound collection device during the data collection process (such as S300 in the above-mentioned embodiment 3): (1) The user's lips are open during brushing, and the brushing action should try to avoid blocking or obstructing the transmission of the sound source; (2) The distance from the sound collection device to the brush head of the electric toothbrush is maintained at about 60 cm, which can be 55 cm, 57 cm, 60 cm, 62 cm or 65 cm; (3) The oral cavity is aligned with the center of the sound collection device as much as possible; (4) Except for the sound of the electric toothbrush, other sounds should be avoided as much as possible.

[0034] In a second aspect, this embodiment provides an acoustic-based tooth brushing activity recognition system for executing the acoustic-based tooth brushing activity recognition method, see Figure 3 , the toothbrushing activity recognition system comprises: A toothbrushing activity area division module is used to divide the tooth surface of a human body into multiple toothbrushing activity areas; A neural network training module is used to train a convolutional neural network by collecting sound signals of an electric toothbrush during brushing to obtain a training model; A data acquisition module is used to collect sound data of the electric toothbrush during the brushing process; A recognition module, for inputting the sound data into the training model, wherein the training model outputs the toothbrushing activity zone where the brush head of the electric toothbrush is located at any moment; The sound source localization module is used to obtain the difference in distance from the first microphone and the second microphone to the brush head of the electric toothbrush, and determine whether the brush head is on the maxillary teeth or the mandibular teeth according to the difference in distance.

[0035] In the description of the above embodiments, specific features, structures, materials or characteristics may be combined in a suitable manner in any one or more embodiments or examples.

[0036] As described above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the said claims.

Claims

1. A method for recognizing toothbrushing activity based on acoustics, characterized in that: include: Divide the tooth surface of the human body into multiple toothbrushing activity areas; The convolutional neural network is trained by collecting the sound signals of the electric toothbrush during the brushing process to obtain a training model; Collect sound data of electric toothbrushes during brushing; The sound data is input into the training model, and the training model outputs the brushing activity area where the brush head of the electric toothbrush is located at any time, so as to realize the recognition of the brushing activity; The convolutional neural network takes the sound signal as input and corresponds the sound signal to each toothbrushing activity area through a fully connected layer.

2. The method for recognizing toothbrushing activity based on acoustics according to claim 1, characterized in that: The sound data of the electric toothbrush during the brushing process is collected using a sound collection device, the sound collection device having a first microphone and a second microphone. When the sound collection device collects the sound data, the first microphone is on top and the second microphone is on the bottom.

3. The method for recognizing toothbrushing activity based on acoustics according to claim 2, characterized in that: After collecting the sound data of the electric toothbrush during the brushing process, the method further includes: Obtaining a difference in distance from the first microphone to the brush head of the electric toothbrush and from the second microphone to the brush head of the electric toothbrush; It is determined whether the brush head is on the maxillary teeth or the mandibular teeth based on the difference in the distances.

4. The method for recognizing toothbrushing activity based on acoustics according to claim 3, characterized in that: The toothbrushing active area includes the left upper cheek surface, the left lower cheek surface, the right upper cheek surface, the right lower cheek surface, the middle upper lip surface, the middle lower lip surface, the left upper occlusal surface, the left lower occlusal surface, the right upper occlusal surface, the right lower occlusal surface, and idling.

5. The method for recognizing toothbrushing activity based on acoustics according to claim 4, characterized in that: The convolutional neural network comprises three convolutional layers and two fully connected layers; Among them, one of the two fully connected layers has 1024 neurons and the other has 512 neurons.

6. The method for recognizing toothbrushing activity based on acoustics according to claim 5, characterized in that: After collecting the sound data of the electric toothbrush during the brushing process, the method further includes: The sound data is converted from a time domain signal into a spectrum signal, and the spectrum signal is spectrum data, and the spectrum data is used to be input into the training model.

7. The method for recognizing toothbrushing activity based on acoustics according to claim 6, characterized in that: The converting the sound data from a time domain signal into a spectrum signal comprises: Step 1, setting the sampling frequency of the sound data to 22.5kHz, and the frame size to the number of sampling points of 2 cycles; the number of sampling points is the ratio of the sampling frequency to the rotation speed of the electric toothbrush; Step 2, adding a Hanning window to the frame data of the sound data; Step 3, performing discrete Fourier transform on the sound data to obtain frequency domain data; Step 4: take the absolute value of the frequency domain data and square it to obtain spectrum data.

8. The method for recognizing toothbrushing activity based on acoustics according to claim 7, characterized in that: After obtaining the spectrum data, the method further includes: Step 5: Perform log amplitude transformation on the spectrum data to obtain logarithmic spectrum data, and the logarithmic spectrum data is used to input into the training model.

9. The method for recognizing toothbrushing activity based on acoustics according to claim 8, characterized in that: In the process of adding the Hanning window to the frame data of the sound data, the overlap rate of the Hanning window is 50%.

10. An acoustic-based toothbrushing activity recognition system, characterized in that: The tooth brushing activity recognition system is used to perform the acoustic-based tooth brushing activity recognition method according to any one of claims 1 to 9, and the tooth brushing activity recognition system comprises: A toothbrushing activity area division module is used to divide the tooth surface of a human body into multiple toothbrushing activity areas; A neural network training module is used to train a convolutional neural network by collecting sound signals of an electric toothbrush during brushing to obtain a training model; A data acquisition module is used to collect sound data of the electric toothbrush during the brushing process; A recognition module, for inputting the sound data into the training model, wherein the training model outputs the toothbrushing activity zone where the brush head of the electric toothbrush is located at any moment; The sound source localization module is used to obtain the difference in distance from the first microphone and the second microphone to the brush head of the electric toothbrush, and determine whether the brush head is on the maxillary teeth or the mandibular teeth according to the difference in distance.

Images