Acoustic analysis and machine learning model-based lossless counting method for tablets in medicine bottle

Abstract

The invention discloses a lossless counting method and device for tablets in a medicine bottle based on acoustic analysis and a machine learning model. The device comprises a sound acquisition module, a signal analysis module and a result playback module. The method comprises the following steps: 1, collecting several common types of medicines, and recording sounds generated when the medicines shake in a medicine box; 2, processing the sound data; 3, training a machine learning model by using the processed data; 4, adjusting the initial parameters and repeating the above steps: when the blind person needs to use, using the lossless counting device to shake and collect sound signals; 6, calculating the standard deviation under each large class and quantity; 7, searching a group with the minimum standard deviation, wherein the quantity of the medicines of the type is the required quantity; 8, the blind person is informed of the remaining amount of traditional Chinese medicine in the box through a sounding device. According to the method, the quantity of the medicines left in the medicine bottle can be effectively judged by utilizing a machine learning method and a prepared model library.

Description

Technical Field

[0001] The invention relates to a non-destructive counting method for pills in a medicine bottle based on acoustic analysis and a machine learning model, and belongs to the fields of acoustic processing and machine learning. Background Art

[0002] For people who need to take medicine for a long time, they usually need to prepare medicine in advance. Normal people can rely on visual effects to estimate the amount of remaining medicine in the medicine bottle, but for the blind, they need to open the medicine box and pour out the medicine, and judge the amount of remaining medicine in the medicine box by touching them one by one. However, this will cause many inconveniences. The first is that it may contaminate the medicine, resulting in loss of efficacy or bacterial infection; the second is that it is difficult for the blind to put the poured medicine back into the bottle through the bottle mouth, which is time-consuming and laborious.

[0003] Announcement No. CN202740391U discloses a voice medicine box for the blind. By setting a voice chip, the voice chip stores information such as the name of the medicine, applicable symptoms, and how to take it. When the blind use the medicine, they touch the touch sensor on the box body or the box cover with their hands. The touch sensor sends a signal to the voice chip, and the voice chip sends the voice signal to the pronunciation device. The pronunciation device makes sounds of information such as the name of the medicine, applicable symptoms, and how to take it, but the voice medicine box for the blind will not announce the quantity in the medicine box.

[0004] Currently, there is no device on the market that can know the amount of remaining medicine in a medicine box / bottle. After investigation, it was found that most blind people can only roughly judge the remaining amount of medicine based on the sound made by shaking the medicine bottle, but this method cannot achieve accurate judgment. Many diseases require continuous and uninterrupted medication, so it is necessary to accurately know the remaining amount of medicine in order to prepare the medicine in time. Summary of the invention

[0005] In view of the defects of the above-mentioned prior art, the present invention proposes a method and device for non-destructive counting of pills in a medicine box based on acoustic analysis and machine learning models. By analyzing samples of different categories and numbers, multiple machine learning models are performed on the same sample, and the number of drugs to be tested can be accurately judged by calculating the standard deviation between the examples and different models. The present invention can be used by blind people to judge the number of remaining drugs in a medicine bottle.

[0006] The method for non-destructive counting of pills in a medicine box based on acoustic analysis and machine learning model proposed by the present invention includes:

[0007] S1: Collect the sound wave spectrum and noise spectrum of the sound produced when different types and quantities of drugs are shaken in the medicine box;

[0008] S2: Use spectrum subtraction to obtain the effective sound spectrum;

[0009] S3: Based on the linear weighting method, the time domain features and frequency domain features of the sound signal are parameterized. The time domain features include short-time energy spectrum (STES) and short-time zero crossing rate (STZCR); the frequency domain features include short-time Fourier transform (STFT), Mel-frequency cepstral coefficients (MFCC), and linear prediction complex cepstral coefficients (LPCC).

[0010] The linear weighted formula is as follows, where each coefficient is trained to obtain the optimal solution.

[0011] MixFeature=α*STES+β*STZCR+γ*STFT+δ*MFCC+ω*LPCC

[0012] S4: The feature parameter combination is reduced in dimension based on the principal component analysis method (PCA) and processed into a 128*128*3 two-dimensional image format data, which is convenient for being sent to the convolutional neural network (CNN) model for processing;

[0013] S5: Use convolutional neural network (CNN) to perform machine learning training on the input data.

[0014] The data was first preprocessed and the file names were named using ij to label them, where i is the number of drugs in the bottle / box and j is the number of experiments.

[0015] In the CNN model, data with the same i are divided into a group. The input of CNN is a wavelet scalogram image of size 128*128 with three color channels (RGB). Each image is represented as a 3D tensor of shape (128,128,3), where the first two dimensions represent the spatial size of the image and the third dimension represents the color channel.

[0016] CNN first applies a series of convolution operations to extract features from the input image. Each convolution layer consists of a set of learnable filters (or kernels) that are convolved with the input image to generate a feature map. The filter is a small matrix (e.g. 3*3) that slides over the input image, computing the dot product between the filter value and the corresponding image value.

[0017] In order to enable machine learning to learn more complex models, we introduced an activation function ReLU, the function is as follows:

[0018] ReLU(x)=max(0,x)

[0019] The ReLU function introduces nonlinearity into the model, enabling it to learn more complex patterns. In the data, it ensures that only positive eigenvalues ​​are passed to the next layer, while setting negative eigenvalues ​​to zero.

[0020] After the convolution and activation operations, a pooling layer is applied to reduce the size of the spatial feature map. In the solution of the present invention, the pooling operation is maximum pooling, which selects the maximum value within a small window (e.g., 2×2) of the feature map. This downsampling operation reduces the computational complexity and provides translation invariance for small displacements in the input image. The maximum pooling expression is as follows:

[0021] P(x,y)=max{I(x+i,y+j)|i,j∈[0,p-1]}

[0022] Where P(x,y) is the pooled feature map and p×p is the size of the pooling window. After several convolutional and pooling layers, the resulting feature map is flattened into a one-dimensional vector and passed to a fully connected (dense) layer. At this stage, each neuron in the fully connected layer is connected to all neurons in the previous layer, enabling the network to learn high-level features extracted by the non-linear combination of convolutional layers. The last layer in a CNN is a fully connected layer with 11 neurons, corresponding to 11 neurons. A softmax activation function is applied to the output of this layer to generate class probabilities:

[0023]

[0024] Where zi is the input to the ith neuron and K is the total number of categories (11 in this case). The softmax function ensures that the output is a probability distribution where the sum of all probabilities equals 1. The category with the highest probability is selected as the predicted category for the input image. This function is optimized using the Adam optimizer to get the final model.

[0025] S6: Adjust the initial parameters and repeat S3 to S5 multiple times. We can get different corresponding models of the sound and quantity of different types of drugs when they are shaken in the medicine bottle.

[0026] S7: In the prediction stage, when we need to explore the amount of medicine left in a medicine bottle, we first shake it by hand and collect sound signals through the microphone of our non-destructive counting device;

[0027] S8: Use spectral subtraction to obtain pure sound data;

[0028] S9: Bring the input data we get into all models;

[0029] S10: Calculate the standard deviation between different quantities of the same type of drug, and take the group with the smallest standard deviation. The quantity corresponding to this group is the required quantity.

[0030] S11: The speaker of our device is used to inform the blind person of the amount of medicine left in the bottle.

[0031] The beneficial effects of the present invention are:

[0032] By proposing a non-destructive counting method and device based on acoustic analysis and machine learning models, combined with traditional acoustics and convolutional neural networks, the machine learning method and the pre-prepared model library can be used to effectively determine the amount of remaining medicine in the medicine bottle when the amount of remaining medicine is small, and a relatively accurate judgment is achieved. This solves the problem that it is difficult for blind people to estimate the amount of remaining medicine in the medicine bottle when taking medicine, and avoids the problem of secondary contamination of medicines caused by touching the medicine by hand. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0034] Figure 1 It is a schematic diagram of the structure of the non-destructive counting device for tablets in medicine bottles based on acoustic analysis and machine learning models proposed by the present invention.

[0035] Figure 2 It is a flow chart of a lossless counting method provided by an embodiment of the present invention.

[0036] Figure 3 It is a schematic diagram of the convolutional neural network structure provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0038] Embodiment 1:

[0039] This embodiment provides a method for non-destructive counting of tablets in a medicine bottle based on acoustic analysis and machine learning models. The method is implemented based on a non-destructive counting device, such as Figure 1 As shown, the lossless technical device includes a sound collection module composed of a microphone for collecting the sound of the medicine in the bottle colliding when it is shaken by hand, a signal analysis module composed of a computing chip for judging the amount of remaining medicine in the medicine bottle / medicine box in combination with the sound signal, and a result playback module composed of a speaker for notifying the blind person of the amount of remaining medicine in the medicine bottle / medicine box after obtaining the result.

[0040] like Figure 2 As shown in the figure, the non-destructive counting method of pills in medicine bottles based on acoustic analysis and machine learning model includes:

[0041] S1: Collect the sound wave spectrum and noise spectrum of the sound produced when different types and quantities of drugs are shaken in medicine bottles / medicine boxes;

[0042] S2: Use spectrum subtraction to obtain the effective sound spectrum;

[0043] S3: Based on the linear weighting method, the time domain features and frequency domain features of the effective sound spectrum are parameter-combined to obtain a combined feature (MixFeature).

[0044] Among them, the time domain features include: short-time energy spectrum (STES), short-time zero-crossing rate (STZCR); the frequency domain features include: short-time Fourier transform (STFT), Mel-frequency cepstral coefficients (MFCC), linear prediction complex cepstral coefficients (LPCC).

[0045] The linear weighted formula is as follows, where each coefficient is trained to obtain the optimal solution.

[0046] MixFeature=α*STES+β*STZCR+γ*STFT+δ*MFCC+ω*LPCC

[0047] S4: The combined features are reduced in dimension based on the principal component analysis (PCA) method and processed into a 128*128*3 two-dimensional image format data, which is convenient for being sent to the convolutional neural network (CNN) model for processing.

[0048] S5: Use a convolutional neural network (CNN) model to perform machine learning training on the input data.

[0049] The data is first preprocessed and the file names are named using ij to mark them, where i is the number of drugs in the bottle / box and j is the number of experiments. In the CNN model, data with the same i are grouped into a group. The input to the CNN is an image of size 128*128 with three color channels (RGB). Each image is represented as a 3D tensor of shape (128,128,3), where the first two dimensions represent the spatial size of the image and the third dimension represents the color channel.

[0050] The CNN model consists of three layers of convolutional pooling layers, flattening layers, fully connected layers, and output layers, such as Figure 3 As shown in the figure, the CNN model first applies a series of convolution operations to extract features from the input image. Each convolution layer consists of a set of learnable filters (or kernels) that are convolved with the input image to generate a feature map. The filter is a small matrix (for example, 3*3) that slides over the input image and calculates the dot product between the filter value and the corresponding image value.

[0051] In order to enable machine learning to learn more complex models, this invention introduces an activation function ReLU, the function is as follows:

[0052] ReLU(x)=max(0,x).

[0053] The ReLU function introduces nonlinearity into the model, enabling it to learn more complex patterns. In the data, it ensures that only positive eigenvalues ​​are passed to the next layer, while setting negative eigenvalues ​​to zero. After the convolution and activation operations, a pooling layer is applied to reduce the size of the spatial feature map. The pooling operation used in the present invention is maximum pooling, which selects the maximum value within a small window (e.g., 2×2) of the feature map. This downsampling operation reduces computational complexity and provides translation invariance for small displacements in the input image. The maximum pooling expression is as follows:

[0054] P(x,y)=max{I(x+i,y+j)|i,j∈[0,p-1]}

[0055] Where P(x,y) is the pooled feature map and p×p is the size of the pooling window. After several convolutional and pooling layers, the resulting feature map is flattened into a one-dimensional vector and passed to a fully connected (dense) layer. At this stage, each neuron in the fully connected layer is connected to all neurons in the previous layer, allowing the network to learn high-level features extracted by the non-linear combination of convolutional layers. The last layer in a CNN is a fully connected layer with 11 neurons, corresponding to 11 neurons.

[0056] A softmax activation function is applied to the output of this layer to generate class probabilities:

[0057]

[0058] Where zi is the input to the ith neuron and K is the total number of categories (11 in this case). The softmax function ensures that the output is a probability distribution where the sum of all probabilities equals 1. The category with the highest probability is selected as the predicted category for the input sound wave spectrum. This function is optimized with the Adam optimizer to get the final model.

[0059] S6: Adjust the initial parameters and repeat S3 to S5 multiple times to obtain different corresponding models of the sound and quantity of multiple different types of drugs when they are shaken in medicine bottles / medicine boxes;

[0060] S7: In the prediction phase, the sound signal is collected by the microphone of the lossless counting device;

[0061] S8: Use spectral subtraction to obtain pure sound data;

[0062] S9: Bring the obtained clean sound data input data into all models;

[0063] S10: Calculate the standard deviation between different quantities of the same type of drug, and take the group with the smallest standard deviation. The quantity corresponding to this group is the required quantity.

[0064] S11: The speaker of the non-destructive counting device is used to inform the blind person of the amount of remaining medicine in the medicine bottle / box.

[0065] To verify the effectiveness of the method of the present invention, 22 sets of data were obtained by hand shaking using the above method. The number of drugs in the bottle was a random value between 1 and 10, generated by a random integer generator. The non-destructive counting device provided by the present invention was used to determine the number of drugs in the bottle, and then the box was opened for verification:

[0066] definition: The results are shown in Table 1, where predicted label is the model prediction label, true label is the true label, and custom loss is the custom loss function.

[0067] Table 1. Results of true labels, model predicted labels and custom loss functions

[0068]

[0069] According to Table 1, the accuracy of counting the number of drugs in the medicine bottles obtained by the non-destructive counting method provided by the present invention is as high as 95.45%, and the standard deviation of the custom loss is as low as 0.0298, which shows that the non-destructive counting device provided by the present invention has good stability.

[0070] In summary, the method provided in this example combines traditional acoustics with convolutional neural networks, and can effectively determine the amount of remaining medicine in the medicine bottle when the amount of remaining medicine is small by using machine learning methods and a pre-prepared model library. Our device also effectively achieved our goal and achieved a relatively accurate judgment.

[0071] Some steps in the embodiments of the present invention may be implemented using software, and the corresponding software program may be stored in a readable storage medium, such as a CD or a hard disk.

[0072] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A non-destructive counting method for tablets in medicine bottles based on acoustic analysis and machine learning models, characterized in that: The method comprises: Step S1: collecting sound wave spectra and noise spectra of the sounds made when different types and quantities of drugs are shaken in medicine bottles / medicine boxes; Step S2: Obtaining an effective sound spectrum by spectral subtraction; Step S3: performing parameter combination on the time domain features and frequency domain features of the effective sound spectrum based on a linear weighting method; Step S4: Reduce the dimension of the characteristic parameter combination based on the principal component analysis method and process it into a 128*128*3 input data; Step S5: Use the convolutional neural network model to perform machine learning training on the input data to obtain a trained model so that the amount of medicine can be judged based on the sound made when the medicine is shaken in the medicine bottle / medicine box.

2. The method according to claim 1, characterized in that The time domain features in step S2 include: short-time energy spectrum and short-time zero-crossing rate; the frequency domain features include: short-time Fourier transform, Mel cepstral coefficients and linear prediction complex cepstral coefficients.

3. The method according to claim 2, characterized in that The convolutional neural network model includes three layers of convolutional pooling layers, flattening layers, fully connected layers and output layers; in the convolutional pooling layer, a series of convolution operations are first applied to extract features from the input image, and each convolutional layer consists of a group of learnable filters; secondly, the introduced activation function ReLU is used to ensure that only positive eigenvalues ​​are passed to the next layer, while setting the negative eigenvalues ​​to zero; after the convolution and activation operations, the pooling layer is applied to reduce the size of the spatial feature map.

4. The method according to claim 3, characterized in that The flattening layer is used to flatten the obtained feature map into a one-dimensional vector and pass it to the fully connected layer. Each neuron in the fully connected layer is connected to all neurons in the previous layer, so that the network can learn the high-level features extracted by the nonlinear combination convolution layer; the output layer uses the softmax activation function to obtain the generated class probability and selects the class with the highest probability as the output result.

5. The method according to claim 4, characterized in that Before the step S5 uses the convolutional neural network model to perform machine learning training on the input data, the step also includes: The input data is preprocessed and classified, wherein data with the same number are grouped together.

6. A non-destructive counting device for tablets in medicine bottles based on acoustic analysis and machine learning models, characterized in that: The device includes a sound collection module for collecting the sound produced by the collision of medicines in the medicine bottle / medicine box, a signal analysis module composed of a computing chip for judging the amount of remaining medicines in the medicine bottle / medicine box in combination with the sound signal, and a result playback module for notifying the blind person of the amount of remaining medicines in the medicine bottle / medicine box after obtaining the result; the computing chip includes the execution steps of the method described in any one of steps 1-5 above.

7. The non-destructive counting device according to claim 6, characterized in that: The sound collection module is implemented by a microphone.

8. The non-destructive counting device according to claim 6, characterized in that: The result playback module is implemented using a loudspeaker.

Images