Snoring detection method, device, electronic device and computer readable storage medium

Abstract

The present application discloses a snoring detection method, device, electronic device and computer-readable storage medium, which relate to the field of data processing. The method comprises: recording the sound features of X sound clips of the target object into a data ring; performing Y target operations on the data ring, wherein each target operation is used to delete the head sound information in each sound clip corresponding to the data ring, and sequentially increment the sound information of the target duration at the end of each sound clip to obtain the updated X sound clips, and recording the sound features of the updated X sound clips in the data ring; normalizing the sound features recorded in the data ring to obtain Y groups of target sound features; and determining whether the sound emitted by the target object is snoring based on the Y+1 groups of target sound features. The present application solves the technical problem of low efficiency of snoring detection caused by the need for longer audio information in the prior art to determine whether the sound emitted by the target object is snoring.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a snoring detection method, device, electronic device and computer-readable storage medium. Background Art

[0002] Obstructive sleep apnea hypopnea syndrome is a common clinical disease, the main symptoms of which are snoring and apnea at night. Patients with obstructive sleep apnea hypopnea syndrome will experience symptoms such as daytime sleepiness and are very likely to develop various cardiovascular and cerebrovascular diseases, which seriously endangers the patient's physical and mental health.

[0003] In the diagnosis process of obstructive sleep apnea hypopnea syndrome, it is usually necessary to identify the exciting parts that cause the patient to snore (such as the neck, tongue root, chest cavity, lungs, etc.). At present, doctors often identify the exciting parts that cause snoring through the snoring audio produced by the patient during sleep. Specifically, by extracting features from the snoring audio, a certain type of feature of the snoring audio is obtained (such as MFCC (Mel-Frequency Cepstral Coefficients) features, GFCC (Gammatone feature cochleagram) features, frequency domain features), and the exciting parts are identified based on one type of features.

[0004] However, the above recognition method usually requires longer audio information to extract enough features, and then to ensure the accuracy of snoring judgment. However, in actual scenarios, since long-term audio information is not easy to obtain, it leads to the problem of low efficiency in snoring detection.

[0005] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0006] The present application provides a snoring detection method, device, electronic device and computer-readable storage medium to at least solve the technical problem in the prior art that a long audio message is required to determine whether the sound emitted by the target object is snoring, resulting in low snoring detection efficiency.

[0007] According to one aspect of the present application, a snoring detection method is provided, comprising: recording sound features of X sound clips of a target object into a data ring, and normalizing the sound features recorded in the data ring to obtain a first group of target sound features, wherein X is an integer greater than 1; after obtaining the first group of target sound features, performing Y target operations on the data ring in a serial manner, wherein each target operation is used to delete the head sound information in each sound clip corresponding to the data ring, and according to the sound sampling time, the sound information of the target duration is incremented at the end of each sound clip to obtain the updated X sound clips; the target operation is also used to record the sound features of the updated X sound clips in the data ring; wherein the duration of the head sound information is equal to the target duration; normalizing the sound features recorded in the data ring after each target operation is completed to obtain Y groups of target sound features, wherein Y is a positive integer; and determining whether the sound emitted by the target object is snoring according to the Y+1 groups of target sound features.

[0008] Optionally, determining whether the sound emitted by the target object is snoring based on Y+1 groups of target sound features includes: generating a label corresponding to each group of target sound features based on each group of target sound features according to the prior knowledge learned by the target model in the model training phase, wherein the label is a first label or a second label, the first label characterizing that the sound corresponding to the target sound feature is snoring, and the second label characterizing that the sound corresponding to the target sound feature is not snoring; wherein the training data of the target model includes historical sound features with known corresponding labels; determining whether the sound emitted by the target object is snoring based on the proportion of the number of target sound features with the first label in the Y+1 groups of target sound features.

[0009] Optionally, whether the sound emitted by the target object is snoring is determined based on the proportion of the number of target sound features of the first label in the Y+1 group of target sound features, including: when it is detected that the proportion of the number is greater than a preset threshold, determining that the sound emitted by the target object is snoring; when it is detected that the proportion of the number is less than or equal to the preset threshold, determining that the sound emitted by the target object is not snoring.

[0010] Optionally, the process of extracting the sound features of a sound clip includes: performing frame processing on each sound clip, wherein the frame processing is used to divide the sound clip into K arrays of equal data size, wherein K is an integer greater than 1; performing Fourier transform and Mel transform on the K arrays corresponding to the sound clip; and using the frequency amplitude obtained by Fourier transform and the target sound frequency obtained by Mel transform as the sound features of the sound clip.

[0011] Optionally, Fourier transform is performed on the K arrays corresponding to the sound clips, including: performing Fourier transform on the K arrays corresponding to the sound clips to obtain multiple transformed complex numbers; calculating the square of the modulus of the multiple transformed complex numbers to obtain the frequency amplitude of the sound clip.

[0012] Optionally, a Mel transform is performed on K arrays corresponding to each sound segment, including: setting a maximum frequency value and a minimum frequency value of the Mel transform; determining a maximum Mel coefficient according to the maximum frequency value of the Mel transform; determining a minimum Mel coefficient according to the minimum frequency value of the Mel transform; determining P target values ​​in a numerical interval formed by the minimum Mel coefficient and the maximum Mel coefficient, wherein the absolute value of the difference between each two adjacent target values ​​is the same value, wherein P is an integer greater than 1; calculating the frequency value corresponding to each target value; and according to the principle that the frequency value corresponding to each target value corresponds to the frequency spectrum order of the Fourier transform, selecting the frequency values ​​adjacent to the frequency spectrum order of the frequency value corresponding to each target value for weighted averaging calculation to obtain the target sound frequency of the sound segment.

[0013] Optionally, the sound features recorded in the data ring are normalized to obtain the target sound features, including: determining the maximum frequency amplitude from the frequency amplitudes of the X sound clips recorded in the data ring; normalizing the target sound frequencies of the X sound clips recorded in the data ring according to the maximum frequency amplitude, and using the normalization result as the target sound feature.

[0014] Optionally, according to the maximum frequency amplitude, the target sound frequencies of the X sound clips recorded in the data ring are normalized, and the normalized result is used as the target sound feature, including: converting the target sound frequency into a target decibel value according to a preset sound frequency value; setting the minimum decibel value and the maximum decibel value in the Fourier transform according to the preset sound frequency value; converting the target decibel value into a first integer value according to the minimum decibel value and the maximum decibel value; calculating the ratio of the first integer value to the maximum frequency amplitude to obtain a normalized value; converting the normalized value into a second integer value, wherein the number of bits of the first integer value is greater than the number of bits of the second integer value; and using the second integer value as the target sound feature.

[0015] According to another aspect of the present application, a snoring detection device is provided, comprising: a first processing unit, configured to record the sound features of X sound clips of a target object into a data ring, and normalize the sound features recorded in the data ring to obtain a first group of target sound features, wherein X is an integer greater than 1; a second processing unit, configured to perform Y target operations on the data ring in a serial manner after obtaining the first group of target sound features, wherein each target operation is used to delete the head sound information in each sound clip corresponding to the data ring, and to increase the sound information of the target duration in sequence at the end of each sound clip according to the sound sampling time, to obtain the updated X sound clips; the target operation is also used to record the sound features of the updated X sound clips in the data ring, wherein the duration of the head sound information is equal to the target duration; a third processing unit, configured to normalize the sound features recorded in the data ring after each target operation is completed, to obtain Y groups of target sound features, wherein Y is a positive integer; and a fourth processing unit, configured to determine whether the sound emitted by the target object is snoring according to the Y+1 groups of target sound features.

[0016] According to another aspect of the present application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, wherein when the computer program is executed, the device where the computer-readable storage medium is located executes the above-mentioned snoring detection method.

[0017] According to another aspect of the present application, an electronic device is also provided, wherein the electronic device includes one or more processors and a memory, the memory is used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors execute the above-mentioned snoring detection method.

[0018] In the present application, the sound features of X sound clips of the target object are first recorded in the data ring, and the sound features recorded in the data ring are normalized to obtain the first group of target sound features, where X is an integer greater than 1. Then, after obtaining the first group of target sound features, Y target operations are performed on the data ring in a serial manner, where each target operation is used to delete the head sound information in each sound clip corresponding to the data ring, and according to the sound sampling time, the sound information of the target duration is incremented at the end of each sound clip to obtain the updated X sound clips; the target operation is also used to record the sound features of the updated X sound clips in the data ring; wherein the duration of the head sound information is equal to the target duration. Finally, the sound features recorded in the data ring after each target operation are normalized to obtain Y groups of target sound features, where Y is a positive integer; according to the Y+1 group of target sound features, it is determined whether the sound emitted by the target object is snoring.

[0019] As can be seen from the above content, first of all, this application uses a data ring to store the sound features of X sound clips of the target object, so that all sound information does not need to be stored in real time, achieving the technical effect of reducing memory overhead. In addition, after obtaining the first set of target sound features, this application performs Y target operations on the data ring in a serial manner, wherein each target operation is used to delete the head sound information in each sound clip corresponding to the data ring, and according to the sound sampling time, the sound information of the target duration is sequentially increased at the end of each sound clip to obtain the updated X sound clips; the target operation is also used to record the sound features of the updated X sound clips in the data ring; wherein the duration of the head sound information is equal to the target duration.

[0020] For example, a deletion quantity M can be set. When the data ring is full, for each sound clip in the data ring (the duration can be recorded as , when performing target operations on the data ring, the beginning of each sound clip can be automatically deleted The sound characteristics of the sound information of the duration, while continuing to collect The sound characteristics of the sound information of the duration are obtained to obtain a new sound segment (the amount of audio data remains unchanged, but the required duration is ) sound features, after the sound features of the original X sound clips in the data ring are all updated, a new data ring can be obtained (essentially, the data ring stores the sound features of the updated X sound clips).

[0021] In this way, multiple sampling of a short-duration audio can be achieved, thereby obtaining sound features of more sound clips. Under the condition of the same audio duration, compared with the prior art, the technical solution of the present application can obtain richer sound features, thereby improving the detection efficiency and accuracy of snoring, and further solving the technical problem of low snoring detection efficiency caused by the prior art that a longer audio information is required to determine whether the sound emitted by the target object is snoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0023] Figure 1 is a flow chart of an optional snoring detection method according to an embodiment of the present application;

[0024] Figure 2 is a schematic diagram of an optional rapid snoring detection according to an embodiment of the present application;

[0025] Figure 3 is a schematic diagram of an optional sound processing method according to an embodiment of the present application;

[0026] Figure 4 is a schematic diagram of a target operation according to an embodiment of the present application;

[0027] Figure 5 is a schematic diagram of an optional snoring detection device according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] It should also be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) collected by this application are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data are in compliance with the relevant laws, regulations and standards of the relevant regions, necessary confidentiality measures are taken, and public order and good customs are not violated, and corresponding operation portals are provided for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or institutions. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or institution through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or institution.

[0031] According to an embodiment of the present application, an embodiment of a snoring detection method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0032] Figure 1 is a flow chart of an optional snoring detection method according to an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0033] Step S101, recording the sound features of X sound clips of the target object into a data ring, and normalizing the sound features recorded in the data ring to obtain a first set of target sound features.

[0034] Wherein, X is an integer greater than 1.

[0035] Optionally, in the present application, a data ring is designed to store the sound features of the sound clips, wherein the data ring stores the sound features of X sound clips at most.

[0036] Optionally, each sound clip can be divided into a two-dimensional array, where the length of the two-dimensional array can be recorded as 2N. The two-dimensional array can be split into two one-dimensional arrays, recorded as N1 and N2, where the length of each one-dimensional array is N. The memory can be divided into two areas to store two one-dimensional arrays respectively. For example, the one-dimensional array N1 is stored in the first area and the one-dimensional array N2 is stored in the second area. Assuming that the acquisition time of the one-dimensional array N1 is later than the acquisition time of the one-dimensional data N2, after the latest one-dimensional array is collected, the one-dimensional array N2 originally stored in the second area is deleted, and the one-dimensional array N1 originally stored in the first area is moved to the second area as a new one-dimensional array N2, and then the latest one-dimensional array is stored in the first area as a new one-dimensional array N1.

[0037] Optionally, after the sound features of X sound clips are filled in the data ring for the first time, the sound features currently recorded in the data ring may be normalized to obtain a first group of target sound features.

[0038] Step S102, after obtaining the first set of target sound features, performing Y target operations on the data ring in a serial manner.

[0039] In step S102, each target operation is used to delete the head sound information in each sound clip corresponding to the data ring, and according to the sound sampling time, the sound information of the target duration is sequentially increased at the end of each sound clip to obtain the updated X sound clips; the target operation is also used to record the sound features of the updated X sound clips in the data ring; wherein the duration of the head sound information is equal to the target duration.

[0040] Optionally, for the data ring, a preset deletion data amount M can be set. After the data ring is filled with the sound features of X sound clips, for each sound clip in the data ring (assuming that the total duration of the X sound clips is T, and the duration of each sound clip is T / X), when the target operation is performed, the sound information of M / X duration (corresponding to the above-mentioned target duration) at the beginning of each sound clip can be automatically deleted, and then the sound information of M / X duration is sequentially incremented at the end of each sound clip according to the sound sampling time. Since there are a total of X sound clips in the data ring, the updated X sound clips can be obtained in the end, where the total duration of the updated X sound clips is also T. Finally, the target operation can record the sound features of the updated X sound clips in the data ring.

[0041] In order to more clearly describe the above target operation, an example is given below for illustration.

[0042] Assume that the data ring stores the sound features of three sound clips, namely, sound clip 1 (collected at 0 seconds to 1 second), sound clip 2 (collected at 1 second to 2 seconds), and sound clip 3 (collected at 2 seconds to 3 seconds), for the first time.

[0043] Perform the first target operation: Assuming the target duration is 0.5 seconds, the updated sound clip 1 records the sound collected at 0.5 seconds to 1.5 seconds, the updated sound clip 2 records the sound collected at 1.5 seconds to 2.5 seconds, and the updated sound clip 3 records the sound collected at 2.5 seconds to 3.5 seconds. The sound features of the three updated sound clips are recorded in the data ring through the target operation.

[0044] Perform the second target operation: the sound clip 1 after the update records the sound collected at 1 second to 2 seconds, the sound clip 2 after the update records the sound collected at 2 seconds to 3 seconds, and the sound clip 3 after the update records the sound collected at 3 seconds to 4 seconds. The sound features of the three sound clips after the update are recorded in the data ring through the target operation.

[0045] As can be seen from the above content, after each target operation, the newly generated sound features will always be recorded in the data ring. In this way, multiple sampling of a short-duration audio can be achieved, thereby obtaining sound features of more sound segments. Under the same audio duration, compared with the prior art, the technical solution of this application can obtain richer sound features, thereby improving the detection efficiency and accuracy of snoring, thereby solving the technical problem of low snoring detection efficiency caused by the prior art that a longer audio information is required to determine whether the sound emitted by the target object is snoring.

[0046] Step S103, normalizing the sound features recorded in the data ring after each target operation is completed to obtain Y groups of target sound features, where Y is a positive integer.

[0047] Optionally, if Y target operations are performed, then after each target operation, the sound features recorded in the data ring must be normalized to obtain a set of target sound features. Therefore, after Y target operations, a total of Y sets of target sound features can be obtained.

[0048] Step S104: determining whether the sound emitted by the target object is snoring according to the Y+1 group of target sound features.

[0049] Optionally, the present application supports the use of a neural network model to determine whether the sound emitted by the target object is snoring according to Y+1 sets of target sound features. The present application also supports the use of data analysis or manual analysis to determine whether the sound emitted by the target object is snoring according to Y+1 sets of target sound features, for example, by comparing the similarity between the target sound features and known snoring features, and determining whether the sound emitted by the target object is snoring according to the similarity.

[0050] Based on the contents of the above steps S101 to S104, it can be known that in the present application, the sound features of the X sound clips of the target object are first recorded in the data ring, and the sound features recorded in the data ring are normalized to obtain the first group of target sound features, where X is an integer greater than 1. Then, after obtaining the first group of target sound features, the data ring is subjected to Y target operations in a serial manner, where each target operation is used to delete the head sound information in each sound clip corresponding to the data ring, and the sound information of the target duration is incremented at the end of each sound clip in accordance with the sound sampling time, to obtain the updated X sound clips; the target operation is also used to record the sound features of the updated X sound clips in the data ring; wherein the duration of the head sound information is equal to the target duration. Finally, the sound features recorded in the data ring after each target operation are normalized to obtain Y groups of target sound features, where Y is a positive integer; and whether the sound emitted by the target object is snoring is determined based on the Y+1 group of target sound features.

[0051] As can be seen from the above content, first of all, this application uses a data ring to store the sound features of X sound clips of the target object, so that all sound information does not need to be stored in real time, achieving the technical effect of reducing memory overhead. In addition, after obtaining the first set of target sound features, this application performs Y target operations on the data ring in a serial manner, wherein each target operation is used to delete the head sound information in each sound clip corresponding to the data ring, and according to the sound sampling time, the sound information of the target duration is sequentially increased at the end of each sound clip to obtain the updated X sound clips; the target operation is also used to record the sound features of the updated X sound clips in the data ring; wherein the duration of the head sound information is equal to the target duration.

[0052] For example, a deletion quantity M can be set. When the data ring is full, for each sound clip in the data ring (the duration can be recorded as ), when performing target operations on the data ring, the beginning of each sound clip can be automatically deleted. The sound characteristics of the sound information of the duration, while continuing to collect The sound characteristics of the sound information of the duration are obtained to obtain a new sound segment (the amount of audio data remains unchanged, but the required duration is ) sound features, after the sound features of the original X sound clips in the data ring are all updated, a new data ring can be obtained (essentially, the data ring stores the sound features of the updated X sound clips).

[0053] In this way, multiple sampling of a short-duration audio can be achieved, thereby obtaining sound features of more sound clips. Under the condition of the same audio duration, compared with the prior art, the technical solution of the present application can obtain richer sound features, thereby improving the detection efficiency and accuracy of snoring, and further solving the technical problem of low snoring detection efficiency caused by the prior art that a longer audio information is required to determine whether the sound emitted by the target object is snoring.

[0054] In an optional embodiment, the prior knowledge learned by the target model in the model training phase can be used to generate labels corresponding to each group of target sound features according to each group of target sound features, wherein the label is a first label or a second label, the first label indicates that the sound corresponding to the target sound feature is snoring, and the second label indicates that the sound corresponding to the target sound feature is not snoring; wherein the training data of the target model includes historical sound features with known corresponding labels. Then, according to the proportion of the number of target sound features with the first label in the number of target sound features in the Y+1 group, it is determined whether the sound emitted by the target object is snoring.

[0055] Optionally, first, you can collect snoring sound data from different groups of people for a certain period of time (for example, 10 hours) through multiple channels, as well as other sounds that may appear at night for a certain period of time, such as flushing sounds, talking sounds, etc. These sounds may interfere with the model's judgment of snoring sounds. After sorting out these data, you can start training the model. This application supports the use of the standard gradient descent method as a training method, and uses the cross entropy loss as a loss function. In the end-to-end self-training process, the convolutional neural network can learn how to automatically extract the most important features for identifying snoring sounds from MFCC, and the fully connected layer can learn how to effectively classify these features. Finally, the trained deep learning model is used as the target model for identifying and detecting snoring sounds.

[0056] Among them, the network structure of the target model can refer to Table 1:

[0057] Table 1

[0058]

[0059] Optionally, when determining whether the sound emitted by the target object is snoring based on the proportion of the number of target sound features of the first label in the Y+1 group of target sound features, when it is detected that the proportion is greater than a preset threshold, the sound emitted by the target object is determined to be snoring; when it is detected that the proportion is less than or equal to the preset threshold, it is determined that the sound emitted by the target object is not snoring.

[0060] Optionally, the present application uses a threshold queue method to determine whether the sound emitted by the target object is snoring. For example, a queue is set to store the results of real-time judgment. Assuming M=1 second, T=2 seconds, and a judgment cycle of 1 minute, there will be 59 detection results in one minute. Among them, each detection result represents the label corresponding to a group of target sound features generated based on a group of target sound features. If the number of times it is judged as snoring is q times (that is, the number of target sound features of the first label is q), the proportion is q / 59, and its value is recorded as Q. Set a threshold ,like , then it is determined that the sound emitted by the target object is snoring; if , it is determined that the sound emitted by the target object is not snoring.

[0061] By using the threshold queue method to determine whether the sound emitted by the target object is snoring, the influence of each group of target sound characteristics can be taken into account, thereby improving the detection accuracy of snoring.

[0062] In an optional embodiment, the process of extracting sound features of a sound clip includes: performing frame processing on each sound clip, wherein the frame processing is used to divide the sound clip into K arrays of equal data size, wherein K is an integer greater than 1; performing Fourier transform and Mel transform on the K arrays corresponding to the sound clip; and using the frequency amplitude obtained by Fourier transform and the target sound frequency obtained by Mel transform as the sound features of the sound clip.

[0063] Optionally, K may be 2, that is, the sound clip may be divided into two arrays of equal data size. In addition, for each sound clip, Fourier transform may be performed to obtain features such as the frequency amplitude of the sound clip; and Mel transform may be performed on each sound clip to obtain features such as the target sound frequency of the sound clip.

[0064] In an optional embodiment, Fourier transform is performed on K arrays corresponding to the sound clip, including: performing Fourier transform on the K arrays corresponding to the sound clip to obtain multiple transformed complex numbers; calculating the square of the modulus of the multiple transformed complex numbers to obtain the frequency amplitude of the sound clip.

[0065] Optionally, assuming that each sound clip is 2 arrays and the total data length of the 2 arrays is 2N, a 2N-point fast Fourier transform can be performed on each sound clip to obtain the transformed unilateral value, a total of (N+1) complex numbers. The square of the modulus of the complex number is taken and recorded as the frequency amplitude of the sound clip.

[0066] Through the above process, relevant features of the frequency amplitude of each sound segment can be extracted and used as a data basis for calculating the target sound features.

[0067] In an optional embodiment, a Mel transform is performed on the K arrays corresponding to each sound segment, including: first setting the maximum frequency value and the minimum frequency value of the Mel transform, then determining the maximum Mel coefficient according to the maximum frequency value of the Mel transform; and determining the minimum Mel coefficient according to the minimum frequency value of the Mel transform. Secondly, in the numerical interval composed of the minimum Mel coefficient and the maximum Mel coefficient, P target values ​​are determined, wherein the absolute value of the difference between each two adjacent target values ​​is the same value, wherein P is an integer greater than 1. Finally, the frequency value corresponding to each target value is calculated, and according to the principle that the frequency value corresponding to each target value corresponds to the frequency spectrum order of the Fourier transform, the frequency values ​​adjacent to the frequency value spectrum order corresponding to each target value are selected for weighted average calculation to obtain the target sound frequency of the sound segment.

[0068] Optionally, first set the frequency maximum value of the Mel transform, and the maximum frequency value and the minimum frequency value are recorded as: .

[0069] Optionally, formula (1) is the conversion between frequency value and Mel value:

[0070]

[0071] Optionally, Substitute them into formula (1) respectively and calculate the minimum Mel coefficient Since snoring recognition simulates human hearing, it is necessary to perform averaging in the Mel domain. The numerical interval formed by the two extreme values ​​is divided into P target values, among which any target value can be recorded as . The corresponding frequency value It can be calculated by formula (2).

[0072]

[0073] Optionally, Corresponding to the frequency spectrum order of the Fourier transform above, on this basis, we can take The weighted average of the nearby frequency values ​​(for example, the two frequency values ​​adjacent to each other) is calculated, where the calculation result is the result of Mel filtering, which is recorded as This linear calculation method does not require multiple Mel filters, which greatly reduces the amount of calculation while retaining the Mel spectrum information.

[0074] In an optional embodiment, the sound features recorded in the data ring are normalized to obtain the target sound features, including: determining the maximum frequency amplitude from the frequency amplitudes of the X sound clips recorded in the data ring, and then normalizing the target sound frequencies of the X sound clips recorded in the data ring according to the maximum frequency amplitude, and using the normalization result as the target sound feature.

[0075] Optionally, when normalizing the sound features recorded in the data ring, it is necessary to extract the maximum amplitude of the sound in the data ring. Due to limited memory, it is impossible to directly obtain the maximum value in the original data, so the following conversion calculation is performed, including:

[0076] When performing Fourier transform on each sound segment, record the maximum amplitude of each sound segment. Since the data ring corresponds to X sound clips, these X sound clips correspond to X , from X The maximum value selected is the maximum amplitude of the sound in the data loop. .

[0077] Later, according to , the target sound frequencies of the X sound clips recorded in the data ring are normalized, and the normalized results are used as the target sound features.

[0078] In an optional embodiment, the target sound frequencies of the X sound clips recorded in the data ring are normalized according to the maximum frequency amplitude, and the normalized result is used as the target sound feature, including: first, according to the preset sound frequency value, the target sound frequency is converted into a target decibel value, and then the minimum decibel value and the maximum decibel value in the Fourier transform are set according to the preset sound frequency value. Secondly, according to the minimum decibel value and the maximum decibel value, the target decibel value is converted into a first integer value, and the ratio of the first integer value to the maximum frequency amplitude is calculated to obtain a normalized value. Subsequently, the normalized value is converted into a second integer value, wherein the number of digits of the first integer value is greater than the number of digits of the second integer value. Finally, the second integer value is used as the target sound feature.

[0079] Optionally, the result of the Mel filtering (i.e. the target sound frequency mentioned above) is theoretically a floating point number when stored, but the floating point number occupies too much memory, so this application converts the result of Mel filtering into a first integer value (such as a value in int16 format) for storage and transmission. Specifically, the following steps are included:

[0080] First, the result of Mel filtering Convert to decibel value to prevent the logarithm from being uncalculated and add a minimum value (corresponding to the preset sound frequency value mentioned above). The target decibel value can be calculated by the following formula (3): .

[0081]

[0082] Secondly, the target decibel value Convert to the first integer value, for example, convert to a value in int16 format, recorded as . Specifically include: pre-setting the minimum decibel value Since the maximum sampling value is 1, the maximum decibel value of the 1024-point FFT (Fourier transform) is 512, so the corresponding maximum decibel value is set to The conversion formula between the target decibel value and the first integer value can be obtained as formula (4).

[0083]

[0084] Further, the ratio of the first integer value to the maximum frequency amplitude is calculated to obtain a normalized value The reference formula is formula (5).

[0085]

[0086] Finally, since the input data of the model design is int8 format, we can The minimum decibel value used when converting to int8 format (i.e. converting the normalized value to the second integer value) , due to the normalization The maximum value is 1, so the maximum decibel value used when converting to int8 format .

[0087] It should be noted that the derivation of the above normalization process is as follows:

[0088] In this application, the sound features recorded in the data ring include Mel spectrum values. The maximum value of the above is the amplitude value of the Fourier transform. The Fourier transform formula is linear, such as the following formula (6) and formula (7).

[0089]

[0090] When the original data is multiplied (divided) by a certain value, the multiplication and division can be performed directly on the value after Fourier transformation. At the same time, the Mel filter is a nonlinear filter and cannot be directly multiplied or divided. However, in this application, the Mel filter is taken The weighted average of two nearby values ​​is a linear filter. Therefore, the above formula (5) can be used to complete the normalization while retaining the sound characteristics.

[0091] It should be noted that the target sound feature is essentially a normalized feature extracted based on X sound clips, and the X sound clips are dynamically changing. Therefore, through this design, not only a large number of target sound features can be obtained, but the target sound feature includes not only the feature information of one sound clip, but the feature information of X sound clips, which shows that the target sound feature itself also has sufficient feature information. Finally, the normalization process can eliminate the influence of volume and ensure the detection effect of the device at different distances. After converting to int8 format, the calculation process of subsequent models is simplified and the calculation efficiency is improved.

[0092] In an optional embodiment, Figure 2 is a schematic diagram of an optional fast snoring detection according to an embodiment of the present application, such as Figure 2 As shown in the figure, the microphone is used to collect sound data to the MCU. Sound feature extraction is completed by obtaining sound features through Fourier transform, Mel transform and filtering. Then, the trained deep learning model is used to infer and calculate whether the current sound segment is snoring. Finally, the threshold queue method is used to determine whether it is snoring.

[0093] The collected sound data can be 16 bits, ranging from -32768 to 32767. Considering the range of the model processing data, the original data is normalized to -1 to 1.

[0094] In addition, the process of sound feature extraction includes: obtaining an audio of a fixed length; dividing the audio into frames; adding windows to the audio; and performing Fourier transform and Mel transform on the audio.

[0095] Optionally, in the process of real-time calculation, it is impossible to store a fixed-length audio to complete the above calculation. Therefore, when implementing sound feature extraction on an MCU (Microcontroller Unit), it is necessary to consider the memory and amount of calculation required for calculation. In order to reduce memory overhead, this application sets up a data ring for real-time recording of sound features. In the initial state, the data ring is set to store audio with a duration of T.

[0096] In an optional embodiment, Figure 3 is a schematic diagram of an optional sound processing method according to an embodiment of the present application, such as Figure 3 As shown, the following steps are included:

[0097] S1: Perform frame operation on the collected sound information.

[0098] like Figure 3 As shown in the figure, an array of size 2N (divided into new data N1 and old data N2) is allocated in the memory to store the original sound data sampled by the sound sampling module (DMA). After sampling the latest data, the last sampled data (N1) is automatically copied to the memory (N2) storing the old data, and the latest sampled data is used as the updated N1.

[0099] S2: Fast Fourier transform.

[0100] like Figure 3 As shown in the figure, the data of N1+N2 is a sound segment, which can be used to extract sound features. Its data length is 2N (N1+N2). Perform a 2N-point fast Fourier transform on the sound segment to obtain the transformed unilateral value, which is a total of (N+1) complex numbers. The square of the modulus of the complex number is taken and recorded as the frequency amplitude.

[0101] S3: Mel transform.

[0102] like Figure 3 As shown, first set the frequency value of the Mel transform, the maximum frequency value and the minimum frequency value are recorded as: Specifically, the Mel transform can be implemented by the above formulas (1)-(2), which will not be described in detail here.

[0103] S4: Set the fast detection moving block M.

[0104] like Figure 3 As shown, a deletion number M can be set. When the data ring is full, for each sound clip in the data ring (the duration can be recorded as ), when performing target operations on the data ring, the beginning of each sound clip can be automatically deleted. The sound characteristics of the sound information of the duration, while continuing to collect The sound characteristics of the sound information of the duration are obtained to obtain a new sound segment (the amount of audio data remains unchanged, but the required duration is ) sound features, after all the sound features of the original X sound clips in the data ring are updated, a new data ring can be obtained (essentially, the data ring stores the sound features of the updated X sound clips). This method can realize multiple audio detections in a short period of time, and only needs to wait for M corresponding time to form a new data ring, which greatly reduces missed detections due to improper slicing. Among them, Figure 4 This is a schematic diagram of a target operation according to an embodiment of the present application, which can be used for reference.

[0105] S5: Normalization (judgment step).

[0106] Please refer to the introduction process of formulas (3)-(5) above, which will not be repeated here.

[0107] It should be noted that after collecting X sound clips for the first time, firstly, step S1, step S2, step S3, step S5 are executed, and then step S4 is performed. In the subsequent sound collection process, the method of step S4 is used to repeat steps S1-S3, that is, each time it takes time M to form a new data ring, and then enter S5.

[0108] From the above content, it can be seen that the present application can realize multiple sampling of a short-duration audio, so as to obtain sound features of more sound clips. Under the condition of the same audio duration, compared with the prior art, the technical solution of the present application can obtain richer sound features, thereby improving the detection efficiency and accuracy of snoring, and thus solving the technical problem of low snoring detection efficiency caused by the prior art that longer audio information is required to determine whether the sound emitted by the target object is snoring.

[0109] According to another aspect of the embodiment of the present application, a snoring detection device is also provided, wherein: Figure 5 is a schematic diagram of an optional snoring detection device according to an embodiment of the present application, such as Figure 5 As shown, the snoring detection device includes: a first processing unit 501 , a second processing unit 502 , a third detection unit 503 and a fourth processing unit 504 .

[0110] Optionally, the first processing unit 501 is used to record the sound features of X sound clips of the target object into the data ring, and normalize the sound features recorded in the data ring to obtain a first group of target sound features, wherein X is an integer greater than 1; the second processing unit 502 is used to perform Y target operations on the data ring in a serial manner after obtaining the first group of target sound features, wherein each target operation is used to delete the head sound information in each sound clip corresponding to the data ring, and increase the sound information of the target duration at the end of each sound clip in sequence according to the sound sampling time to obtain the updated X sound clips; the target operation is also used to record the sound features of the updated X sound clips in the data ring; wherein the duration of the head sound information is equal to the target duration; the third processing unit 503 is used to normalize the sound features recorded in the data ring after each target operation is completed to obtain Y groups of target sound features, wherein Y is a positive integer; the fourth processing unit 504 is used to determine whether the sound emitted by the target object is snoring according to the Y+1 groups of target sound features.

[0111] Optionally, the fourth processing unit 504 includes: a first processing subunit and a second processing subunit. The first processing subunit is used to generate a label corresponding to each group of target sound features according to each group of target sound features through the prior knowledge learned by the target model in the model training stage, wherein the label is a first label or a second label, the first label indicates that the sound corresponding to the target sound feature is snoring, and the second label indicates that the sound corresponding to the target sound feature is not snoring; wherein the training data of the target model includes historical sound features with known corresponding labels; the second processing subunit is used to determine whether the sound emitted by the target object is snoring according to the proportion of the number of target sound features of the first label in the number of Y+1 groups of target sound features.

[0112] Optionally, the second processing subunit includes: a first determination module and a second determination module. The first determination module is used to determine that the sound emitted by the target object is snoring when the detected number ratio is greater than a preset threshold; the second determination module is used to determine that the sound emitted by the target object is not snoring when the detected number ratio is less than or equal to the preset threshold.

[0113] Optionally, the snoring detection device further includes: a frame processing unit, a data conversion unit and a fifth processing unit. The frame processing unit is used to perform frame processing on each sound segment, wherein the frame processing is used to divide the sound segment into K arrays of the same data size, wherein K is an integer greater than 1; the data conversion unit is used to perform Fourier transform and Mel transform on the K arrays corresponding to the sound segment; the fifth processing unit is used to use the frequency amplitude obtained by Fourier transform and the target sound frequency obtained by Mel transform as the sound feature of the sound segment.

[0114] Optionally, the data conversion unit includes: a Fourier transform subunit and a first calculation subunit. The Fourier transform subunit is used to perform Fourier transform on the K arrays corresponding to the sound segment to obtain a plurality of transformed complex numbers; and the first calculation subunit is used to calculate the square of the modulus of the plurality of transformed complex numbers to obtain the frequency amplitude of the sound segment.

[0115] Optionally, the data conversion unit includes: a setting subunit, a first determination subunit, a second determination subunit, a second calculation subunit, and a third calculation subunit. The setting subunit is used to set the maximum frequency value and the minimum frequency value of the Mel transform; the first determination subunit is used to determine the maximum Mel coefficient according to the maximum frequency value of the Mel transform; the minimum Mel coefficient is determined according to the minimum frequency value of the Mel transform; the second determination subunit is used to determine P target values ​​in the numerical interval composed of the minimum Mel coefficient and the maximum Mel coefficient, wherein the absolute value of the difference between each two adjacent target values ​​is the same value, wherein P is an integer greater than 1; the second calculation subunit is used to calculate the frequency value corresponding to each target value; the third calculation subunit is used to select the frequency values ​​corresponding to each target value in the frequency spectrum order corresponding to each target value according to the principle that the frequency value corresponding to each target value corresponds to the frequency spectrum order of the Fourier transform, and perform weighted average calculation to obtain the target sound frequency of the sound segment.

[0116] Optionally, the snoring detection device further includes: a first determining unit and a second determining unit. The first determining unit is used to determine the maximum frequency amplitude from the frequency amplitudes of the X sound segments recorded in the data ring; the second determining unit is used to normalize the target sound frequencies of the X sound segments recorded in the data ring according to the maximum frequency amplitude, and use the normalization result as the target sound feature.

[0117] Optionally, the second determination unit includes: a first conversion subunit, a decibel value setting subunit, a second conversion subunit, a fourth calculation subunit, a third conversion subunit, and a third determination subunit. The first conversion subunit is used to convert the target sound frequency into a target decibel value according to a preset sound frequency value; the decibel value setting subunit is used to set the minimum decibel value and the maximum decibel value in the Fourier transform according to the preset sound frequency value; the second conversion subunit is used to convert the target decibel value into a first integer value according to the minimum decibel value and the maximum decibel value; the fourth calculation subunit is used to calculate the ratio of the first integer value to the maximum frequency amplitude to obtain a normalized value; the third conversion subunit is used to convert the normalized value into a second integer value, wherein the number of digits of the first integer value is greater than the number of digits of the second integer value; the third determination subunit is used to use the second integer value as the target sound feature.

[0118] According to another aspect of the present application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, wherein when the computer program is executed, the device where the computer-readable storage medium is located executes the above-mentioned snoring detection method.

[0119] According to another aspect of the present application, an electronic device is also provided, wherein the electronic device includes one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors are enabled to run the programs, wherein the programs are configured to execute the above-mentioned snoring detection method when running.

[0120] The above-mentioned embodiments or examples disclosed in the present application are not exhaustive, but are only illustrative of some embodiments or examples, and are not intended to be specific limitations on the scope of protection disclosed in the present application. In the absence of contradiction, each step in a certain embodiment or example in the present application can be implemented as an independent example, and the steps can be combined arbitrarily. For example, the scheme after removing some steps in a certain embodiment or example can also be implemented as an independent example, and the order of the steps in a certain embodiment or example can be arbitrarily exchanged. In addition, the optional methods or optional examples in a certain embodiment or example can be combined arbitrarily; in addition, the various embodiments or examples can be combined arbitrarily, for example, some or all steps of different embodiments or examples can be combined arbitrarily, and a certain embodiment or example can be combined arbitrarily with the optional methods or optional examples of other embodiments or examples.

[0121] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0122] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0123] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0124] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0125] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0126] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc., which can store program code.

[0127] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A snoring detection method, characterized in that: include: Recording the sound features of X sound clips of the target object into a data ring, and normalizing the sound features recorded in the data ring to obtain a first set of target sound features, where X is an integer greater than 1; After obtaining the first group of target sound features, performing Y target operations on the data ring in a serial manner, wherein each target operation is used to delete the head sound information in each sound segment corresponding to the data ring, and to sequentially increase the sound information of the target duration at the end of each sound segment according to the sound sampling time, to obtain the updated X sound segments; the target operation is also used to record the sound features of the updated X sound segments in the data ring; wherein the duration of the head sound information is equal to the target duration; Normalizing the sound features recorded in the data loop after each completion of the target operation to obtain Y groups of target sound features, where Y is a positive integer; Determine whether the sound emitted by the target object is snoring according to the Y+1 group of target sound features.

2. The snoring detection method according to claim 1, characterized in that: Determining whether the sound emitted by the target object is snoring according to the Y+1 group of target sound features includes: Generate a label corresponding to each group of the target sound features according to each group of the target sound features through the prior knowledge learned by the target model in the model training phase, wherein the label is a first label or a second label, the first label indicates that the sound corresponding to the target sound feature is snoring, and the second label indicates that the sound corresponding to the target sound feature is not snoring; wherein the training data of the target model includes historical sound features with known corresponding labels; It is determined whether the sound emitted by the target object is snoring according to the proportion of the number of the target sound features of the first label in the number of the target sound features of the Y+1 group.

3. The snoring detection method according to claim 2, characterized in that: Determining whether the sound emitted by the target object is snoring according to the proportion of the number of the target sound features of the first label in the number of the target sound features of the Y+1 group includes: In the case where it is detected that the proportion of the number is greater than a preset threshold, determining that the sound emitted by the target object is the snoring sound; When it is detected that the number ratio is less than or equal to the preset threshold, it is determined that the sound emitted by the target object is not the snoring sound.

4. The snoring detection method according to claim 1, characterized in that: The process of extracting the sound features of the sound clip includes: Performing frame processing on each of the sound clips, wherein the frame processing is used to divide the sound clips into K arrays with the same data size, wherein K is an integer greater than 1; Perform Fourier transform and Mel transform on K arrays corresponding to the sound clips; The frequency amplitude obtained by the Fourier transform and the target sound frequency obtained by the Mel transform are used as the sound features of the sound segment.

5. The snoring detection method according to claim 4, characterized in that: Performing Fourier transform on the K arrays corresponding to the sound clips includes: Performing Fourier transform on the K arrays corresponding to the sound clips to obtain a plurality of transformed complex numbers; The squares of the moduli of the transformed multiple complex numbers are calculated to obtain the amplitudes of multiple frequencies of the sound segment.

6. The snoring detection method according to claim 4, characterized in that: Mel transform is performed on the K arrays corresponding to each sound clip, including: Setting the maximum frequency value and the minimum frequency value of the Mel transform; Determine a maximum Mel coefficient according to a maximum frequency value of the Mel transform; determine a minimum Mel coefficient according to a minimum frequency value of the Mel transform; In the numerical interval formed by the minimum Mel coefficient and the maximum Mel coefficient, P target values ​​are determined, wherein the absolute value of the difference between each two adjacent target values ​​is the same value, and P is an integer greater than 1; Calculate the frequency value corresponding to each target value; According to the principle that the frequency value corresponding to each target value corresponds to the frequency spectrum order of the Fourier transform, the frequency values ​​adjacent to the frequency spectrum order of the frequency value corresponding to each target value are selected for weighted average calculation to obtain the target sound frequency of the sound segment.

7. The snoring detection method according to claim 4, characterized in that: Normalizing the sound features recorded in the data ring to obtain target sound features includes: Determining a maximum frequency amplitude from the frequency amplitudes of the X sound clips recorded in the data ring; According to the maximum frequency amplitude, the target sound frequencies of the X sound clips recorded in the data ring are normalized, and the normalization result is used as the target sound feature.

8. The snoring detection method according to claim 7, characterized in that: According to the maximum frequency amplitude, the target sound frequencies of the X sound clips recorded in the data ring are normalized, and the normalization result is used as the target sound feature, including: According to a preset sound frequency value, converting the target sound frequency into a target decibel value; Setting the minimum decibel value and the maximum decibel value in the Fourier transform according to the preset sound frequency value; Converting the target decibel value into a first integer value according to the minimum decibel value and the maximum decibel value; Calculating a ratio of the first integer value to the maximum frequency amplitude to obtain a normalized value; Converting the normalized value to a second integer value, wherein the number of digits of the first integer value is greater than the number of digits of the second integer value; The second integer value is used as the target sound feature.

9. A snoring detection device, characterized in that: include: A first processing unit is used to record the sound features of X sound clips of the target object into a data ring, and normalize the sound features recorded in the data ring to obtain a first set of target sound features, where X is an integer greater than 1; a second processing unit, configured to perform Y target operations on the data ring in a serial manner after obtaining the first group of target sound features, wherein each target operation is used to delete the head sound information in each sound segment corresponding to the data ring, and to sequentially increase the sound information of the target duration at the end of each sound segment according to the sound sampling time, to obtain the updated X sound segments; the target operation is also used to record the sound features of the updated X sound segments in the data ring; wherein the duration of the head sound information is equal to the target duration; A third processing unit is used to perform normalization processing on the sound features recorded in the data ring after each completion of the target operation to obtain Y groups of target sound features, where Y is a positive integer; The fourth processing unit is used to determine whether the sound emitted by the target object is snoring according to the Y+1 groups of target sound features.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located executes the snoring detection method according to any one of claims 1 to 8.

11. An electronic device, characterized in that: The device comprises one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the snoring detection method according to any one of claims 1 to 8.

Images