Snore detection method and apparatus, electronic device, and computer-readable storage medium

Abstract

The present application relates to the field of data processing. Disclosed are a snore detection method and apparatus, an electronic device, and a computer-readable storage medium. The method comprises: recording sound features of X sound clips of a target subject into a data ring; performing Y target operations on the data ring, wherein each target operation is used for deleting head sound information in each sound clip corresponding to the data ring and sequentially adding sound information for a target duration at the tail of each sound clip, so as to obtain X updated sound clips, and recording sound features of the X updated sound clips into the data ring; normalizing the sound features recorded in the data ring to obtain Y groups of target sound features; and on the basis of the Y+1 groups of target sound features, determining whether the sound produced by the target subject is a snore. The present application solves the technical problem in the prior art of a low snore detection efficiency because long video information is required for determining whether a sound produced by a target subject is a snore.

Description

Snoring detection methods, devices, electronic equipment and computer-readable storage media

[0001] Cross-reference to related applications

[0002] This application claims priority to Chinese Patent Application No. 2024114918741, filed on October 24, 2024, entitled “Snoring Detection Method, Apparatus, Electronic Device and Computer-Readable Storage Medium”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of data processing, and more specifically, to a snoring detection method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0004] Obstructive sleep apnea-hypopnea syndrome (OSAHS) is a common clinical condition characterized by snoring and pauses in breathing at night. Patients with OSAHS often experience daytime sleepiness and are highly susceptible to various cardiovascular and cerebrovascular diseases, seriously jeopardizing their physical and mental health.

[0005] In the diagnosis of obstructive sleep apnea-hypopnea syndrome, it is usually necessary to identify the triggering site (e.g., neck, base of tongue, chest cavity, lungs, etc.) that causes snoring. Currently, doctors often identify the triggering site by analyzing the audio recordings of snoring produced during sleep. Specifically, feature extraction is performed on the snoring audio recordings to obtain a specific type of feature (e.g., MFCC (Mel-Frequency Cepstral Coefficients), GFCC (Gammatone feature cochleagram), or frequency domain features), and the triggering site is identified based on this type of feature.

[0006] However, the above-mentioned recognition methods usually require a long amount of audio information to extract enough features in order to ensure the accuracy of snoring detection. In real-world scenarios, however, it is not easy to obtain long-term audio information, which leads to low efficiency in snoring detection.

[0007] There is currently no effective solution to the above problems. Summary of the Invention

[0008] This application provides a snoring detection method, apparatus, electronic device, and computer-readable storage medium to at least solve the technical problem of low snoring detection efficiency caused by the need for relatively long audio information in the prior art to determine whether the sound emitted by the target object is snoring.

[0009] According to one aspect of this application, a snoring detection method is provided, comprising: recording the sound features of X sound segments of a target object into a data ring, and normalizing the sound features recorded in the data ring to obtain a first set of target sound features, wherein X is an integer greater than 1; after obtaining the first set of target sound features, performing Y target operations on the data ring in a serial manner, wherein each target operation is used to delete the head sound information in each sound segment corresponding to the data ring, and sequentially incrementing the sound information of the target duration at the end of each sound segment according to the sound sampling time to obtain updated X sound segments; the target operation is also used to record the sound features of the updated X sound segments in the data ring; wherein the duration of the head sound information is equal to the target duration; normalizing the sound features recorded in the data ring after each target operation to obtain Y sets of target sound features, wherein Y is a positive integer; and determining whether the sound emitted by the target object is snoring based on the Y+1 sets of target sound features.

[0010] Optionally, determining whether the sound emitted by the target object is snoring based on the Y+1 group of target sound features includes: generating a label corresponding to each group of target sound features based on the prior knowledge learned by the target model during the model training phase, wherein the label is a first label or a second label, the first label indicating that the sound corresponding to the target sound feature is snoring, and the second label indicating that the sound corresponding to the target sound feature is not snoring; wherein the training data of the target model includes historical sound features with known corresponding labels; and determining whether the sound emitted by the target object is snoring based on the proportion of the number of target sound features with the first label in the Y+1 group of target sound features.

[0011] Optionally, the determination of whether the sound emitted by the target object is snoring is based on the proportion of the number of target sound features of the first label in the target sound features of group Y+1, including: determining that the sound emitted by the target object is snoring when the proportion of the detected number is greater than a preset threshold; and determining that the sound emitted by the target object is not snoring when the proportion of the detected number is less than or equal to the preset threshold.

[0012] Optionally, the process of extracting the sound features of a sound segment includes: performing frame segmentation on each sound segment, wherein the frame segmentation is used to divide the sound segment into K arrays of equal data size, where K is an integer greater than 1; performing Fourier transform and Mel transform on the K arrays corresponding to the sound segment; and using the frequency amplitude obtained from the Fourier transform and the target sound frequency obtained from the Mel transform as the sound features of the sound segment.

[0013] Optionally, performing a Fourier transform on the K arrays corresponding to the sound segment includes: performing a Fourier transform on the K arrays corresponding to the sound segment to obtain multiple complex numbers after transformation; calculating the square of the modulus of the multiple complex numbers after transformation to obtain the frequency amplitude of the sound segment.

[0014] Optionally, a Mel transform is performed on the K arrays corresponding to each sound segment, including: setting the maximum and minimum frequency values ​​of the Mel transform; determining the maximum Mel coefficient based on the maximum frequency value of the Mel transform; determining the minimum Mel coefficient based on the minimum frequency value of the Mel transform; determining P target values ​​in the numerical interval formed by the minimum and maximum Mel coefficients, wherein the absolute value of the difference between any two adjacent target values ​​is the same value, where P is an integer greater than 1; calculating the frequency value corresponding to each target value; and, according to the principle that the frequency value corresponding to each target value corresponds to the spectral order of the Fourier transform, selecting the frequency values ​​adjacent to the spectral order of the frequency value corresponding to each target value for weighted average calculation to obtain the target sound frequency of the sound segment.

[0015] Optionally, the sound features recorded in the data loop are normalized to obtain the target sound features, including: determining the maximum frequency amplitude from the frequency amplitudes of X sound segments recorded in the data loop; normalizing the target sound frequencies of the X sound segments recorded in the data loop based on the maximum frequency amplitude, and using the normalization result as the target sound features.

[0016] Optionally, based on the maximum frequency amplitude, the target sound frequencies of the X sound segments recorded in the data loop are normalized, and the normalization result is used as the target sound feature. This includes: converting the target sound frequency into a target decibel value based on a preset sound frequency value; setting the minimum and maximum decibel values ​​in the Fourier transform based on the preset sound frequency values; converting the target decibel value into a first integer value based on the minimum and maximum decibel values; calculating the ratio of the first integer value to the maximum frequency amplitude to obtain a normalized value; converting the normalized value into a second integer value, wherein the number of bits in the first integer value is greater than the number of bits in the second integer value; and using the second integer value as the target sound feature.

[0017] According to another aspect of this application, a snoring detection device is also provided, comprising: a first processing unit, configured to record the sound features of X sound segments of a target object into a data ring, and normalize the sound features recorded in the data ring to obtain a first set of target sound features, wherein X is an integer greater than 1; a second processing unit, configured to perform Y target operations on the data ring in a serial manner after obtaining the first set of target sound features, wherein each target operation is configured to delete the head sound information in each sound segment corresponding to the data ring, and sequentially increment the sound information of the target duration at the end of each sound segment according to the sound sampling time to obtain updated X sound segments; the target operation is also configured to record the sound features of the updated X sound segments in the data ring; wherein the duration of the head sound information is equal to the target duration; a third processing unit, configured to normalize the sound features recorded in the data ring after each target operation to obtain Y sets of target sound features, wherein Y is a positive integer; and a fourth processing unit, configured to determine whether the sound emitted by the target object is snoring based on the Y+1 sets of target sound features.

[0018] According to another aspect of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, wherein when the computer program is executed, the device on which the computer-readable storage medium is located performs the above-described snoring detection method.

[0019] According to another aspect of this application, an electronic device is also provided, wherein the electronic device includes one or more processors and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the snoring detection method described above.

[0020] In this application, the sound features of X sound segments of the target object are first recorded in a data ring, and the sound features recorded in the data ring are normalized to obtain a first set of target sound features, where X is an integer greater than 1. Then, after obtaining the first set of target sound features, Y target operations are performed on the data ring in a serial manner. Each target operation deletes the head sound information from each sound segment corresponding to the data ring and sequentially increments the sound information of the target duration at the end of each sound segment according to the sound sampling time, resulting in updated X sound segments. The target operation also records the sound features of the updated X sound segments in the data ring; where the duration of the head sound information is equal to the target duration. Finally, the sound features recorded in the data ring after each target operation are normalized to obtain Y sets of target sound features, where Y is a positive integer. Based on the Y+1 sets of target sound features, it is determined whether the sound emitted by the target object is snoring.

[0021] As described above, this application firstly uses a data ring to store the sound features of X sound segments of the target object, thereby eliminating the need to store all sound information in real time and reducing memory overhead. Secondly, after obtaining the first set of target sound features, this application performs Y target operations on the data ring sequentially. Each target operation deletes the head sound information from each sound segment corresponding to the data ring and sequentially increments the target duration sound information at the end of each sound segment according to the sound sampling time, resulting in updated X sound segments. The target operation also records the sound features of the updated X sound segments in the data ring; wherein the duration of the head sound information is equal to the target duration.

[0022] For example, a deletion limit M can be set. When the data loop is completely filled, for each sound segment (the duration of which can be denoted as M) in the data loop... When performing target operations on the data loop, the beginning of each sound segment can be automatically deleted. The duration of the audio information and its acoustic characteristics are collected simultaneously. The duration of the sound information and its sound characteristics are used to obtain a new sound segment (at this time, the amount of audio data remains the same, but the required duration is...). Once the sound features of the original X sound segments in the data ring are all updated, a new data ring can be obtained (essentially, the data ring stores the updated sound features of the X sound segments).

[0023] This method allows for multiple sampling of a short audio clip, resulting in more sound features from various audio segments. Compared to existing technologies, this application's solution provides richer sound features for the same audio duration, thereby improving the efficiency and accuracy of snoring detection. This solves the technical problem of low snoring detection efficiency caused by the need for longer audio information in existing technologies to determine whether the sound emitted by the target object is snoring. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0025] Figure 1 is a flowchart of an optional snoring detection method according to an embodiment of this application;

[0026] Figure 2 is a schematic diagram of an optional rapid snoring detection according to an embodiment of this application;

[0027] Figure 3 is a schematic diagram of an optional sound processing method according to an embodiment of this application;

[0028] Figure 4 is a schematic diagram of a target operation according to an embodiment of this application;

[0029] Figure 5 is a schematic diagram of an optional snoring detection device according to an embodiment of this application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] It should also be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) collected in this application are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with the relevant laws, regulations, and standards of the relevant regions, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding operation entry points are provided for users to choose to authorize or refuse. For example, this system has interfaces with relevant users or organizations. Before obtaining relevant information, a request to obtain the information needs to be sent to the aforementioned user or organization through the interface, and the relevant information is obtained only after receiving consent from the aforementioned user or organization.

[0033] According to an embodiment of this application, an embodiment of a snoring detection method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0034] Figure 1 is a flowchart of an optional snoring detection method according to an embodiment of this application. As shown in Figure 1, the method includes the following steps:

[0035] Step S101: Record the sound features of X sound segments of the target object into the data ring, and normalize the sound features recorded in the data ring to obtain the first set of target sound features.

[0036] Where X is an integer greater than 1.

[0037] Optionally, in this application, a data ring is designed to store the sound features of sound segments, wherein the data ring stores the sound features of up to X sound segments.

[0038] Optionally, each sound segment can be divided into a two-dimensional array, where the length of the two-dimensional array can be denoted as 2N. The two-dimensional array can be further divided into two one-dimensional arrays, denoted as N1 and N2, where the length of each one-dimensional array is N. The memory can be divided into two regions to store the two one-dimensional arrays respectively. For example, the first region stores the one-dimensional array N1 and the second region stores the one-dimensional array N2. Assuming that the acquisition time of one-dimensional array N1 is later than the acquisition time of one-dimensional data N2, after acquiring the latest one-dimensional array, the one-dimensional array N2 originally stored in the second region is deleted, the one-dimensional array N1 originally stored in the first region is moved to the second region as the new one-dimensional array N2, and then the latest one-dimensional array is stored in the first region as the new one-dimensional array N1.

[0039] Optionally, after the sound features of X sound segments are first filled into the data ring, the sound features currently recorded in the data ring can be normalized to obtain the first set of target sound features.

[0040] Step S102: After obtaining the first set of target sound features, perform Y target operations on the data loop in a serial manner.

[0041] In step S102, each target operation is used to delete the head sound information in each sound segment corresponding to the data ring, and sequentially increment the sound information of the target duration at the end of each sound segment according to the sound sampling time to obtain the updated X sound segments; the target operation is also used to record the sound features of the updated X sound segments in the data ring; wherein, the duration of the head sound information is equal to the target duration.

[0042] Optionally, for the data ring, a preset deletion amount M can be set. After the data ring is filled with the sound features of X sound segments, for each sound segment in the data ring (assuming the total duration of the X sound segments is T, and the duration of each sound segment is T / X), when performing the target operation, the sound information of the beginning of each sound segment with a duration of M / X (corresponding to the target duration mentioned above) can be automatically deleted. Then, according to the sound sampling time, the sound information of the duration of M / X is sequentially incremented at the end of each sound segment. Since there are a total of X sound segments in the data ring, the updated X sound segments can be obtained in the end, where the total duration of the updated X sound segments is also T. Finally, the target operation can record the sound features of the updated X sound segments in the data ring.

[0043] To more clearly describe the target operation described above, the following example will be used for illustration.

[0044] Suppose that the data ring initially stores the sound features of three sound segments: sound segment 1 (collected from 0 to 1 second), sound segment 2 (collected from 1 to 2 seconds), and sound segment 3 (collected from 2 to 3 seconds).

[0045] Perform the first target operation: Assuming the target duration is 0.5 seconds, the updated sound segment 1 records the sound acquired from 0.5 seconds to 1.5 seconds, the updated sound segment 2 records the sound acquired from 1.5 seconds to 2.5 seconds, and the updated sound segment 3 records the sound acquired from 2.5 seconds to 3.5 seconds. The target operation records the sound characteristics of the three updated sound segments in the data loop.

[0046] A second target operation is performed: the updated sound segment 1 records the sound acquired from 1 to 2 seconds; the updated sound segment 2 records the sound acquired from 2 to 3 seconds; and the updated sound segment 3 records the sound acquired from 3 to 4 seconds. The target operation records the sound characteristics of the three updated sound segments in the data loop.

[0047] As described above, after each target operation, the newly generated sound features are always recorded in the data loop. This method allows for multiple sampling of a short audio clip, resulting in more sound features from various audio segments. Compared to existing technologies, this application's solution yields richer sound features for the same audio duration, thereby improving the efficiency and accuracy of snoring detection. This solves the technical problem of low snoring detection efficiency caused by the need for longer audio information in existing technologies to determine whether the sound emitted by the target object is snoring.

[0048] Step S103: Normalize the sound features recorded in the data loop after each target operation to obtain Y sets of target sound features, where Y is a positive integer.

[0049] Optionally, if Y target operations are performed, the sound features recorded in the data loop must be normalized after each target operation to obtain a set of target sound features. Therefore, after Y target operations, a total of Y sets of target sound features can be obtained.

[0050] Step S104: Determine whether the sound emitted by the target object is snoring based on the target sound characteristics of group Y+1.

[0051] Optionally, this application supports using a neural network model to determine whether the sound emitted by a target object is snoring based on Y+1 sets of target sound features. This application also supports determining whether the sound emitted by a target object is snoring based on Y+1 sets of target sound features through data analysis or manual analysis. For example, comparing the similarity between the target sound features and known snoring features to determine whether the sound emitted by the target object is snoring.

[0052] Based on the above steps S101 to S104, in this application, the sound features of X sound segments of the target object are first recorded in a data ring, and the sound features recorded in the data ring are normalized to obtain a first set of target sound features, where X is an integer greater than 1. Then, after obtaining the first set of target sound features, Y target operations are performed on the data ring in a serial manner. Each target operation is used to delete the head sound information in each sound segment corresponding to the data ring, and according to the sound sampling time, the sound information of the target duration is sequentially incremented at the end of each sound segment to obtain updated X sound segments. The target operation is also used to record the sound features of the updated X sound segments in the data ring; where the duration of the head sound information is equal to the target duration. Finally, the sound features recorded in the data ring after each target operation are normalized to obtain Y sets of target sound features, where Y is a positive integer; based on the Y+1 sets of target sound features, it is determined whether the sound emitted by the target object is snoring.

[0053] As described above, this application firstly uses a data ring to store the sound features of X sound segments of the target object, thereby eliminating the need to store all sound information in real time and reducing memory overhead. Secondly, after obtaining the first set of target sound features, this application performs Y target operations on the data ring sequentially. Each target operation deletes the head sound information from each sound segment corresponding to the data ring and sequentially increments the target duration sound information at the end of each sound segment according to the sound sampling time, resulting in updated X sound segments. The target operation also records the sound features of the updated X sound segments in the data ring; wherein the duration of the head sound information is equal to the target duration.

[0054] For example, a deletion limit M can be set. When the data loop is completely filled, for each sound segment (the duration of which can be denoted as M) in the data loop... When performing target operations on the data loop, it can automatically delete the beginning of each sound segment. The duration of the audio information and its acoustic characteristics are collected simultaneously. The duration of the sound information and its sound characteristics are used to obtain a new sound segment (at this time, the amount of audio data remains the same, but the required duration is...). Once the sound features of the original X sound segments in the data ring are all updated, a new data ring can be obtained (essentially, the data ring stores the updated sound features of the X sound segments).

[0055] This method allows for multiple sampling of a short audio clip, resulting in more sound features from various audio segments. Compared to existing technologies, this application's solution provides richer sound features for the same audio duration, thereby improving the efficiency and accuracy of snoring detection. This solves the technical problem of low snoring detection efficiency caused by the need for longer audio information in existing technologies to determine whether the sound emitted by the target object is snoring.

[0056] In one optional embodiment, prior knowledge learned by the target model during model training can be used to generate labels corresponding to each set of target sound features. Each label is either a first label or a second label. The first label indicates that the sound corresponding to the target sound feature is snoring, and the second label indicates that the sound corresponding to the target sound feature is not snoring. The training data of the target model includes historical sound features with known corresponding labels. Then, based on the proportion of the number of target sound features with the first label in the Y+1 set of target sound features, it is determined whether the sound emitted by the target object is snoring.

[0057] Optionally, snoring sound data from different groups of people for a certain period of time (e.g., 10 hours) can be collected from multiple channels, along with other sounds that may occur at night, such as flushing sounds and talking sounds, for a certain period of time. These sounds may interfere with the model's judgment of snoring sounds. After processing this data, the model can be trained. This application supports using the standard gradient descent method as the training method and cross-entropy loss as the loss function. During end-to-end self-training, the convolutional neural network can learn how to automatically extract the most important features for snoring sound recognition from the MFCC, while the fully connected layer can learn how to effectively classify these features. Finally, the trained deep learning model is used as the target model for recognizing and detecting snoring sounds.

[0058] The network structure of the target model can be found in Table 1:

[0059] Table 1

[0060] Optionally, when determining whether the sound emitted by the target object is snoring based on the proportion of the number of target sound features of the first label in the target sound features of group Y+1, if the proportion of the number of the first label is greater than a preset threshold, the sound emitted by the target object is determined to be snoring; if the proportion of the number of the first label is less than or equal to the preset threshold, the sound emitted by the target object is determined not to be snoring.

[0061] Optionally, this application uses a threshold queue method to determine whether the sound emitted by the target object is snoring. For example, a queue is set up to store the real-time judgment results. Assuming M = 1 second, T = 2 seconds, and one judgment cycle is 1 minute, then 59 detection results will be obtained in one minute. Each detection result represents the generation of a label corresponding to a set of target sound features based on that set of target sound features. If the number of times it is judged as snoring is q (i.e., the number of target sound features in the first label is q), then the proportion is q / 59, and its value is denoted as Q. A threshold Q is set. t , if Q>Q t If Q ≤ Q, then the sound emitted by the target object is determined to be snoring; t If so, then it is determined that the sound emitted by the target object is not snoring.

[0062] The threshold queue method is used to determine whether the sound emitted by the target object is snoring. It takes into account the influence of the characteristics of each group of target sounds, thereby improving the accuracy of snoring detection.

[0063] In one optional embodiment, the process of extracting the sound features of a sound segment includes: performing frame segmentation on each sound segment, wherein the frame segmentation is used to divide the sound segment into K arrays of equal data size, where K is an integer greater than 1; performing Fourier transform and Mel transform on the K arrays corresponding to the sound segment; and using the frequency amplitude obtained from the Fourier transform and the target sound frequency obtained from the Mel transform as the sound features of the sound segment.

[0064] Optionally, K can be 2, meaning the sound segments can be divided into two arrays of equal size. Furthermore, for each sound segment, a Fourier transform can be performed to obtain features such as frequency amplitude; a Mel transform can also be performed on each sound segment to obtain features such as the target sound frequency.

[0065] In one optional embodiment, performing a Fourier transform on the K arrays corresponding to the sound segment includes: performing a Fourier transform on the K arrays corresponding to the sound segment to obtain multiple complex numbers after transformation; and calculating the square of the modulus of the multiple complex numbers after transformation to obtain the frequency amplitude of the sound segment.

[0066] Optionally, assuming each sound segment is a 2-array, and the total data length of the 2 arrays is 2N, a 2N-point Fast Fourier Transform can be performed on each sound segment to obtain the transformed one-sided values, which are a total of (N+1) complex numbers. The square of the modulus of the complex number is taken as the frequency amplitude of the sound segment.

[0067] Through the above process, the relevant features of the frequency amplitude of each sound segment can be extracted, which can be used as the data basis for calculating the target sound features.

[0068] In one optional embodiment, a Mel transform is performed on the K arrays corresponding to each sound segment, including: first, setting the maximum and minimum frequency values ​​of the Mel transform; then, determining the maximum Mel coefficient based on the maximum frequency value and the minimum Mel coefficient based on the minimum frequency value. Next, within the numerical range formed by the minimum and maximum Mel coefficients, P target values ​​are determined, where the absolute value of the difference between any two adjacent target values ​​is the same, and P is an integer greater than 1. Finally, the frequency value corresponding to each target value is calculated. Following the principle that the frequency value corresponding to each target value corresponds to the spectral order of the Fourier transform, a weighted average is calculated using frequency values ​​whose spectral order is adjacent to the frequency value corresponding to each target value to obtain the target sound frequency of the sound segment.

[0069] Optionally, first set the maximum and minimum frequencies of the Mel transform, denoted as fmax and fmin, respectively. min f max .

[0070] Optionally, formula (1) is the conversion between frequency values ​​and Mel values:

[0071] Optionally, f min and f max Substituting into formula (1), the minimum Mel coefficient m is calculated accordingly. min Maximum Mel coefficient m max Since snoring recognition simulates human hearing, it needs to be evenly divided in the Mel domain, based on m... min m max The numerical interval formed by the two extreme values ​​is divided equally among P target values, where any one target value can be denoted as m. p m p The corresponding frequency value f p It can be calculated using formula (2).

[0072] Optionally, f p Corresponding to the spectral order of the Fourier transform mentioned above, based on this, we can take f p The weighted average of nearby frequency values ​​(e.g., two adjacent frequency values) is calculated, and the result is the Mel filter result, denoted as m′. p This linear calculation method eliminates the need for multiple Mel filters, significantly reducing computational complexity while preserving information from the Mel spectrum.

[0073] In one optional embodiment, the sound features recorded in the data ring are normalized to obtain target sound features, including: determining the maximum frequency amplitude from the frequency amplitudes of X sound segments recorded in the data ring, then normalizing the target sound frequencies of the X sound segments recorded in the data ring according to the maximum frequency amplitude, and using the normalization result as the target sound features.

[0074] Optionally, when normalizing the sound features recorded in the data loop, it is necessary to extract the maximum amplitude of the sound within the data loop. Due to limited memory, the maximum value cannot be directly obtained from the original data. Therefore, the following transformation calculations are performed, including:

[0075] When performing a Fourier transform on each sound segment, record the maximum amplitude of each sound segment. Since the data ring corresponds to X sound segments, these X sound segments correspond to X... From X The maximum value selected is the maximum amplitude f of the sound within the data loop. max .

[0076] Subsequently, it can be based on f max The target sound frequencies of the X sound segments recorded in the data loop are normalized, and the normalization result is used as the target sound feature.

[0077] In one optional embodiment, the target sound frequencies of X sound segments recorded in the data loop are normalized based on the maximum frequency amplitude, and the normalization result is used as the target sound feature. This includes: first, converting the target sound frequency into a target decibel value based on a preset sound frequency value; then, setting the minimum and maximum decibel values ​​in the Fourier transform based on the preset sound frequency value. Next, converting the target decibel value into a first integer value based on the minimum and maximum decibel values, and calculating the ratio of the first integer value to the maximum frequency amplitude to obtain a normalized value. Subsequently, converting the normalized value into a second integer value, wherein the number of bits in the first integer value is greater than the number of bits in the second integer value. Finally, using the second integer value as the target sound feature.

[0078] Alternatively, the result of Mel filtering, m′ p The target sound frequency (as mentioned above) is theoretically stored as a floating-point number, but floating-point numbers occupy too much memory. Therefore, this application converts the Mel filter result into a first integer value (e.g., an int16 format value) for storage and transmission. Specifically, this includes the following steps:

[0079] First, the result m′ of the Mel filter p To convert to decibel values ​​and prevent logarithmic calculations from being uncomputable, a minimum value m is added. e(Corresponding to the preset sound frequency value mentioned above). The target decibel value m can be calculated using the following formula (3). p (dB). m p (dB)=20*lg(m′ p +m e (3)

[0080] Secondly, the target decibel value m p (dB) is converted to a first integer value, for example, to an int16 format number, denoted as Specifically, this includes: presetting the minimum decibel value D. min =20lg(m e Since the maximum sample value is 1, the maximum decibel value of a 1024-point FFT (Fourier Transform) is 512. Therefore, the corresponding maximum decibel value is set to D. max =20log 10 (512 2 +m e The conversion formula between the target decibel value and the first integer value can be obtained as formula (4).

[0081] Furthermore, the ratio of the first integer value to the maximum frequency amplitude is calculated to obtain the normalized value m″. p The reference formula is formula (5).

[0082] Finally, since the input data for the model design is int8 format, m″ can be... p Convert the value to int8 format (i.e., convert the normalized value to a second integer value), and the minimum decibel value D used when converting to int8 format. min =20*lg(m e ), due to normalization m″ p The maximum value is 1, so the maximum decibel value D used when converting to int8 format is 1. max =0.

[0083] It should be noted that the derivation of the above normalization process is as follows:

[0084] In this application, the acoustic features recorded in the data loop include Mel-spectral values. The maximum value f mentioned above... max It is the amplitude value of the Fourier transform. The Fourier transform formula is linear, as shown in formulas (6) and (7) below.

[0085] When the original data is multiplied (divided) by a certain value, the multiplication and division can be performed directly on the Fourier transform result. However, the Mel filter is a nonlinear filter and cannot be directly multiplied or divided. But in this application, the Mel filter takes f... p The two nearest values ​​are weighted and averaged, which is a linear filter. Therefore, normalization can be achieved using the above formula (5) while preserving the sound characteristics.

[0086] It should be noted that the target sound features are essentially normalized features extracted from X sound segments, and these X sound segments are dynamically changing. Therefore, this design not only allows for the acquisition of a large number of target sound features, but also includes feature information from X sound segments, indicating that the target sound features themselves possess sufficiently rich feature information. Finally, normalization processing can eliminate the influence of volume, ensuring the detection performance of the device at different distances. The normalized m″... p Converting to int8 format simplifies the subsequent calculation process of the model and improves computational efficiency.

[0087] In an optional embodiment, Figure 2 is a schematic diagram of an optional rapid snoring detection method according to an embodiment of this application. As shown in Figure 2, sound data is first acquired by a microphone and transmitted to the MCU. Sound feature extraction is completed by obtaining sound features through Fourier transform, Mel transform, and filtering. Then, a trained deep learning model is used to infer and calculate whether the current sound segment is snoring. Finally, a threshold queue method is used to determine whether it is snoring.

[0088] The collected audio data can be 16 bits, ranging from -32768 to 32767. Considering the range of data the model can process, the original data is normalized to -1 to 1.

[0089] In addition, the process of sound feature extraction includes: acquiring an audio segment of fixed duration; segmenting the audio into frames; windowing the audio; and performing Fourier transform and Mel transform on the audio.

[0090] Optionally, during real-time computation, it is not possible to store a fixed-length audio segment to complete the aforementioned calculations. Therefore, when implementing sound feature extraction on an MCU (Microcontroller Unit), the required memory and computational load need to be considered. To reduce memory overhead, this application sets up a data loop for real-time recording of sound features. Initially, the data loop stores audio segments of duration T.

[0091] In an optional embodiment, FIG3 is a schematic diagram of an optional sound processing method according to an embodiment of the present application. As shown in FIG3, it includes the following steps:

[0092] S1: Perform frame segmentation on the acquired sound information.

[0093] As shown in Figure 3, a 2N array (divided into new data N1 and old data N2) is allocated in memory to store the raw sound data obtained by the sound sampling module (DMA). After the latest data is sampled, the previously sampled data (N1) is automatically copied to the memory storing the old data (N2), and the latest sampled data is used as the updated N1.

[0094] S2: Fast Fourier Transform.

[0095] As shown in Figure 3, the data N1+N2 is a sound segment that can be used to extract sound features, and its data length is 2N(N1+N2). A 2N-point Fast Fourier Transform is performed on the sound segment to obtain the transformed one-sided values, which consist of (N+1) complex numbers. The square of the modulus of the complex number is taken as the frequency amplitude.

[0096] S3: Mel transform.

[0097] As shown in Figure 3, first, the maximum and minimum frequencies of the Mel transform are set, denoted as fmax and fmin, respectively. min and f max The Mel transform can be implemented using the formulas (1)-(2) above, which will not be elaborated further here.

[0098] S4: Set up fast detection of moving block M.

[0099] As shown in Figure 3, a deletion quantity M can be set. When the data loop is completely filled, for each sound segment (the duration can be denoted as M) in the data loop... When performing target operations on the data loop, it can automatically delete the beginning of each sound segment. The duration of the audio information and its acoustic characteristics are collected simultaneously. The duration of the sound information and its sound characteristics are used to obtain a new sound segment (at this time, the amount of audio data remains the same, but the required duration is...). The sound features of the X sound segments in the original data ring are updated, and a new data ring is obtained (essentially, the data ring stores the updated sound features of the X sound segments). This method allows for multiple audio detections in a short period of time, requiring only a wait of M time to form a new data ring, greatly reducing missed detections due to improper slicing. Figure 4 is a schematic diagram of a target operation according to an embodiment of this application, which can be referred to.

[0100] S5: Normalization (judgment step).

[0101] Referring to the introduction of formulas (3)-(5) above, the process will not be repeated here.

[0102] It should be noted that after X sound segments are acquired for the first time, steps S1, S2, S3, and S5 are executed first, followed by step S4. In subsequent sound acquisition processes, the method of step S4 is used to repeat steps S1-S3, that is, each time a new data loop is formed after a time of M, and then step S5 is entered.

[0103] As can be seen from the above, this application can sample a short audio file multiple times to obtain more sound features of the sound segments. Under the same audio duration, compared with the prior art, the technical solution of this application can obtain richer sound features, thereby improving the detection efficiency and accuracy of snoring. This solves the technical problem of low snoring detection efficiency caused by the need for longer audio information in the prior art to determine whether the sound emitted by the target object is snoring.

[0104] According to another aspect of the embodiments of this application, a snoring detection device is also provided, wherein FIG5 is a schematic diagram of an optional snoring detection device according to an embodiment of this application. As shown in FIG5, the snoring detection device includes: a first processing unit 501, a second processing unit 502, a third detection unit 503 and a fourth processing unit 504.

[0105] Optionally, the first processing unit 501 is used to record the sound features of X sound segments of the target object into a data ring, and normalize the sound features recorded in the data ring to obtain a first set of target sound features, where X is an integer greater than 1; the second processing unit 502 is used to perform Y target operations on the data ring in a serial manner after obtaining the first set of target sound features, where each target operation is used to delete the head sound information in each sound segment corresponding to the data ring, and sequentially increment the sound information of the target duration at the end of each sound segment according to the sound sampling time to obtain updated X sound segments; the target operation is also used to record the sound features of the updated X sound segments in the data ring; where the duration of the head sound information is equal to the target duration; the third processing unit 503 is used to normalize the sound features recorded in the data ring after each target operation to obtain Y sets of target sound features, where Y is a positive integer; the fourth processing unit 504 is used to determine whether the sound emitted by the target object is snoring based on the Y+1 sets of target sound features.

[0106] Optionally, the fourth processing unit 504 includes: a first processing subunit and a second processing subunit. The first processing subunit is used to generate a label corresponding to each set of target sound features based on the prior knowledge learned by the target model during the model training phase. The label is either a first label or a second label, where the first label indicates that the sound corresponding to the target sound feature is snoring, and the second label indicates that the sound corresponding to the target sound feature is not snoring. The training data of the target model includes historical sound features with known corresponding labels. The second processing subunit is used to determine whether the sound emitted by the target object is snoring based on the proportion of the number of target sound features with the first label in the Y+1 set of target sound features.

[0107] Optionally, the second processing subunit includes: a first determining module and a second determining module. The first determining module is used to determine that the sound emitted by the target object is snoring when the detected quantity percentage is greater than a preset threshold; the second determining module is used to determine that the sound emitted by the target object is not snoring when the detected quantity percentage is less than or equal to the preset threshold.

[0108] Optionally, the snoring detection device further includes: a frame processing unit, a data transformation unit, and a fifth processing unit. The frame processing unit is used to perform frame processing on each sound segment, dividing the sound segment into K arrays of equal data size, where K is an integer greater than 1. The data transformation unit is used to perform Fourier transform and Mel transform on the K arrays corresponding to the sound segment. The fifth processing unit is used to use the frequency amplitude obtained from the Fourier transform and the target sound frequency obtained from the Mel transform as the sound features of the sound segment.

[0109] Optionally, the data transformation unit includes a Fourier transform subunit and a first calculation subunit. The Fourier transform subunit is used to perform Fourier transform on the K arrays corresponding to the sound segment to obtain multiple transformed complex numbers; the first calculation subunit is used to calculate the square of the modulus of the multiple transformed complex numbers to obtain the frequency amplitude of the sound segment.

[0110] Optionally, the data transformation unit includes: a setting subunit, a first determining subunit, a second determining subunit, a second calculation subunit, and a third calculation subunit. The setting subunit is used to set the maximum and minimum frequency values ​​of the Mel Transform; the first determining subunit is used to determine the maximum Mel coefficient based on the maximum frequency value of the Mel Transform; and the minimum Mel coefficient based on the minimum frequency value of the Mel Transform; the second determining subunit is used to determine P target values ​​within the numerical interval formed by the minimum and maximum Mel coefficients, wherein the absolute value of the difference between any two adjacent target values ​​is the same, and P is an integer greater than 1; the second calculation subunit is used to calculate the frequency value corresponding to each target value; and the third calculation subunit is used to select frequency values ​​whose spectral order is adjacent to the frequency value corresponding to each target value according to the principle that the frequency value corresponding to each target value corresponds to the spectral order of the Fourier Transform, and perform a weighted average calculation to obtain the target sound frequency of the sound segment.

[0111] Optionally, the snoring detection device further includes: a first determining unit and a second determining unit. The first determining unit is used to determine the maximum frequency amplitude from the frequency amplitudes of X sound segments recorded in the data loop; the second determining unit is used to normalize the target sound frequencies of the X sound segments recorded in the data loop based on the maximum frequency amplitude, and use the normalization result as a target sound feature.

[0112] Optionally, the second determining unit includes: a first conversion subunit, a decibel value setting subunit, a second conversion subunit, a fourth calculation subunit, a third conversion subunit, and a third determining subunit. The first conversion subunit is used to convert the target sound frequency into a target decibel value based on a preset sound frequency value; the decibel value setting subunit is used to set the minimum and maximum decibel values ​​in the Fourier transform based on the preset sound frequency value; the second conversion subunit is used to convert the target decibel value into a first integer value based on the minimum and maximum decibel values; the fourth calculation subunit is used to calculate the ratio of the first integer value to the maximum frequency amplitude to obtain a normalized value; the third conversion subunit is used to convert the normalized value into a second integer value, wherein the number of bits in the first integer value is greater than the number of bits in the second integer value; and the third determining subunit is used to use the second integer value as a target sound feature.

[0113] According to another aspect of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, wherein when the computer program is executed, the device on which the computer-readable storage medium is located performs the above-described snoring detection method.

[0114] According to another aspect of this application, an electronic device is also provided, wherein the electronic device includes one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are configured to run the programs, wherein the programs are configured to execute the above-described snoring detection method during runtime.

[0115] The above-described embodiments or examples disclosed in this application are not exhaustive, but merely illustrative of some embodiments or examples, and are not intended to limit the scope of protection of this application. Unless otherwise specified, each step in a particular embodiment or example can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a particular embodiment or example can also be implemented as an independent embodiment, and the order of the steps in a particular embodiment or example can be arbitrarily interchanged. Furthermore, optional methods or examples in a particular embodiment or example can be arbitrarily combined; moreover, various embodiments or examples can be arbitrarily combined. For example, some or all steps of different embodiments or examples can be arbitrarily combined, and a particular embodiment or example can be arbitrarily combined with optional methods or examples of other embodiments or examples.

[0116] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0117] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0118] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0119] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0120] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0121] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0122] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for detecting snoring, characterized in that, include: Record the sound features of X sound segments of the target object into a data ring, and normalize the sound features recorded in the data ring to obtain the first set of target sound features, where X is an integer greater than 1; After obtaining the first set of target sound features, the data ring is subjected to Y target operations in a serial manner. Each target operation is used to delete the head sound information in each sound segment corresponding to the data ring, and to sequentially increment the sound information of the target duration at the end of each sound segment according to the sound sampling time, to obtain X updated sound segments. The target operation is also used to record the sound features of the updated X sound segments in the data ring. The duration of the head sound information is equal to the target duration. The sound features recorded in the data loop after each completion of the target operation are normalized to obtain Y sets of target sound features, where Y is a positive integer; Based on the target sound characteristics described in group Y+1, determine whether the sound emitted by the target object is snoring.

2. The snoring detection method according to claim 1, characterized in that, Determining whether the sound emitted by the target object is snoring based on the target sound characteristics described in group Y+1 includes: Based on the prior knowledge learned by the target model during the model training phase, a label corresponding to each set of target sound features is generated. The label is either a first label or a second label. The first label indicates that the sound corresponding to the target sound feature is snoring, and the second label indicates that the sound corresponding to the target sound feature is not snoring. The training data of the target model includes historical sound features with known corresponding labels. Based on the proportion of the number of target sound features in the first tag to the number of target sound features in the Y+1 group, it is determined whether the sound emitted by the target object is snoring.

3. The snoring detection method according to claim 2, characterized in that, Based on the proportion of the number of target sound features in the first tag to the number of target sound features in the Y+1 group, determine whether the sound emitted by the target object is snoring, including: If the detected proportion of the quantity is greater than a preset threshold, the sound emitted by the target object is determined to be the snoring sound; If the detected quantity percentage is less than or equal to the preset threshold, it is determined that the sound emitted by the target object is not the snoring sound.

4. The snoring detection method according to claim 1, characterized in that, The process of extracting the acoustic features of the sound segment includes: Each sound segment is subjected to frame segmentation processing, wherein the frame segmentation processing is used to divide the sound segment into K arrays of the same data size, where K is an integer greater than 1; Perform Fourier transform and Mel transform on the K arrays corresponding to the sound segment; The frequency amplitude obtained from the Fourier transform and the target sound frequency obtained from the Mel transform are used as the sound features of the sound segment.

5. The snoring detection method according to claim 4, characterized in that, Perform a Fourier transform on the K arrays corresponding to the sound segment, including: Perform a Fourier transform on the K arrays corresponding to the sound segment to obtain multiple complex numbers after transformation; The amplitudes of multiple frequencies of the sound segment are obtained by calculating the squares of the moduli of the transformed complex numbers.

6. The snoring detection method according to claim 4, characterized in that, For each sound segment, perform Mel transform on the K arrays, including: Set the maximum and minimum frequency values ​​of the Mel transform; The maximum Mel coefficient is determined based on the maximum frequency value of the Mel transform; the minimum Mel coefficient is determined based on the minimum frequency value of the Mel transform. Within the numerical range formed by the minimum Mel coefficient and the maximum Mel coefficient, P target values ​​are determined, wherein the absolute value of the difference between any two adjacent target values ​​is the same value, and P is an integer greater than 1. Calculate the frequency value corresponding to each target value; Based on the principle that the frequency value corresponding to each target value corresponds to the spectral order of the Fourier transform, the frequency values ​​adjacent to the spectral order of the frequency value corresponding to each target value are selected for weighted average calculation to obtain the target sound frequency of the sound segment.

7. The snoring detection method according to claim 4, characterized in that, The sound features recorded in the data loop are normalized to obtain the target sound features, including: The maximum frequency amplitude is determined from the frequency amplitudes of the X sound segments recorded in the data loop; Based on the maximum frequency amplitude, the target sound frequencies of the X sound segments recorded in the data loop are normalized, and the normalization result is used as the target sound feature.

8. The snoring detection method according to claim 7, characterized in that, Based on the maximum frequency amplitude, the target sound frequencies of the X sound segments recorded in the data loop are normalized, and the normalization result is used as the target sound feature, including: Based on a preset sound frequency value, the target sound frequency is converted into a target decibel value; The minimum and maximum decibel values ​​in the Fourier transform are set according to the preset sound frequency values; Based on the minimum decibel value and the maximum decibel value, the target decibel value is converted into a first integer value; Calculate the ratio of the first integer value to the maximum frequency amplitude to obtain a normalized value; The normalized value is converted into a second integer value, wherein the number of digits in the first integer value is greater than the number of digits in the second integer value; The second integer value is used as the target sound feature.

9. A snoring detection device, characterized in that, include: The first processing unit is used to record the sound features of X sound segments of the target object into a data ring, and to normalize the sound features recorded in the data ring to obtain a first set of target sound features, where X is an integer greater than 1; The second processing unit is configured to perform Y target operations on the data ring in a serial manner after obtaining the first set of target sound features. Each target operation is configured to delete the head sound information in each sound segment corresponding to the data ring, and sequentially increment the sound information of the target duration at the end of each sound segment according to the sound sampling time to obtain X updated sound segments. The target operation is also configured to record the sound features of the updated X sound segments in the data ring. The duration of the head sound information is equal to the target duration. The third processing unit is used to normalize the sound features recorded in the data loop after each completion of the target operation to obtain Y sets of target sound features, where Y is a positive integer; The fourth processing unit is used to determine whether the sound emitted by the target object is snoring based on the target sound characteristics of the Y+1 group.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed, the device containing the computer-readable storage medium performs the snoring detection method according to any one of claims 1 to 8.

11. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to perform the snoring detection method according to any one of claims 1 to 8.

Images