A method and device for determining a snoring sound signal, an electronic device and a storage medium

Abstract

The application provides a method and device for determining snoring sound signals, electronic equipment and a storage medium, and belongs to the field of signal processing. The method comprises the following steps: acquiring a sound signal, detecting a human voice segment of the sound signal, and detecting at least two human voice segments before a target human voice segment; if a starting frame of the target human voice segment is detected in the sound signal, determining whether a first current frame in the target human voice segment is a snoring sound segment starting frame; if the snoring sound segment starting frame is determined, determining a snoring sound segment ending frame from a next frame of the snoring sound segment starting frame; and taking the sound signal between the snoring sound segment starting frame and the snoring sound segment ending frame as a snoring sound signal. The application can realize real-time snoring sound signal detection on an embedded system.

Description

[0001] This application is a divisional application of the application with the application date of September 15, 2022, the application number of CN 202211118325.0, and the invention name of "a method, device, electronic equipment and storage medium for determining snoring sound signal". TECHNICAL FIELD

[0002] The present application relates to the field of signal processing, in particular to a method, device, electronic equipment and storage medium for determining snoring sound signal. BACKGROUND

[0003] Snoring is a heavy nasal breathing sound after falling asleep. Snoring not only disturbs the user himself / herself and affects the companion, but also threatens the health of the user himself / herself.

[0004] For snoring segment detection, there are few real-time detection methods that can be applied to embedded systems at present. Most of them are post-processing, fixed model and parameter methods, such as Gaussian mixture model, hidden Markov model, and more complex machine learning algorithms such as clustering, support vector machine and even deep learning algorithm. These algorithms are complex in calculation, poor in real-time performance, and even require server-level computing and storage resources for deep learning algorithms, which are difficult to implement on embedded systems.

[0005] Therefore, there is an urgent need for a method that can realize real-time snoring sound signal detection on an embedded system. SUMMARY

[0006] In order to solve the problems of the prior art, the embodiments of the present application provide a method, device, electronic equipment and storage medium for determining snoring sound signal. The technical solution is as follows:

[0007] According to an aspect of the present application, a method for determining snoring sound signal is provided, which comprises:

[0008] Obtaining a sound signal, and performing human voice segment detection on the sound signal, and detecting at least two human voice segments before a target human voice segment;

[0009] If the starting frame of the target human voice segment is detected in the sound signal, it is determined whether the first current frame in the target human voice segment is a snoring sound segment starting frame;

[0010] If the snoring sound segment starting frame is determined, the snoring sound segment ending frame is determined from the next frame of the snoring sound segment starting frame;

[0011] The sound signal between the snoring sound segment starting frame and the snoring sound segment ending frame is taken as a snoring sound signal.

[0012] According to another aspect of the present application, a device for determining snoring sound signal is provided, which comprises:

[0013] The acquisition module is configured to acquire a sound signal, detect a human voice segment in the sound signal, and detect at least two human voice segments before a target human voice segment.

[0014] The first determination module is configured to determine whether a first current frame is a snoring sound segment start frame in the target human voice segment if a start frame of the target human voice segment is detected in the sound signal.

[0015] The second determination module is configured to determine a snoring sound segment end frame from a next frame of the snoring sound segment start frame if the snoring sound segment start frame is determined, and take a sound signal between the snoring sound segment start frame and the snoring sound segment end frame as a snoring sound signal.

[0016] According to another aspect of the present application, an electronic device is provided, comprising:

[0017] a processor; and

[0018] a memory storing a program,

[0019] wherein the program includes instructions that, when executed by the processor, cause the processor to perform the method for determining a snoring sound signal.

[0020] According to another aspect of the present application, a non-transitory computer readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to perform the method for determining a snoring sound signal.

[0021] In the present application, after a sound signal is collected in real time, human voice segment detection can be performed in real time, human voice segments in the sound signal are determined, and after at least two human voice segments are determined, a snoring sound segment start frame and a snoring sound segment end frame are detected frame by frame from a start frame of a target human voice segment. Since the calculation amount of the above processing is smaller than that of Gaussian mixture model, hidden Markov model, clustering model, support vector machine and other machine learning algorithms, real-time snoring sound signal detection can be realized on an embedded system. BRIEF DESCRIPTION OF DRAWINGS

[0022] More details, features and advantages of the present application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0023] Figure 1 A schematic flow chart of a method for determining a snoring sound signal is shown according to an exemplary embodiment of the present application Figure 1 ;

[0024] Figure 2 A schematic flow chart of a method for determining a snoring sound signal is shown according to an exemplary embodiment of the present application Figure 2 ;

[0025] Figure 3A flowchart of a method of determining a vocal segment according to an exemplary embodiment of the present application is shown;

[0026] Figure 4 A flowchart of a method of determining a snoring signal according to an exemplary embodiment of the present application is shown Figure 3 ;

[0027] Figure 4 A flowchart of a method of determining a snoring signal according to an exemplary embodiment of the present application is shown Figure 5 ;

[0028] Figure 5 A flowchart of a method of determining a snoring signal according to an exemplary embodiment of the present application is shown Figure 3 ;

[0029] Figure 8 A flowchart of a method of determining a snoring segment start frame according to an exemplary embodiment of the present application is shown;

[0030] Figure 9 A schematic block diagram of a snoring segment detection apparatus according to an exemplary embodiment of the present application is shown;

[0031] Figure 1 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present application is shown. DETAILED DESCRIPTION

[0032] Embodiments of the present application will now be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present application are shown in the drawings, it is understood that the present application can be embodied in various forms and should not be interpreted in a limited sense. Rather, these embodiments are provided so that the present application will be thoroughly and completely understood. It should be understood that the drawings and embodiments of the present application are for illustrative purposes only and are not intended to limit the scope of the present application.

[0033] It should be understood that the various steps of the method embodiments of the present application can be performed in different orders and / or in parallel. Furthermore, the method embodiments can include additional steps and / or omit performing the steps shown. The scope of the present application is not limited in this respect.

[0034] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc., mentioned in this invention are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0035] It should be noted that the terms "a" and "a plurality of" used in this invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0036] The names of the messages or information exchanged between the multiple devices in the embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0037] This invention provides a method for determining snoring signals. This method can be implemented using devices employing embedded systems, such as massagers, smart bracelets, mobile terminals, and smart speakers. The method provided by this invention can be performed by any of the aforementioned devices, or by multiple devices working together; this invention does not limit this approach.

[0038] The following will refer to Figure 1 The flowchart shown illustrates the method for determining snoring signals, and the method is described below.

[0039] like Figure 2 As shown, the method includes the following steps 101-104.

[0040] Step 101: Acquire the sound signal, detect human voice segments in the sound signal, and detect at least two human voice segments before the target human voice segment;

[0041] Step 102: If the starting frame of the target human voice segment is detected in the sound signal, then determine whether the first current frame in the target human voice segment is the starting frame of the snoring segment.

[0042] Step 103: If the starting frame of the snoring segment is determined, the ending frame of the snoring segment is determined from the next frame after the starting frame of the snoring segment.

[0043] Step 104: Use the sound signal between the start frame and end frame of the snoring segment as the snoring signal.

[0044] The first current frame includes two cases: the first case is that the first current frame is the starting frame of the target human voice segment; the second case is that the first current frame is any intermediate frame after the starting frame of the target human voice segment.

[0045] Optionally, for the first case mentioned above, i.e., the first current frame is the starting frame of the target human voice segment, the process of determining whether the first current frame in the target human voice segment is the starting frame of the snoring segment can be as follows:

[0046] In the target vocal segment, based on the similarity of preset characteristics and / or snoring characteristics of the starting frame of at least two previous vocal segments with the target vocal segment, it is determined whether the starting frame of the target vocal segment is the starting frame of the snoring segment.

[0047] In some possible embodiments, a similarity score of the starting frame of the target voice segment can be calculated based on the similarity of the preset characteristics of the starting frames of the at least two voice segments and the target voice segment, and based on the similarity score of the starting frame of the target voice segment, it can be determined whether the starting frame of the target voice segment is the starting frame of the snoring segment.

[0048] In some possible embodiments, a snoring characteristic score of the starting frame of the target voice segment can be calculated based on the snoring characteristics of the starting frames of at least two voice segments and the target voice segment, and the starting frame of the target voice segment can be determined as the starting frame of the snoring segment based on the snoring characteristic score of the starting frame of the target voice segment.

[0049] In some possible embodiments, a similarity score for the starting frame of the target voice segment can be calculated based on the similarity of preset characteristics between the starting frames of at least two voice segments and the target voice segment; a snoring characteristic score for the starting frame of the target voice segment can be calculated based on the snoring characteristics of the starting frames of at least two voice segments and the target voice segment; and a determination can be made based on the similarity score and the snoring characteristic score as to whether the starting frame of the target voice segment is the starting frame of a snoring segment.

[0050] Optionally, the similarity of the above-mentioned preset characteristics may include any one or more of the following: energy similarity, spectral similarity, Mel-frequency cepstral coefficient similarity, and noise similarity;

[0051] Optional snoring characteristics include any one or more of the following: snoring energy characteristics, snoring fundamental frequency characteristics, and snoring zero-crossing rate characteristics.

[0052] For energy similarity, the process of calculating the similarity score of the starting frame of the target human voice segment can be as follows: calculate the average energy of each human voice segment in at least two human voice segments, and calculate the difference ratio between the average energy of each human voice segment. If the difference ratio is within a preset ratio range, then add the corresponding first similarity score to the similarity score of the starting frame of the target human voice segment.

[0053] For spectral similarity, the process of calculating the similarity score of the starting frame of the target human voice segment can be as follows: determine the number of first peaks in the spectrum of at least two human voice segments that meet the condition of overlapping peak positions. If the number of first peaks is greater than the first threshold, then add the corresponding second similarity score to the similarity score of the starting frame of the target human voice segment.

[0054] For Mel-Cepstral Coefficient Similarity, the similarity score of the starting frame of the target vocal segment can be calculated as follows: Calculate the Mel-Cepstral Coefficient of each vocal segment in at least two vocal segments and the Mel-Cepstral Coefficient of the starting frame of the target vocal segment. Calculate the first cross-correlation coefficient between any two Mel-Cepstral Coefficients in the starting frames of the at least two vocal segments and the target vocal segment respectively. If the first cross-correlation coefficient is greater than a first preset threshold, then add the corresponding third similarity score to the similarity score of the starting frame of the target vocal segment.

[0055] For noise similarity, the similarity score of the starting frame of the target human voice segment can be calculated as follows: Calculate the second cross-correlation coefficient between any Mel-Cepstral Coefficient of the starting frame of at least two human voice segments and the target human voice segment and the Mel-Cepstral Coefficient of the preset noise. If any second cross-correlation coefficient is greater than a second preset threshold, subtract the corresponding fourth similarity score from the similarity score of the starting frame of the target human voice segment; if all second cross-correlation coefficients are less than a third preset threshold, add the corresponding fifth similarity score to the similarity score of the starting frame of the target human voice segment.

[0056] Specifically, for spectral similarity, the process of calculating the similarity score of the starting frame of the target human voice segment can further include:

[0057] If the number of first peaks is greater than a preset threshold, then in the spectrum of the starting frame of the target voice segment, determine the number of second peaks that meet the condition of peak position overlap with at least two other voice segments. If the number of second peaks is greater than a second threshold, then add the corresponding sixth similarity score to the similarity score of the starting frame of the target voice segment.

[0058] To calculate the snoring characteristic score of the starting frame of the target vocal segment, the following steps can be taken:

[0059] Calculate the average low-frequency energy and average mid-frequency energy of each voice segment in at least two voice segments, and calculate the first ratio of the average low-frequency energy to the average mid-frequency energy of each voice segment. If the first ratio is greater than the ratio threshold, add the corresponding first snoring characteristic score to the snoring characteristic score of the starting frame of the target voice segment.

[0060] Calculate the average low-frequency energy and average mid-frequency energy of the starting frame of the target human voice segment, and calculate the second ratio of the average low-frequency energy to the average mid-frequency energy of the starting frame of the target human voice segment. If the second ratio is greater than the ratio threshold, add the corresponding second snoring characteristic score to the snoring characteristic score of the starting frame of the target human voice segment.

[0061] To calculate the snoring characteristic score of the starting frame of the target human voice segment, the fundamental frequency characteristics of snoring can be determined as follows:

[0062] Calculate the fundamental frequency of each voice segment in at least two voice segments, and calculate the fundamental frequency difference between each voice segment. If the fundamental frequency of each voice segment is within the range of snoring fundamental frequency, and the fundamental frequency difference is within the range of fundamental frequency difference, then add the corresponding third snoring characteristic score to the snoring characteristic score of the starting frame of the target voice segment.

[0063] Calculate the fundamental frequency of the starting frame of the target human voice segment. If the fundamental frequency of the starting frame of the target human voice segment is within the range of the snoring fundamental frequency, then add the corresponding fourth snoring characteristic score to the snoring characteristic score of the starting frame of the target human voice segment.

[0064] For the snoring zero-crossing rate characteristic, the snoring characteristic score of the starting frame of the target human voice segment can be calculated as follows:

[0065] Calculate the average zero-crossing rate of each voice segment in at least two voice segments. If the average zero-crossing rate of each voice segment is within the range of snoring zero-crossing rate, add the corresponding fifth snoring characteristic score to the snoring characteristic score of the starting frame of the target voice segment.

[0066] Calculate the zero-crossing rate of the starting frame of the target human voice segment. If the zero-crossing rate of the starting frame of the target human voice segment is within the range of snoring zero-crossing rates, then add the corresponding sixth snoring characteristic score to the snoring characteristic score of the starting frame of the target human voice segment.

[0067] The ratio threshold used can be updated in the following way:

[0068] When it is determined that the starting frame of the target human voice segment is the starting frame of the snoring segment, the ratio threshold is updated based on the first ratio and the second ratio mentioned above; and / or

[0069] If, after determining that the starting frame of the target human voice segment is not the starting frame of the snoring segment, the intermediate frame of the target human voice segment is determined to be the starting frame of the snoring segment, then the ratio threshold is updated based on the ratio of the average low-frequency energy and the average mid-frequency energy of the average spectrum between the starting frame of the target human voice segment and the starting frame of the snoring segment.

[0070] Optionally, for the second case above, i.e., the first current frame is any intermediate frame after the starting frame of the target human voice segment, the process of determining whether the first current frame in the target human voice segment is the starting frame of the snoring segment can be as follows:

[0071] If the intermediate frame of the target human voice segment satisfies any one or more of the following snoring initiation conditions, then the intermediate frame of the target human voice segment is determined as the snoring initiation frame.

[0072] The above-mentioned snoring initiation conditions include: snoring duration condition, Mel-frequency cepstral coefficient condition, non-noise condition, zero-crossing rate condition, spectrum condition, and fundamental frequency condition.

[0073] Determining that the intermediate frames of the target human voice segment meet the snoring duration condition means that: the number of frames between the starting frame of the target human voice segment and the first current frame is within the snoring duration range, and / or, the duration between the starting frame of the target human voice segment and the ending frame of the previous human voice segment is greater than the snoring interval duration.

[0074] Determining that the intermediate frames of the target vocal segment satisfy the Mel-Cepstral Coefficient condition means: calculating the first Mel-Cepstral Coefficient mean between the starting frame and the intermediate frames of the target vocal segment, calculating the first cross-correlation coefficient between the first Mel-Cepstral Coefficient mean and the Mel-Cepstral Coefficient reference value, and determining that the first cross-correlation coefficient is greater than the first preset threshold.

[0075] Determining that the intermediate frames of the target human voice segment meet the non-noise condition means: calculating the second cross-correlation coefficient between the mean of the first Mel-Cepstral Coefficients and the Mel-Cepstral Coefficients of the preset noise, and determining that the second cross-correlation coefficient is less than the second preset threshold.

[0076] Determining that the intermediate frames of the target vocal segment meet the zero-crossing rate condition means: calculating the average zero-crossing rate between the starting frame and the intermediate frames of the target vocal segment, and determining that the average zero-crossing rate is within the preset zero-crossing rate range.

[0077] Determining that the intermediate frame of the target vocal segment meets the spectral condition means: calculating the spectral mean between the starting frame and the intermediate frame of the target vocal segment, calculating the low-frequency energy mean and mid-frequency energy mean on the spectral mean, calculating the ratio of the mean of the low-frequency energy mean and the mean of the mid-frequency energy mean, and determining that the ratio is greater than the ratio threshold.

[0078] Determining that the intermediate frame of the target human voice segment meets the fundamental frequency condition means: determining the fundamental frequency value on the mean of the spectrum, and determining that the fundamental frequency value of the mean of the spectrum is within the range of the fundamental frequency of snoring.

[0079] The ratio threshold used can be updated in the following way:

[0080] When the starting frame of the target vocal segment is determined to be the starting frame of the snoring segment, the average low-frequency energy and average mid-frequency energy of each vocal segment in at least two vocal segments preceding the target vocal segment are calculated, and a first ratio of the average low-frequency energy to the average mid-frequency energy of each vocal segment is calculated; the average low-frequency energy and average mid-frequency energy of the starting frame of the target vocal segment are calculated, and a second ratio of the average low-frequency energy to the average mid-frequency energy of the starting frame of the target vocal segment is calculated; based on the first and second ratios, the ratio threshold is updated; and / or

[0081] When it is determined that the middle frame of the target human voice segment is the starting frame of the snoring segment, the ratio threshold is updated based on the ratio of the average low-frequency energy and the average mid-frequency energy of the average spectrum between the starting frame of the target human voice segment and the starting frame of the snoring segment.

[0082] The reference values ​​for Mel-frequency cepstral coefficients used can be updated in the following way:

[0083] When it is determined that the starting frame of the target vocal segment is the starting frame of the snoring segment, the Mel-Cepstral Coefficient (MCC) baseline value is updated based on the MMC of each of the at least two vocal segments preceding the target vocal segment and the MMC of the starting frame of the target vocal segment; and / or

[0084] When it is determined that the middle frame of the target human voice segment is the starting frame of the snoring segment, the baseline value of the Mel-Cepstral Coefficient is updated based on the mean of the Mel-Cepstral Coefficients between the starting frame of the target human voice segment and the starting frame of the snoring segment.

[0085] Optionally, the condition for determining whether the intermediate frame of the target human voice segment is the starting frame of the snoring segment can be: at least one snoring signal is detected in at least two human voice segments.

[0086] Optionally, the processing in step 103 above can be as follows:

[0087] Starting from the next frame after the beginning of the snoring segment, the second current frame is processed as follows:

[0088] Determine whether the second current frame is the end frame of the target human voice segment;

[0089] If the second current frame is the end frame of the target human voice segment, then the second current frame is taken as the end frame of the snoring segment;

[0090] If the second current frame is not the end frame of the target human voice segment, then determine whether the second current frame is the end frame of the snoring segment. If it is, then the second current frame is used as the end frame of the snoring segment; if not, then the above detection process is performed on the next frame of the second current frame.

[0091] Optionally, the process for determining whether the second current frame is the end frame of the snoring segment can be as follows:

[0092] Based on the Mel-Cepstral Coefficient baseline value and the Mel-Cepstral Coefficient of the second current frame, determine whether the second current frame is the end frame of the snoring segment.

[0093] Specifically, this can be done by: calculating the mean of the second Mel-Cepstral coefficients between the starting frame of the snoring segment and the second current frame, and calculating the third cross-correlation coefficient between the mean of the second Mel-Cepstral coefficients and the reference value of the Mel-Cepstral coefficients; if the third cross-correlation coefficient is determined to be less than the third preset threshold, then the second current frame is determined to be the ending frame of the snoring segment.

[0094] The reference values ​​for Mel-frequency cepstral coefficients used can be updated in the following way:

[0095] When it is determined that the starting frame of the target vocal segment is the starting frame of the snoring segment, the Mel-Cepstral Coefficient (MCC) baseline value is updated based on the MMC of each of the at least two vocal segments preceding the target vocal segment and the MMC of the starting frame of the target vocal segment; and / or

[0096] When it is determined that the middle frame of the target human voice segment is the starting frame of the snoring segment, the baseline value of the Mel-Cepstral Coefficient is updated based on the mean of the Mel-Cepstral Coefficients between the starting frame of the target human voice segment and the starting frame of the snoring segment.

[0097] In this embodiment of the invention, after real-time acquisition of the sound signal, human voice segments can be detected in real time to determine the human voice segments in the sound signal. After determining at least two human voice segments, starting from the start frame of the target human voice segment, the start frame and end frame of the snoring segment are detected frame by frame. Since the computational complexity of the above processing is relatively small compared to machine learning algorithms such as Gaussian mixture models, hidden Markov models, clustering models, and support vector machines, real-time snoring signal detection can be implemented on embedded systems.

[0098] The following will refer to Figure 2 The flowchart shown illustrates a method for determining snoring signals, illustrating the method provided in this embodiment. In this embodiment, the similarity of preset characteristics is primarily used to determine whether the starting frame of the target human voice segment is the starting frame of the snoring segment.

[0099] like Figure 3 As shown, the method for determining snoring signals provided in this embodiment includes the following steps 201-204.

[0100] Step 201: Acquire the sound signal, perform human voice segment detection on the sound signal, and detect at least two human voice segments before the target human voice segment.

[0101] In one possible implementation, the device may be equipped with an audio acquisition device that can acquire sound signals in real time. The acquired sound signals are then processed through signal framing, pre-emphasis, impulse noise removal, and mean removal, and then human voice segments are detected in each frame of the sound signal to identify the human voice segments within the sound signal.

[0102] Specifically, refer to Figure 4 The flowchart of the method for determining the human voice segment shown can be used to determine whether each frame of sound signal is the starting frame by referring to the following steps 301-303, and to determine the ending frame based on the following step 304.

[0103] Step 301: Take the current audio signal frame and the previous preset number of audio signal frames as the first target audio frame segment to be detected.

[0104] In one possible implementation, after determining the first target sound frame segment, the frame energy, zero-crossing rate, spectral entropy, and transform-smoothed spectral entropy value of each sound signal frame can be calculated. The transform-smoothed spectral entropy value refers to the result of calculating the spectral entropy of the power spectrum of the sound signal frame, followed by smoothing and transformation. Specifically, the calculation method can be as follows: calculate the direct spectral entropy or segmented spectral entropy of the power spectrum of the sound signal frame, perform smoothing, and finally take the negative of the spectral entropy value and add a shift parameter to obtain the transform-smoothed spectral entropy value of the sound signal frame.

[0105] Step 302: Based on the preset noise, the frame energy, spectral entropy, transform smoothing spectral entropy value, harmonic properties, and zero-crossing rate of the current sound signal frame, determine whether the current sound signal frame is a human voice frame.

[0106] In one possible implementation, when it is determined that the current sound signal frame meets the human voice frame condition, the current sound signal frame is determined to be a human voice frame, and the process proceeds to step 303, where a judgment is made based on the first target sound frame segment. The aforementioned human voice frame condition may include any one or more of the following: frame energy greater than a frame energy threshold, spectral entropy greater than a spectral entropy threshold, transform smoothing spectral entropy value greater than a transform smoothing spectral entropy value threshold, possessing harmonic characteristics, zero-crossing rate greater than a human voice frame zero-crossing rate threshold, and noise characteristics that do not conform to preset noise. The preset noise may include car engine noise, car horn noise, alarm noise, etc., and this embodiment does not limit this.

[0107] When it is determined that the current audio signal frame does not meet any of the above human voice frame conditions, it is determined that the current audio signal frame is not a human voice frame, and the next frame is detected, and steps 301-303 are repeated.

[0108] The above thresholds can be set based on a large number of experiments, and can be dynamically updated. Furthermore, upper and lower limits of the thresholds can be set based on a large number of experiments to keep these thresholds within a reasonable range. This embodiment does not limit this.

[0109] As an example, the aforementioned threshold can be updated when it differs significantly from the value to be compared. For instance, if the transform smoothing spectral entropy value of the current frame differs significantly from the corresponding transform smoothing spectral entropy threshold, the transform smoothing spectral entropy threshold can be updated. Specifically:

[0110] When the value to be compared is greater than the corresponding threshold and the difference reaches the set value, the threshold is increased based on the first weighting factor corresponding to the increased threshold.

[0111] When the value to be compared is less than the corresponding threshold and the difference reaches the set value, the threshold is reduced based on the second weighting factor corresponding to the threshold.

[0112] In one possible implementation, the threshold can be adjusted based on the following formula (1):

[0113] Th_new = (1-a)*Th+a*x (1)

[0114] Where Th_new is the updated threshold, Th is the original threshold, x is the value to be compared, and a is the weighting factor, with a value ranging from (0, 1). When increasing the threshold, a can be a1; when decreasing the threshold, a can be a2.

[0115] The first weighting factor a1 corresponding to increasing the threshold and the second weighting factor a2 corresponding to decreasing the threshold can be set according to experience. The values ​​of a1 and a2 can be the same or different. This embodiment does not limit this.

[0116] Step 303: Based on the preset noise, the frame energy, spectral entropy, transform smooth spectral entropy, harmonic properties and zero-crossing rate of the first target sound frame, determine whether the current sound signal frame is the starting frame.

[0117] In one possible implementation, within the first target audio frame segment, it can be determined whether each audio signal frame satisfies the aforementioned human voice frame condition, and the number of frames that satisfy the human voice frame condition can be counted. When the number of frames that satisfy the human voice frame condition is greater than a frame count threshold, the current audio signal frame can be used as the starting frame.

[0118] When the number of frames that meet the human voice frame condition is not greater than the frame number threshold, the current sound signal frame can be considered not to be a human voice frame. Then, the next frame is detected, and steps 301-303 are repeated.

[0119] The method for determining snoring signals provided in this embodiment can detect at least two human voice segments before detecting snoring segments in the target human voice segment. Therefore, if no two human voice segments are detected in the sound signal, after detecting the start frame in step 303, the process proceeds to step 304 to detect the end frame; if at least two human voice segments are detected in the sound signal, after detecting the start frame in step 303, the process proceeds to step 202 to determine whether the start frame is the start frame of the snoring segment.

[0120] Step 304: Take the current sound signal frame and the previous preset number of sound signal frames as the second target sound frame segment to be detected. Based on the preset noise, the frame energy, spectral entropy, transform smooth spectral entropy, harmonic properties and zero crossing rate of the second target sound frame segment, determine whether the current sound signal frame is the end frame.

[0121] In one possible implementation, after detecting the start frame in step 303, after a certain number of frames, the current audio signal frame is checked frame by frame to determine whether it is the end frame. Specifically, in the second target audio frame segment, it can be determined whether each audio signal frame meets the above-mentioned human voice frame condition, and the number of frames that meet the human voice frame condition is counted.

[0122] When the number of frames that meet the human voice frame condition is greater than the frame number threshold, the current sound signal frame is considered to be a human voice frame. Then, the processing of step 304 is performed on the next frame to determine whether the sound signal of the frame is the end frame.

[0123] When the number of frames that meet the human voice frame condition is not greater than the frame number threshold, the current audio signal frame is taken as the end frame. In this case, the audio signal between the start frame and the end frame can be considered as a single human voice segment.

[0124] After that, the processing of steps 301-304 above can be performed on the next frame after the end frame to determine each human voice segment in the sound signal, and snoring segment detection can be performed in the human voice segment based on steps 202-204 below.

[0125] Step 202: If the starting frame of the target human voice segment is detected in the sound signal, then based on the similarity of the preset characteristics of the starting frame of the target human voice segment with the starting frame of the target human voice segment, determine whether the starting frame of the target human voice segment is the starting frame of the snoring segment.

[0126] In one possible implementation, when two voice segments are detected in the sound signal, snoring segment detection can begin from the third voice segment. Optionally, to ensure the accuracy of snoring segment detection, in at least two voice segments preceding the target voice segment, the length of each voice segment meets a preset length condition (e.g., within the snoring length range), and the interval between two adjacent voice segments is greater than a preset interval duration.

[0127] In this embodiment, the human voice segment for which snoring is detected is referred to as the target human voice segment, and the same applies below.

[0128] Compared to spoken voices, snoring has a certain degree of repetition. Therefore, whenever the starting frame of a target voice segment is detected, it can be determined whether at least two preceding voice segments are similar. If the preceding two voice segments are similar, the probability that the collected sound belongs to snoring is relatively high. Furthermore, it can be determined whether the starting frame of the target voice segment is similar to at least two preceding voice segments. If the starting frame of the target voice segment is similar to at least two preceding voice segments, it indicates that the probability of that starting frame belonging to snoring is also relatively high, and this starting frame can be used as the starting frame of the snoring segment.

[0129] Optionally, to further improve the accuracy of determining whether the starting frame is the starting frame of the snoring segment, the processing in step 202 above can be as follows:

[0130] Calculate the similarity score of the starting frame based on the similarity of the preset characteristics of the starting frames of at least two human voice segments and the target human voice segment.

[0131] Based on the similarity score of the starting frame, it is determined whether the starting frame of the target human voice segment is the starting frame of the snoring segment.

[0132] The similarity of the aforementioned preset characteristics may include any one or more of the following: energy similarity, spectral similarity, Mel-frequency cepstral coefficient similarity, and noise similarity.

[0133] In one possible implementation, it can be determined whether the starting frame of the target vocal segment has similar energy to at least two previous vocal segments. If there is energy similarity, the similarity score of the starting frame can be increased.

[0134] It can be determined whether the starting frame of the target vocal segment has similarity to the spectrum of at least two previous vocal segments. If there is spectral similarity, the similarity score of the starting frame can be increased.

[0135] It can be determined whether the starting frame of the target vocal segment has similarity to the Mel-Cepstral coefficients of at least two previous vocal segments. If there is similarity in Mel-Cepstral coefficients, the similarity score of the starting frame can be increased.

[0136] By using the noise characteristics of preset noise, it can be determined whether the starting frame of the target human voice segment matches the noise characteristics of at least two previous human voice segments. If they match, it indicates that the starting frame or human voice segment may belong to noise, and the similarity score of the starting frame can be reduced.

[0137] The specific calculation methods for each of the above similarities will be introduced below.

[0138] For energy similarity, the similarity score of the starting frame can be calculated as follows: Calculate the average energy of each voice segment in at least two voice segments, and calculate the ratio of the difference between the average energy of each voice segment. If the ratio of the difference is within a preset range, add the corresponding first similarity score to the similarity score of the starting frame. As an example, process the two voice segments preceding the target voice segment. Sort the frame energy of each frame in each voice segment in descending order, take the first certain number of frames (e.g., the first 10 frames), calculate the average frame energy and use it as the average energy of that voice segment; calculate the difference between the average energy of the two voice segments, and calculate the ratio of this difference (e.g., calculate the ratio between the difference in average energy and the average energy of any voice segment); if this ratio of the difference is within a preset range (e.g., 0 to 0.1), it indicates that the energy of the two voice segments is similar, and 1 point can be added to the similarity score of the starting frame.

[0139] For spectral similarity, the similarity score of the starting frame can be calculated as follows: Determine the number of first peaks in the spectra of at least two voice segments that meet the condition of overlapping peak positions. If the number of first peaks is greater than a first threshold, then add a corresponding second similarity score to the similarity score of the starting frame. As an example, process two voice segments preceding the target voice segment, calculate the average spectrum of each voice segment, and search for a certain number (e.g., 5) of peaks in the average spectrum, and record the corresponding positions of the peaks. Compare the peak positions of the two voice segments one by one, and record the number of peaks with similar positions (i.e., the above-mentioned condition of overlapping peak positions). If the number of similar peak positions exceeds the first threshold, it indicates that the spectra of the two voice segments are similar, and then 1 point can be added to the similarity score of the starting frame.

[0140] For Mel-Cepstral Coefficient Similarity, the similarity score of the starting frame can be calculated as follows: Calculate the Mel-Cepstral Coefficient of each of the at least two human voice segments and the Mel-Cepstral Coefficient of the starting frame of the target human voice segment. Calculate the first cross-correlation coefficient between any two Mel-Cepstral Coefficients in the starting frames of the at least two human voice segments and the target human voice segment, respectively. If the first cross-correlation coefficient is greater than a first preset threshold, then add the corresponding third similarity score to the similarity score of the starting frame. As an example, the starting frame of the target vocal segment and the two preceding vocal segments are processed. The Mel-Cepstral Coefficients (MCCs) of the starting frame and each vocal segment are calculated separately. For a vocal segment, the MMCs of each frame can be calculated, the average MMC is obtained, and the average MMC is used as the MMC of the vocal segment. The cross-correlation coefficient between the MMCs of the two vocal segments is calculated. If the cross-correlation coefficient is greater than a first preset threshold, it indicates that the two vocal segments have MMC similarity, and 1 point can be added to the similarity score of the starting frame. The cross-correlation coefficients between the starting frame and the first vocal segment, and the cross-correlation coefficients between the starting frame and the second vocal segment are also calculated. If both cross-correlation coefficients are greater than the first preset threshold, it indicates that the starting frame and both vocal segments have MMC similarity, and 1 point can be added to the similarity score of the starting frame.

[0141] For noise similarity, the similarity score of the starting frame can be calculated as follows: calculate the second cross-correlation coefficient between any Mel-Cepstral Coefficient of the starting frame of at least two human voice segments and the target human voice segment and the Mel-Cepstral Coefficient of the preset noise. If any second cross-correlation coefficient is greater than a second preset threshold, subtract the corresponding fourth similarity score from the similarity score of the starting frame; if all second cross-correlation coefficients are less than a third preset threshold, add the corresponding fifth similarity score to the similarity score of the starting frame. As an example, the starting frame of the target vocal segment and the two preceding vocal segments are processed. The Mel-frequency cepstral coefficients (MFCs) of the starting frame and each vocal segment, as well as the MFC of the preset noise, are calculated. The cross-correlation coefficients between the preset noise and the MFC of the first vocal segment, and between the preset noise and the MFC of the second vocal segment, are calculated. If any cross-correlation coefficient is greater than a second preset threshold, it indicates that any vocal segment is similar to the preset noise, and the similarity score of the starting frame is reduced by 1 point. If both cross-correlation coefficients are less than a third preset threshold, it indicates that neither vocal segment is similar to the preset noise, and the similarity score of the starting frame is increased by 1 point. The cross-correlation coefficients between the preset noise and the MFC of the starting frame are calculated. If the cross-correlation coefficient is greater than a second preset threshold, it indicates that the starting frame is similar to the preset noise, and the similarity score of the starting frame is reduced by 1 point. If the cross-correlation coefficient is less than a third preset threshold, it indicates that the starting frame is not similar to the preset noise, and the similarity score of the starting frame is increased by 1 point.

[0142] Optionally, for spectral similarity, the process of calculating the similarity score of the starting frame can further include: if the number of first peaks is greater than a preset threshold, then in the spectrum of the starting frame, determine the number of second peaks that satisfy the condition of peak position overlap with at least two human voice segments; if the number of second peaks is greater than a second threshold, then add a corresponding sixth similarity score to the similarity score of the starting frame. As an example, in the above process of calculating spectral similarity, if the number of peak positions that are close in two human voice segments exceeds the first threshold, then the maximum number of peaks can be searched in the spectrum of the starting frame, and the corresponding positions of the peaks can be recorded; the peak positions of the starting frame and the two human voice segments are compared one by one, and the number of peak positions that are close is recorded; if the number of peak positions that are close exceeds the second threshold, it indicates that the spectrum of the starting frame and the two human voice segments are similar, and then 1 point can be added to the similarity score of the starting frame.

[0143] After completing the above similarity calculation, the similarity score of the starting frame can be judged. If the similarity score is greater than the score threshold, the starting frame can be regarded as the starting frame of the snoring segment; if the similarity score is not greater than the score threshold, the starting frame can be considered not to be the starting frame of the snoring segment.

[0144] It should be noted that the scores corresponding to the above similarities can be adjusted. For example, if the accuracy of the Mel-Cepstral Coefficients similarity is higher, the third score corresponding to the Mel-Cepstral Coefficients similarity can be increased. This embodiment does not limit this.

[0145] If the starting frame of the target human voice segment is not the starting frame of the snoring segment, then two processing methods can be used thereafter:

[0146] The first processing method is to detect the end frame of the target human voice segment frame by frame and no longer detect the snoring segment;

[0147] The second processing method is to determine whether the intermediate frame is the starting frame of the snoring segment during the process of detecting the ending frame of the target human voice segment frame by frame after determining the starting frame of the target human voice segment.

[0148] In both of the above processing methods, the specific processing of detecting the end frame of the target human voice segment frame by frame is the same as step 304 above, and will not be repeated here.

[0149] For the first processing method, if the starting frame of the target human voice segment is not the starting frame of the snoring segment, it is considered that the target human voice segment does not contain snoring. Therefore, snoring segment detection will not continue in this target human voice segment. After the next human voice segment is detected, snoring segment detection will be performed on the human voice segment.

[0150] To improve the accuracy of snoring segment detection, the second processing method described above can also be used to determine whether the intermediate frame of the target voice segment is the starting frame of the snoring segment. The specific processing method can employ existing snoring segment detection methods; this embodiment does not limit the specific method for determining whether the intermediate frame of the target voice segment is the starting frame of the snoring segment. Optionally, to further improve the accuracy of snoring segment detection, this application will introduce a method for determining whether the intermediate frame of the target voice segment is the starting frame of the snoring segment in another embodiment; this embodiment will not elaborate on this method here.

[0151] Optionally, the condition for adopting the second processing method described above can be that at least one snoring signal is detected in the human voice segments detected before the target human voice segment. As an example, assuming that the starting frame is not determined to be the starting frame of the snoring segment in the detected third human voice segment, then snoring segment detection is not continued in the third human voice segment; if the starting frame is determined to be the starting frame of the snoring segment in the detected fourth human voice segment, then the corresponding snoring signal is determined in the fourth human voice segment, and the snoring count is incremented by 1; when the fifth human voice segment is detected, if the starting frame is not determined to be the starting frame of the snoring segment, then the second processing method described above can be used to determine whether the intermediate frame is the starting frame of the snoring segment based on the snoring count. The reason for adopting this optional scheme is that when no snoring signal has been detected, the probability that the intermediate frame of the human voice segment is the starting frame of the snoring segment is small. In order to reduce the consumption of computing resources, the determination of the starting frame of the snoring segment can be performed on the intermediate frame after the snoring signal is detected.

[0152] For ease of explanation, in this embodiment, the current frame involved in the detection of the start frame of the snoring segment is referred to as the first current frame, and the current frame involved in the detection of the end frame of the snoring segment is referred to as the second current frame.

[0153] Step 203: If the starting frame of the snoring segment is determined, the ending frame of the snoring segment is determined starting from the next frame after the starting frame of the snoring segment.

[0154] Specifically, the processing in step 203 can be as follows:

[0155] Starting from the next frame after the beginning of the snoring segment, the second current frame is processed as follows:

[0156] Determine whether the second current frame is the end frame of the target human voice segment;

[0157] If the second current frame is the end frame of the target human voice segment, then the second current frame is taken as the end frame of the snoring segment;

[0158] If the second current frame is not the end frame of the target human voice segment, then determine whether the second current frame is the end frame of the snoring segment. If it is, then the second current frame is used as the end frame of the snoring segment; if not, then the above detection process is performed on the next frame of the second current frame.

[0159] In one possible implementation, after determining the start frame of the snoring segment, it can be determined whether the next frame is the end frame. The specific processing is the same as step 304 above, and will not be repeated here. If an end frame is determined, it is considered that the snoring has also ended, and the second current frame (i.e., the end frame) can be used as the end frame of the snoring segment.

[0160] If the second current frame is not determined as the end frame, then determine whether the second current frame is the end frame of the snoring segment.

[0161] Optionally, it can be determined whether the second current frame is the end frame of the snoring segment based on the Mel cepstral coefficient reference value and the Mel cepstral coefficient of the second current frame.

[0162] The Mel cepstral coefficient reference value refers to the Mel cepstral coefficient reference value of snoring.

[0163] In one possible implementation, the Mel-Cepstral Coefficients of the second current frame can be calculated, and the cross-correlation coefficient between the Mel-Cepstral Coefficients and a reference value can be calculated. It can then be determined whether the cross-correlation coefficient is less than a corresponding threshold. If the cross-correlation coefficient is less than the corresponding threshold, the second current frame can be considered not to belong to snoring, and thus the second current frame is designated as the end frame of the snoring segment.

[0164] In another possible implementation, the Mel-frequency cepstral coefficients (MFCCs) of each frame between the start frame of the snoring segment and the second current frame can be calculated, and their average value can be taken as the average MFCC from the start frame to the second current frame. The cross-correlation coefficient between this average MFCC and the baseline MFCC is calculated, and it is determined whether the cross-correlation coefficient is less than a corresponding threshold. If the cross-correlation coefficient is less than the corresponding threshold, the second current frame can be considered not to belong to snoring, and this second current frame is then taken as the end frame of the snoring segment. Compared with the previous implementation, the end frame of the snoring segment can be determined after a certain delay of a certain number of frames. Based on this, it can be ensured that the snoring signal contains a larger amount of snoring information.

[0165] If the cross-correlation coefficient is not less than the corresponding threshold, then the next frame will continue to be processed in step 203.

[0166] Alternatively, if the aforementioned cross-correlation coefficient is not less than the corresponding threshold, then it is further determined whether the number of frames from the start frame of the snoring segment to the second current frame is greater than the snoring frame count threshold. If the number of frames is greater than the snoring frame count threshold, then the second current frame can be used as the end frame of the snoring segment; if the number of frames is not greater than the snoring frame count threshold, then the processing of step 203 above continues for the next frame.

[0167] Optionally, the aforementioned Mel-Cepstral coefficient reference values ​​can be updated adaptively. This embodiment provides two update methods as follows:

[0168] Method 1: When it is determined that the starting frame of the target vocal segment is the starting frame of the snoring segment, the above-mentioned Mel-Cepstral Coefficient baseline value is updated based on the Mel-Cepstral Coefficients of each vocal segment in at least two vocal segments preceding the target vocal segment and the Mel-Cepstral Coefficients of the starting frame of the target vocal segment.

[0169] In one possible implementation, when the starting frame of the target vocal segment is determined to be the starting frame of the snoring segment, the Mel-spectral coefficients (MSCs) of each frame in the at least two vocal segments can be calculated, and the MSC of the starting frame (i.e., the starting frame of the snoring segment) can also be calculated. The average MSC of the at least two vocal segments and the starting frame is then obtained. This average MSC is used as the reference value for the MSCs used in the target vocal segment.

[0170] Method 2: If, after determining that the starting frame of the target human voice segment is not the starting frame of the snoring segment, the intermediate frame of the target human voice segment is determined to be the starting frame of the snoring segment, then the Mel-Cepstral Coefficient Baseline Value is updated based on the mean Mel-Cepstral Coefficient between the starting frame of the target human voice segment and the starting frame of the snoring segment.

[0171] In one possible implementation, when it is determined that the intermediate frame of the target vocal segment is the starting frame of the snoring segment, the Mel-frequency cepstral coefficients of each frame from the starting frame of the target vocal segment to the starting frame of the snoring segment can be calculated, and the average value of the Mel-frequency cepstral coefficients can be obtained. Furthermore, the reference value of the Mel-frequency cepstral coefficients can be replaced with this average value.

[0172] For the detection of the snoring segment of the next target human voice segment, if the starting frame of the snoring segment is determined, the reference value of the Mel-Cepstral Coefficient used by the target human voice segment can also be determined based on the above processing, thereby realizing the adaptive update of the reference value of the Mel-Cepstral Coefficient.

[0173] Since the snoring signals of different target human voice segments may have different Mel-Cepstral Coefficients, through the above-mentioned update process, the reference value of the Mel-Cepstral Coefficient used for each target human voice segment can be adapted to the snoring of that target human voice segment, thereby improving the accuracy of determining the end frame of the snoring segment and thus improving the accuracy of snoring segment detection.

[0174] Step 204: Use the sound signal between the start frame and end frame of the snoring segment as the snoring signal.

[0175] In one possible implementation, the start frame and end frame of the snoring segment can be recorded. When analysis of the snoring signal is required, the corresponding snoring signal can be extracted based on the start and end frames of the snoring segment. This embodiment does not limit the subsequent processing of the snoring signal.

[0176] This embodiment can achieve the following beneficial effects:

[0177] (1) After real-time acquisition of sound signals, human voice segments can be detected in real time to determine the human voice segments in the sound signal. After determining at least two human voice segments, starting from the starting frame of the target human voice segment, snoring segment detection is performed based on the similarity of the preset characteristics of the starting frame of the at least two human voice segments and the target human voice segment. Since the computational cost of the above processing is relatively small compared to machine learning algorithms such as Gaussian mixture models, hidden Markov models, clustering models, and support vector machines, real-time snoring segment detection can be implemented on embedded systems.

[0178] (2) In the process of calculating the similarity score of the starting frame, the similarity of multiple preset characteristics such as energy similarity, spectrum similarity, Mel-frequency cepstral coefficient similarity, and noise similarity is adopted, which improves the detection accuracy and noise resistance.

[0179] (3) Adaptively adjust the reference value of the Mel-Cepstral Coefficient so that the reference value of the Mel-Cepstral Coefficient used for each target human voice segment can be matched with the snoring of the target human voice segment, thereby improving the accuracy of determining the end frame of the snoring segment and thus improving the accuracy of snoring segment detection.

[0180] The following will refer to Figure 4 The flowchart shown illustrates a method for determining snoring signals, illustrating the method provided in this embodiment. In this embodiment, snoring characteristics are primarily used to determine whether the starting frame of the target human voice segment is the starting frame of the snoring segment.

[0181] like Figure 3 As shown, the method for determining snoring signals provided in this embodiment includes the following steps 401-404.

[0182] Step 401: Acquire the sound signal, perform human voice segment detection on the sound signal, and detect at least two human voice segments before the target human voice segment.

[0183] In one possible implementation, the device may be equipped with an audio acquisition device that can acquire sound signals in real time. The acquired sound signals are then processed through signal framing, pre-emphasis, impulse noise removal, and mean removal, and then human voice segments are detected in each frame of the sound signal to identify the human voice segments within the sound signal.

[0184] The specific method for determining the human voice segment can be found in the above embodiments. Figure 5 The flowchart of the method for determining the human voice segment shown is similar to the above embodiment, and will not be repeated in this embodiment.

[0185] After determining the end frame of the human voice segment, the processing steps 301-304 above can be performed on the next frame after the end frame to determine each human voice segment in the sound signal, and snoring segment detection can be performed in the human voice segment based on the following steps 402-404.

[0186] The method for determining snoring signals provided in this embodiment can detect at least two human voice segments before detecting snoring segments in the target human voice segment. Therefore, if no human voice segments are detected in the sound signal, the end frame can be detected after the start frame is detected; if at least two human voice segments are detected in the sound signal, the following step 402 is entered after the start frame is detected to determine whether the start frame is the start frame of the snoring segment.

[0187] Step 402: If the starting frame of the target human voice segment is detected in the sound signal, then based on the snoring characteristics of the starting frames of at least two human voice segments and the target human voice segment, determine whether the starting frame of the target human voice segment is the starting frame of the snoring segment.

[0188] In one possible implementation, when two voice segments are detected in the sound signal, snoring segment detection can begin from the third voice segment. Optionally, to ensure the accuracy of snoring segment detection, in at least two voice segments preceding the target voice segment, the length of each voice segment meets a preset length condition (e.g., within the snoring length range), and the interval between two adjacent voice segments is greater than a preset interval duration.

[0189] In this embodiment, the human voice segment for which snoring is detected is referred to as the target human voice segment, and the same applies below.

[0190] Snoring has certain characteristics. Therefore, whenever the starting frame of a target person's voice segment is detected, it can be determined whether at least two voice segments preceding the target person's voice segment and the starting frame possess snoring characteristics. If snoring characteristics are present, the probability that the collected sound belongs to snoring is high, and this starting frame can be used as the starting frame of the snoring segment.

[0191] Optionally, to further improve the accuracy of determining whether the starting frame is the starting frame of the snoring segment, the processing in step 402 above can be as follows:

[0192] Based on the snoring characteristics of the starting frames of at least two human voice segments and the target human voice segment, calculate the snoring characteristic score of the starting frame.

[0193] Based on the snoring characteristic score of the starting frame, determine whether the starting frame of the target human voice segment is the starting frame of the snoring segment.

[0194] The aforementioned snoring characteristics may include any one or more of the following: snoring energy characteristics, snoring fundamental frequency characteristics, and snoring zero-crossing rate characteristics.

[0195] The principle behind snoring characteristics is as follows:

[0196] Snoring is characterized by higher low-frequency energy than mid-frequency energy. Therefore, whether a sound is snoring can be determined by whether the low-frequency energy is higher than the mid-frequency energy. Low frequency refers to the frequency range of 50Hz to 400Hz, and mid frequency refers to the frequency range of 400Hz to 4000Hz. The settings of low and mid frequencies are adapted to the characteristics of snoring.

[0197] Since the fundamental frequency of snoring falls within a certain range, the fundamental frequency range of snoring can be determined by statistically analyzing the pre-collected snoring signals. Therefore, the fundamental frequency range of snoring can be used to determine whether a snoring sound possesses fundamental frequency characteristics.

[0198] The zero-crossing rate of snoring also falls within a certain range. Frame zero-crossing rate statistics can be performed on pre-collected snoring signals to determine the range of snoring zero-crossing rates. Therefore, the range of snoring zero-crossing rates can be used to determine whether a snoring device possesses the zero-crossing rate characteristic.

[0199] The specific calculation methods for each of the above-mentioned snoring characteristics will be introduced below.

[0200] To calculate the snoring energy characteristics score of the starting frame, the following steps can be taken:

[0201] Calculate the average low-frequency energy and average mid-frequency energy of each voice segment in at least two voice segments, and calculate the first ratio of the average low-frequency energy to the average mid-frequency energy of each voice segment. If the first ratio is greater than the ratio threshold, add the corresponding first snoring characteristic score to the snoring characteristic score of the starting frame.

[0202] Calculate the average low-frequency energy and average mid-frequency energy of the starting frame, and calculate the second ratio of the average low-frequency energy to the average mid-frequency energy of the starting frame. If the second ratio is greater than the ratio threshold, add the corresponding second snoring characteristic score to the snoring characteristic score of the starting frame.

[0203] As an example, two voice segments preceding the target voice segment are processed. For each voice segment, each frame of sound signal can be processed by Fast Fourier Transform (FFT) to obtain the Fourier transform result, and the average frequency band energy is calculated using the following formula (2):

[0204]

[0205] Where A_mean is the average energy of the frequency band, n1 is the frequency point corresponding to the lower limit frequency, n2 is the frequency point corresponding to the upper limit frequency, and |f(n)| is the absolute value of the nth spectral value in the Fourier transform result. Alternatively, the above |f(n)| can also be replaced by the square of the absolute value of the nth spectral value in the Fourier transform result, i.e., |f(n)| 2 .

[0206] When calculating the low-frequency energy mean, the above A_mean can represent the low-frequency energy mean, where n1 is the frequency point corresponding to the lower limit frequency of the low frequency (e.g., 50Hz) and n2 is the frequency point corresponding to the upper limit frequency of the low frequency (e.g., 400Hz).

[0207] When calculating the average intermediate frequency energy, A_mean above can represent the average intermediate frequency energy, n1 is the frequency point corresponding to the lower limit of the intermediate frequency (e.g., 400Hz), and n2 is the frequency point corresponding to the upper limit of the intermediate frequency (e.g., 4000Hz).

[0208] Furthermore, the ratio of the average low-frequency energy to the average mid-frequency energy of each voice segment can be calculated, which is referred to as the first ratio in this embodiment. If the first ratio of both voice segments is greater than the ratio threshold, it indicates that both voice segments possess snoring energy characteristics, and 1 point can be added to the snoring characteristic score of the starting frame.

[0209] Similarly, the ratio of the average low-frequency energy to the average mid-frequency energy of the starting frame can be calculated; in this embodiment, this is referred to as the second ratio. If the second ratio of the starting frame is greater than the ratio threshold, it indicates that the starting frame possesses snoring energy characteristics, and 1 point can be added to the snoring characteristic score of the starting frame.

[0210] To calculate the snoring characteristic score of the starting frame, the fundamental frequency characteristics of snoring can be determined as follows:

[0211] Calculate the fundamental frequency of each voice segment in at least two voice segments, and calculate the fundamental frequency difference between each voice segment. If the fundamental frequency of each voice segment is within the range of snoring fundamental frequency, and the fundamental frequency difference is within the range of fundamental frequency difference, then add the corresponding third snoring characteristic score to the snoring characteristic score of the starting frame.

[0212] Calculate the fundamental frequency of the starting frame. If the fundamental frequency of the starting frame is within the range of snoring fundamental frequency, add the corresponding fourth snoring characteristic score to the snoring characteristic score of the starting frame.

[0213] As an example, two voice segments preceding the target voice segment are processed. The average spectrum of each voice segment is calculated, and the fundamental frequency is determined from the average spectrum as the fundamental frequency of that voice segment. Then, the fundamental frequency difference between the two voice segments is calculated. If the fundamental frequency of each voice segment is within the range of snoring fundamental frequency, it indicates that both voice segments possess snoring fundamental frequency characteristics. If the fundamental frequency difference is within the range of fundamental frequency difference, it indicates that the fundamental frequencies of the two voice segments have a certain continuity, which conforms to the characteristics of snoring. In this case, 1 point can be added to the snoring characteristic score of the starting frame.

[0214] If the fundamental frequency of the starting frame is within the range of snoring fundamental frequency, it indicates that the starting frame has snoring fundamental frequency characteristics, and 1 point can be added to the snoring characteristic score of the starting frame.

[0215] For the snoring zero-crossing rate characteristic, the snoring characteristic score of the starting frame can be calculated as follows:

[0216] Calculate the average zero-crossing rate of each voice segment in at least two voice segments. If the average zero-crossing rate of each voice segment is within the range of snoring zero-crossing rate, add the corresponding fifth snoring characteristic score to the snoring characteristic score of the starting frame.

[0217] Calculate the zero-crossing rate of the starting frame. If the zero-crossing rate of the starting frame is within the snoring zero-crossing rate range, then add the corresponding sixth snoring characteristic score to the snoring characteristic score of the starting frame.

[0218] As an example, we can process two voice segments preceding the target voice segment. For each voice segment, we can calculate the zero-crossing rate of each frame and take the average of the values. If the average zero-crossing rate of each voice segment is within the range of snoring zero-crossing rates, it indicates that both voice segments possess snoring zero-crossing rate characteristics, and we can add 1 point to the snoring characteristic score of the starting frame. Alternatively, the process of calculating the average zero-crossing rate can also be as follows: for each voice segment, we can calculate the zero-crossing rate of each frame, sort the zero-crossing rates of each frame in descending order, and take the average of the zero-crossing rates of the preceding set number of frames (e.g., 10 frames) as the average zero-crossing rate of that voice segment.

[0219] Similarly, the zero-crossing rate is calculated for the starting frame. If the zero-crossing rate of the starting frame is within the range of snoring zero-crossing rates, it indicates that the starting frame has snoring zero-crossing rate characteristics, and 1 point can be added to the snoring characteristic score of the starting frame.

[0220] After completing the above calculation of snoring characteristics, the snoring characteristic score of the starting frame can be judged. If the snoring characteristic score is greater than the score threshold, the starting frame can be regarded as the starting frame of the snoring segment; if the snoring characteristic score is not greater than the score threshold, the starting frame can be considered not to be the starting frame of the snoring segment.

[0221] It should be noted that the scores corresponding to the various snoring characteristics mentioned above can be adjusted. For example, if the accuracy of the snoring energy characteristics is higher, the score corresponding to the snoring energy characteristics can be increased. This embodiment does not limit this.

[0222] Optionally, the threshold ratio of the low-frequency energy mean to the mid-frequency energy mean used in the above calculation of snoring energy characteristics can be adaptively updated. This embodiment provides two update methods as follows:

[0223] Method 1: When it is determined that the starting frame of the target human voice segment is the starting frame of the snoring segment, the ratio threshold in the spectrum condition can be updated based on the first ratio and the second ratio determined in the above process.

[0224] In one possible implementation, when it is determined that the starting frame of the target human voice segment is the starting frame of the snoring segment, a weighted average can be performed based on the first ratio of at least two human voice segments preceding the target human voice segment, the second ratio of the target human voice segment, and the ratio threshold to be updated. The resulting weighted average is used as the updated ratio threshold, which is then used as the ratio threshold for the next target human voice segment.

[0225] Method 2: If, after determining that the starting frame of the target human voice segment is not the starting frame of the snoring segment, the intermediate frame of the target human voice segment is determined to be the starting frame of the snoring segment, then the ratio threshold is updated based on the ratio of the average low-frequency energy and the average mid-frequency energy of the average spectrum between the starting frame of the target human voice segment and the starting frame of the snoring segment.

[0226] In one possible implementation, when it is determined that the middle frame of the target human voice segment is the starting frame of the snoring segment, the above-mentioned mean ratio can be weighted and averaged with the ratio threshold to be updated, and the resulting weighted average can be used as the updated ratio threshold, which is then used as the ratio threshold for the next target human voice segment.

[0227] The ratio threshold that has never been updated can be set as the initial value. The ratio threshold can also be set with upper and lower limits based on a large number of experiments to keep the ratio threshold within a reasonable range.

[0228] Since the energy ratio of low and mid frequencies in each user's snoring is different, and the snoring of the same user may also be different at different times, the above processing can adaptively adjust the low and mid frequency ratio threshold when the starting frame of the target voice segment is determined to be the starting frame of the snoring segment. This makes the ratio threshold match the current snoring, improves the accuracy of determining the starting frame of the snoring segment, and thus improves the accuracy of snoring segment detection.

[0229] If the starting frame of the target human voice segment is not the starting frame of the snoring segment, then two processing methods can be used thereafter:

[0230] The first processing method is to detect the end frame of the target human voice segment frame by frame and no longer detect the snoring segment;

[0231] The second processing method is to determine whether the intermediate frame is the starting frame of the snoring segment during the process of detecting the ending frame of the target human voice segment frame by frame after determining the starting frame of the target human voice segment.

[0232] In both of the above processing methods, the specific processing of detecting the end frame of the target human voice segment frame by frame is the same as step 304 above, and will not be repeated here.

[0233] For the first processing method, if the starting frame of the target human voice segment is not the starting frame of the snoring segment, it is considered that the target human voice segment does not contain snoring. Therefore, snoring segment detection will not continue in this target human voice segment. After the next human voice segment is detected, snoring segment detection will be performed on the human voice segment.

[0234] To improve the accuracy of snoring segment detection, the second processing method described above can also be used to determine whether the intermediate frame of the target voice segment is the starting frame of the snoring segment. The specific processing method can employ existing snoring segment detection methods; this embodiment does not limit the specific method for determining whether the intermediate frame of the target voice segment is the starting frame of the snoring segment. Optionally, to further improve the accuracy of snoring segment detection, this application will introduce a method for determining whether the intermediate frame of the target voice segment is the starting frame of the snoring segment in another embodiment; this embodiment will not elaborate on this method here.

[0235] Optionally, the condition for adopting the second processing method described above can be that at least one snoring signal is detected in the human voice segments detected before the target human voice segment. As an example, assuming that the starting frame is not determined to be the starting frame of the snoring segment in the detected third human voice segment, then snoring segment detection is not continued in the third human voice segment; if the starting frame is determined to be the starting frame of the snoring segment in the detected fourth human voice segment, then the corresponding snoring signal is determined in the fourth human voice segment, and the snoring count is incremented by 1; when the fifth human voice segment is detected, if the starting frame is not determined to be the starting frame of the snoring segment, then the second processing method described above can be used to determine whether the intermediate frame is the starting frame of the snoring segment based on the snoring count. The reason for adopting this optional scheme is that when no snoring signal has been detected, the probability that the intermediate frame of the human voice segment is the starting frame of the snoring segment is small. In order to reduce the consumption of computing resources, the determination of the starting frame of the snoring segment can be performed on the intermediate frame after the snoring signal is detected.

[0236] For ease of explanation, in this embodiment, the current frame involved in the detection of the start frame of the snoring segment is referred to as the first current frame, and the current frame involved in the detection of the end frame of the snoring segment is referred to as the second current frame.

[0237] Step 403: If the starting frame of the snoring segment is determined, the ending frame of the snoring segment is determined starting from the next frame after the starting frame of the snoring segment.

[0238] Specifically, the processing in step 403 can be as follows:

[0239] Starting from the next frame after the beginning of the snoring segment, the second current frame is processed as follows:

[0240] Determine whether the second current frame is the end frame of the target human voice segment;

[0241] If the second current frame is the end frame of the target human voice segment, then the second current frame is taken as the end frame of the snoring segment;

[0242] If the second current frame is not the end frame of the target human voice segment, then determine whether the second current frame is the end frame of the snoring segment. If it is, then the second current frame is used as the end frame of the snoring segment; if not, then the above detection process is performed on the next frame of the second current frame.

[0243] In one possible implementation, after determining the start frame of the snoring segment, it can be determined whether the next frame is the end frame. The specific processing is the same as step 304 above, and will not be repeated here. If an end frame is determined, it is considered that the snoring has also ended, and the second current frame (i.e., the end frame) can be used as the end frame of the snoring segment.

[0244] If the second current frame is not determined as the end frame, then determine whether the second current frame is the end frame of the snoring segment.

[0245] Optionally, it can be determined whether the second current frame is the end frame of the snoring segment based on the Mel cepstral coefficient reference value and the Mel cepstral coefficient of the second current frame.

[0246] The Mel cepstral coefficient reference value refers to the Mel cepstral coefficient reference value of snoring.

[0247] In one possible implementation, the Mel-Cepstral Coefficients of the second current frame can be calculated, and the cross-correlation coefficient between the Mel-Cepstral Coefficients and a reference value can be calculated. It can then be determined whether the cross-correlation coefficient is less than a corresponding threshold. If the cross-correlation coefficient is less than the corresponding threshold, the second current frame can be considered not to belong to snoring, and thus the second current frame is designated as the end frame of the snoring segment.

[0248] In another possible implementation, the Mel-frequency cepstral coefficients (MFCCs) of each frame between the start frame of the snoring segment and the second current frame can be calculated, and the average value can be taken as the average MFCC from the start frame of the snoring segment to the second current frame. The cross-correlation coefficient between this average MFCC and the reference value of the MFCC is calculated, and it is determined whether the cross-correlation coefficient is less than a corresponding threshold. If the cross-correlation coefficient is less than the corresponding threshold, the second current frame can be considered not to belong to snoring, and then this second current frame is taken as the end frame of the snoring segment. Compared with the previous implementation, the end frame of the snoring segment can be determined after a certain delay of a certain number of frames. Based on this, it can be ensured that the snoring signal contains a larger amount of snoring information.

[0249] If the cross-correlation coefficient is not less than the corresponding threshold, then the processing of step 403 above will continue for the next frame.

[0250] Alternatively, if the aforementioned cross-correlation coefficient is not less than the corresponding threshold, then it is further determined whether the number of frames from the start frame of the snoring segment to the second current frame is greater than the snoring frame count threshold. If the number of frames is greater than the snoring frame count threshold, then the second current frame can be used as the end frame of the snoring segment; if the number of frames is not greater than the snoring frame count threshold, then the processing of step 403 above continues for the next frame.

[0251] Optionally, the aforementioned Mel-Cepstral coefficient reference values ​​can be updated adaptively. This embodiment provides two update methods as follows:

[0252] Method 1: When it is determined that the starting frame of the target vocal segment is the starting frame of the snoring segment, the above-mentioned Mel-Cepstral Coefficient baseline value is updated based on the Mel-Cepstral Coefficients of each vocal segment in at least two vocal segments preceding the target vocal segment and the Mel-Cepstral Coefficients of the starting frame of the target vocal segment.

[0253] In one possible implementation, when the starting frame of the target vocal segment is determined to be the starting frame of the snoring segment, the Mel-spectral coefficients (MSCs) of each frame in the at least two vocal segments can be calculated, and the MSC of the starting frame (i.e., the starting frame of the snoring segment) can also be calculated. The average MSC of the at least two vocal segments and the starting frame is then obtained. This average MSC is used as the reference value for the MSCs used in the target vocal segment.

[0254] Method 2: If, after determining that the starting frame of the target human voice segment is not the starting frame of the snoring segment, the intermediate frame of the target human voice segment is determined to be the starting frame of the snoring segment, then the Mel-Cepstral Coefficient Baseline Value is updated based on the mean Mel-Cepstral Coefficient between the starting frame of the target human voice segment and the starting frame of the snoring segment.

[0255] In one possible implementation, when it is determined that the intermediate frame of the target vocal segment is the starting frame of the snoring segment, the Mel-frequency cepstral coefficients of each frame from the starting frame of the target vocal segment to the starting frame of the snoring segment can be calculated, and the average value of the Mel-frequency cepstral coefficients can be obtained. Furthermore, the reference value of the Mel-frequency cepstral coefficients can be replaced with this average value.

[0256] For the detection of the snoring segment of the next target human voice segment, if the starting frame of the snoring segment is determined, the reference value of the Mel-Cepstral Coefficient used by the target human voice segment can also be determined based on the above processing, thereby realizing the adaptive update of the reference value of the Mel-Cepstral Coefficient.

[0257] Since the snoring signals of different target human voice segments may have different Mel-Cepstral Coefficients, through the above-mentioned update process, the reference value of the Mel-Cepstral Coefficient used for each target human voice segment can be adapted to the snoring of that target human voice segment, thereby improving the accuracy of determining the end frame of the snoring segment and thus improving the accuracy of snoring segment detection.

[0258] Step 404: The sound signal between the start frame and the end frame of the snoring segment is taken as the snoring signal.

[0259] In one possible implementation, the start frame and end frame of the snoring segment can be recorded. When analysis of the snoring signal is required, the corresponding snoring signal can be extracted based on the start and end frames of the snoring segment. This embodiment does not limit the subsequent processing of the snoring signal.

[0260] This embodiment can achieve the following beneficial effects:

[0261] (1) After real-time acquisition of sound signals, human voice segments can be detected in real time to determine the human voice segments in the sound signal. After determining at least two human voice segments, snoring segment detection is performed based on the snoring characteristics of the at least two human voice segments and the starting frame of the target human voice segment, starting from the beginning frame of the target human voice segment. Since the computational cost of the above processing is relatively small compared to machine learning algorithms such as Gaussian mixture models, hidden Markov models, clustering models, and support vector machines, real-time snoring segment detection can be implemented on embedded systems.

[0262] (2) In the process of calculating the snoring characteristic score of the starting frame, a variety of snoring characteristics such as snoring energy characteristics, snoring fundamental frequency characteristics, and snoring zero-crossing rate characteristics are used, which improves the detection accuracy and noise resistance.

[0263] (3) Adaptively adjust the ratio threshold of low frequency and mid frequency so that the ratio threshold is adapted to the current snoring and improves the accuracy of determining the starting frame of the snoring segment.

[0264] (4) Adaptively adjust the reference value of the Mel cepstral coefficient so that the reference value of the Mel cepstral coefficient used for each target human voice segment can be matched with the snoring of the target human voice segment, thereby improving the accuracy of determining the end frame of the snoring segment.

[0265] The following will refer to Figure 5 The flowchart shown illustrates a method for determining snoring signals, illustrating the method provided in this embodiment. In this embodiment, both the similarity of preset characteristics and snoring characteristics are used to determine whether the starting frame of the target human voice segment is the starting frame of the snoring segment.

[0266] like Figure 3 As shown, the method for determining snoring signals provided in this embodiment includes the following steps 501-504.

[0267] Step 501: Acquire the sound signal, detect human voice segments in the sound signal, and detect at least two human voice segments before the target human voice segment.

[0268] In one possible implementation, the device may be equipped with an audio acquisition device that can acquire sound signals in real time. The acquired sound signals are then processed through signal framing, pre-emphasis, impulse noise removal, and mean removal, and then human voice segments are detected in each frame of the sound signal to identify the human voice segments within the sound signal.

[0269] The specific method for determining the human voice segment can be found in the above embodiments. Figure 6 The flowchart of the method for determining the human voice segment shown is similar to the above embodiment, and will not be repeated in this embodiment.

[0270] After determining the end frame of the human voice segment, the processing steps 301-304 above can be performed on the next frame after the end frame to determine each human voice segment in the sound signal, and snoring segment detection can be performed in the human voice segment based on the following steps 502-504.

[0271] The method for determining snoring signals provided in this embodiment can detect at least two human voice segments before detecting snoring segments in the target human voice segment. Therefore, if no human voice segments are detected in the sound signal, the end frame can be detected after the start frame is detected; if at least two human voice segments are detected in the sound signal, the following step 502 is entered after the start frame is detected to determine whether the start frame is the start frame of the snoring segment.

[0272] Step 502: If the starting frame of the target human voice segment is detected in the sound signal, then based on the similarity of the preset characteristics of the starting frame of the target human voice segment and the snoring characteristics of at least two human voice segments and the starting frame of the target human voice segment, determine whether the starting frame of the target human voice segment is the starting frame of the snoring segment.

[0273] In one possible implementation, when two voice segments are detected in the sound signal, snoring segment detection can begin from the third voice segment. Optionally, to ensure the accuracy of snoring segment detection, in at least two voice segments preceding the target voice segment, the length of each voice segment meets a preset length condition (e.g., within the snoring length range), and the interval between two adjacent voice segments is greater than a preset interval duration.

[0274] In this embodiment, the human voice segment for which snoring is detected is referred to as the target human voice segment, and the same applies below.

[0275] Compared to spoken voices, snoring has a certain degree of repetition. Therefore, whenever the starting frame of a target voice segment is detected, it can be determined whether at least two preceding voice segments are similar. If at least two preceding voice segments are similar, the probability that the collected sound is snoring is high. Furthermore, it can be determined whether the starting frame of the target voice segment is similar to at least two preceding voice segments. If the starting frame of the target voice segment is similar to at least two preceding voice segments, it indicates that the probability that the starting frame is snoring is also high.

[0276] Snoring has certain characteristics. Therefore, whenever the starting frame of a target person's voice segment is detected, the snoring characteristics of at least two preceding voice segments and the starting frame can be calculated. If snoring characteristics are present, the probability that the collected sound is snoring is relatively high.

[0277] Therefore, the similarity of preset characteristics and snoring characteristics can be used to determine whether the starting frame of the target human voice segment is the starting frame of the snoring segment. If it has similarity and snoring characteristics, then the starting frame can be used as the starting frame of the snoring segment.

[0278] Optionally, the processing in step 502 above can be as follows:

[0279] Calculate the similarity score of the starting frame based on the similarity of the preset characteristics of the starting frames of at least two human voice segments and the target human voice segment.

[0280] Based on the snoring characteristics of the starting frames of at least two human voice segments and the target human voice segment, calculate the snoring characteristic score of the starting frame.

[0281] Based on similarity scores and snoring characteristic scores, it is determined whether the starting frame of the target human voice segment is the starting frame of the snoring segment.

[0282] The similarity of the aforementioned preset characteristics includes any one or more of the following: energy similarity, spectral similarity, Mel-frequency cepstral coefficient similarity, and noise similarity. The aforementioned snoring characteristics include any one or more of the following: snoring energy characteristics, snoring fundamental frequency characteristics, and snoring zero-crossing rate characteristics.

[0283] In one possible implementation, if at least two human voice segments have the aforementioned similarity to the starting frame of the target human voice segment, a corresponding score can be added to the similarity score of the starting frame; if at least two human voice segments have the aforementioned snoring characteristics to the starting frame of the target human voice segment, a corresponding score can be added to the snoring characteristic score of the starting frame.

[0284] Furthermore, it can be determined whether the similarity score is greater than the first score threshold and whether the snoring characteristic score is greater than the second score threshold. If both are greater than the threshold, it indicates that the similarity and snoring characteristics are present, and the starting frame can be used as the starting frame of the snoring segment.

[0285] Alternatively, a weighted average of the similarity score and the snoring characteristic score can be taken to obtain the snoring score of the starting frame, and it can be determined whether the snoring score is greater than the third score threshold. If it is greater, it indicates that the frame possesses both similarity and snoring characteristics, and the starting frame can then be used as the starting frame of the snoring segment.

[0286] The similarity and snoring characteristics will be introduced separately below.

[0287] Regarding similarity:

[0288] It can be determined whether the starting frame of the target vocal segment has similar energy to at least two previous vocal segments. If there is energy similarity, the similarity score of the starting frame can be increased.

[0289] It can be determined whether the starting frame of the target vocal segment has similarity to the spectrum of at least two previous vocal segments. If there is spectral similarity, the similarity score of the starting frame can be increased.

[0290] It can be determined whether the starting frame of the target vocal segment has similarity to the Mel-Cepstral coefficients of at least two previous vocal segments. If there is similarity in Mel-Cepstral coefficients, the similarity score of the starting frame can be increased.

[0291] By using the noise characteristics of preset noise, it can be determined whether the starting frame of the target human voice segment matches the noise characteristics of at least two previous human voice segments. If they match, it indicates that the starting frame or human voice segment may belong to noise, and the similarity score of the starting frame can be reduced.

[0292] The specific calculation methods for each of the above similarities will be introduced below.

[0293] For energy similarity, the similarity score of the starting frame can be calculated as follows: Calculate the average energy of each voice segment in at least two voice segments, and calculate the ratio of the difference between the average energy of each voice segment. If the ratio of the difference is within a preset range, add the corresponding first similarity score to the similarity score of the starting frame. As an example, process the two voice segments preceding the target voice segment. Sort the frame energy of each frame in each voice segment in descending order, take the first certain number of frames (e.g., the first 10 frames), calculate the average frame energy and use it as the average energy of that voice segment; calculate the difference between the average energy of the two voice segments, and calculate the ratio of this difference (e.g., calculate the ratio between the difference in average energy and the average energy of any voice segment); if this ratio of the difference is within a preset range (e.g., 0 to 0.1), it indicates that the energy of the two voice segments is similar, and 1 point can be added to the similarity score of the starting frame.

[0294] For spectral similarity, the similarity score of the starting frame can be calculated as follows: Determine the number of first peaks in the spectra of at least two voice segments that meet the condition of overlapping peak positions. If the number of first peaks is greater than a first threshold, then add a corresponding second similarity score to the similarity score of the starting frame. As an example, process two voice segments preceding the target voice segment, calculate the average spectrum of each voice segment, and search for a certain number (e.g., 5) of peaks in the average spectrum, and record the corresponding positions of the peaks. Compare the peak positions of the two voice segments one by one, and record the number of peaks with similar positions (i.e., the above-mentioned condition of overlapping peak positions). If the number of similar peak positions exceeds the first threshold, it indicates that the spectra of the two voice segments are similar, and then 1 point can be added to the similarity score of the starting frame.

[0295] For Mel-Cepstral Coefficient Similarity, the similarity score of the starting frame can be calculated as follows: Calculate the Mel-Cepstral Coefficients of the starting frames of at least two human voice segments and the target human voice segment, and calculate the first cross-correlation coefficient between any two Mel-Cepstral Coefficients in the starting frames of at least two human voice segments and the target human voice segment respectively. If the first cross-correlation coefficient is greater than the first preset threshold, then add the corresponding third similarity score to the similarity score of the starting frame. As an example, the starting frame of the target vocal segment and the two preceding vocal segments are processed. The Mel-Cepstral Coefficients (MCCs) of the starting frame and each vocal segment are calculated separately. For a vocal segment, the MMCs of each frame can be calculated, the average MMC is obtained, and the average MMC is used as the MMC of the vocal segment. The cross-correlation coefficient between the MMCs of the two vocal segments is calculated. If the cross-correlation coefficient is greater than a first preset threshold, it indicates that the two vocal segments have MMC similarity, and 1 point can be added to the similarity score of the starting frame. The cross-correlation coefficients between the starting frame and the first vocal segment, and the cross-correlation coefficients between the starting frame and the second vocal segment are also calculated. If both cross-correlation coefficients are greater than the first preset threshold, it indicates that the starting frame and both vocal segments have MMC similarity, and 1 point can be added to the similarity score of the starting frame.

[0296] For noise similarity, the similarity score of the starting frame can be calculated as follows: calculate the second cross-correlation coefficient between any Mel-Cepstral Coefficient of the starting frame of at least two human voice segments and the target human voice segment and the Mel-Cepstral Coefficient of the preset noise. If any second cross-correlation coefficient is greater than a second preset threshold, subtract the corresponding fourth similarity score from the similarity score of the starting frame; if all second cross-correlation coefficients are less than a third preset threshold, add the corresponding fifth similarity score to the similarity score of the starting frame. As an example, the starting frame of the target vocal segment and the two preceding vocal segments are processed. The Mel-frequency cepstral coefficients (MFCs) of the starting frame and each vocal segment, as well as the MFC of the preset noise, are calculated. The cross-correlation coefficients between the preset noise and the MFC of the first vocal segment, and between the preset noise and the MFC of the second vocal segment, are calculated. If any cross-correlation coefficient is greater than a second preset threshold, it indicates that any vocal segment is similar to the preset noise, and the similarity score of the starting frame is reduced by 1 point. If both cross-correlation coefficients are less than a third preset threshold, it indicates that neither vocal segment is similar to the preset noise, and the similarity score of the starting frame is increased by 1 point. The cross-correlation coefficients between the preset noise and the MFC of the starting frame are calculated. If the cross-correlation coefficient is greater than a second preset threshold, it indicates that the starting frame is similar to the preset noise, and the similarity score of the starting frame is reduced by 1 point. If the cross-correlation coefficient is less than a third preset threshold, it indicates that the starting frame is not similar to the preset noise, and the similarity score of the starting frame is increased by 1 point.

[0297] Optionally, for spectral similarity, the process of calculating the similarity score of the starting frame can further include: if the number of first peaks is greater than a preset threshold, then in the spectrum of the starting frame, determine the number of second peaks that satisfy the condition of peak position overlap with at least two human voice segments; if the number of second peaks is greater than a second threshold, then add a corresponding sixth similarity score to the similarity score of the starting frame. As an example, in the above process of calculating spectral similarity, if the number of peak positions that are close in two human voice segments exceeds the first threshold, then the maximum number of peaks can be searched in the spectrum of the starting frame, and the corresponding positions of the peaks can be recorded; the peak positions of the starting frame and the two human voice segments are compared one by one, and the number of peak positions that are close is recorded; if the number of peak positions that are close exceeds the second threshold, it indicates that the spectrum of the starting frame and the two human voice segments are similar, and then 1 point can be added to the similarity score of the starting frame.

[0298] The principle behind snoring characteristics is as follows:

[0299] Snoring is characterized by higher low-frequency energy than mid-frequency energy. Therefore, whether a sound is snoring can be determined by whether the low-frequency energy is higher than the mid-frequency energy. Low frequency refers to the frequency range of 50Hz to 400Hz, and mid frequency refers to the frequency range of 400Hz to 4000Hz. The settings of low and mid frequencies are adapted to the characteristics of snoring.

[0300] Since the fundamental frequency of snoring falls within a certain range, the fundamental frequency range of snoring can be determined by statistically analyzing the pre-collected snoring signals. Therefore, the fundamental frequency range of snoring can be used to determine whether a snoring sound possesses fundamental frequency characteristics.

[0301] The zero-crossing rate of snoring also falls within a certain range. Frame zero-crossing rate statistics can be performed on pre-collected snoring signals to determine the range of snoring zero-crossing rates. Therefore, the range of snoring zero-crossing rates can be used to determine whether a snoring device possesses the zero-crossing rate characteristic.

[0302] The specific calculation methods for each of the above-mentioned snoring characteristics will be introduced below.

[0303] To calculate the snoring energy characteristics score of the starting frame, the following steps can be taken:

[0304] Calculate the average low-frequency energy and average mid-frequency energy of each voice segment in at least two voice segments, and calculate the first ratio of the average low-frequency energy to the average mid-frequency energy of each voice segment. If the first ratio is greater than the ratio threshold, add the corresponding first snoring characteristic score to the snoring characteristic score of the starting frame.

[0305] Calculate the average low-frequency energy and average mid-frequency energy of the starting frame, and calculate the second ratio of the average low-frequency energy to the average mid-frequency energy of the starting frame. If the second ratio is greater than the ratio threshold, add the corresponding second snoring characteristic score to the snoring characteristic score of the starting frame.

[0306] As an example, two voice segments preceding the target voice segment are processed. For each voice segment, each frame of sound signal can be processed by Fast Fourier Transform (FFT) to obtain the Fourier transform result, and the average frequency band energy is calculated using the above formula (2).

[0307] Furthermore, the ratio of the average low-frequency energy to the average mid-frequency energy of each voice segment can be calculated, which is referred to as the first ratio in this embodiment. If the first ratio of both voice segments is greater than the ratio threshold, it indicates that both voice segments possess snoring energy characteristics, and 1 point can be added to the snoring characteristic score of the starting frame.

[0308] Similarly, the ratio of the average low-frequency energy to the average mid-frequency energy of the starting frame can be calculated; in this embodiment, this is referred to as the second ratio. If the second ratio of the starting frame is greater than the ratio threshold, it indicates that the starting frame possesses snoring energy characteristics, and 1 point can be added to the snoring characteristic score of the starting frame.

[0309] To calculate the snoring characteristic score of the starting frame, the fundamental frequency characteristics of snoring can be determined as follows:

[0310] Calculate the fundamental frequency of each voice segment in at least two voice segments, and calculate the fundamental frequency difference between each voice segment. If the fundamental frequency of each voice segment is within the range of snoring fundamental frequency, and the fundamental frequency difference is within the range of fundamental frequency difference, then add the corresponding third snoring characteristic score to the snoring characteristic score of the starting frame.

[0311] Calculate the fundamental frequency of the starting frame. If the fundamental frequency of the starting frame is within the range of snoring fundamental frequency, add the corresponding fourth snoring characteristic score to the snoring characteristic score of the starting frame.

[0312] As an example, two voice segments preceding the target voice segment are processed. The average spectrum of each voice segment is calculated, and the fundamental frequency is determined from the average spectrum as the fundamental frequency of that voice segment. Then, the fundamental frequency difference between the two voice segments is calculated. If the fundamental frequency of each voice segment is within the range of snoring fundamental frequency, it indicates that both voice segments possess snoring fundamental frequency characteristics. If the fundamental frequency difference is within the range of fundamental frequency difference, it indicates that the fundamental frequencies of the two voice segments have a certain continuity, which conforms to the characteristics of snoring. In this case, 1 point can be added to the snoring characteristic score of the starting frame.

[0313] If the fundamental frequency of the starting frame is within the range of snoring fundamental frequency, it indicates that the starting frame has snoring fundamental frequency characteristics, and 1 point can be added to the snoring characteristic score of the starting frame.

[0314] For the snoring zero-crossing rate characteristic, the snoring characteristic score of the starting frame can be calculated as follows:

[0315] Calculate the average zero-crossing rate of each voice segment in at least two voice segments. If the average zero-crossing rate of each voice segment is within the range of snoring zero-crossing rate, add the corresponding fifth snoring characteristic score to the snoring characteristic score of the starting frame.

[0316] Calculate the zero-crossing rate of the starting frame. If the zero-crossing rate of the starting frame is within the snoring zero-crossing rate range, then add the corresponding sixth snoring characteristic score to the snoring characteristic score of the starting frame.

[0317] As an example, we can process two voice segments preceding the target voice segment. For each voice segment, we can calculate the zero-crossing rate of each frame and take the average of the values. If the average zero-crossing rate of each voice segment is within the range of snoring zero-crossing rates, it indicates that both voice segments possess snoring zero-crossing rate characteristics, and we can add 1 point to the snoring characteristic score of the starting frame. Alternatively, the process of calculating the average zero-crossing rate can also be as follows: for each voice segment, we can calculate the zero-crossing rate of each frame, sort the zero-crossing rates of each frame in descending order, and take the average of the zero-crossing rates of the preceding set number of frames (e.g., 10 frames) as the average zero-crossing rate of that voice segment.

[0318] Similarly, the zero-crossing rate is calculated for the starting frame. If the zero-crossing rate of the starting frame is within the range of snoring zero-crossing rates, it indicates that the starting frame has snoring zero-crossing rate characteristics, and 1 point can be added to the snoring characteristic score of the starting frame.

[0319] It should be noted that the above scores can be adjusted. For example, if the accuracy of the snoring energy characteristics is higher, the score corresponding to the snoring energy characteristics can be increased. This embodiment does not limit this.

[0320] Optionally, the threshold ratio of the low-frequency energy mean to the mid-frequency energy mean used in the above calculation of snoring energy characteristics can be adaptively updated. This embodiment provides two update methods as follows:

[0321] Method 1: When it is determined that the starting frame of the target human voice segment is the starting frame of the snoring segment, the ratio threshold in the spectrum condition can be updated based on the first ratio and the second ratio determined in the above process.

[0322] In one possible implementation, when it is determined that the starting frame of the target human voice segment is the starting frame of the snoring segment, a weighted average can be performed based on the first ratio of at least two human voice segments preceding the target human voice segment, the second ratio of the target human voice segment, and the ratio threshold to be updated. The resulting weighted average is used as the updated ratio threshold, which is then used as the ratio threshold for the next target human voice segment.

[0323] Method 2: If, after determining that the starting frame of the target human voice segment is not the starting frame of the snoring segment, the intermediate frame of the target human voice segment is determined to be the starting frame of the snoring segment, then the ratio threshold is updated based on the ratio of the average low-frequency energy and the average mid-frequency energy of the average spectrum between the starting frame of the target human voice segment and the starting frame of the snoring segment.

[0324] In one possible implementation, when it is determined that the middle frame of the target human voice segment is the starting frame of the snoring segment, the above-mentioned mean ratio can be weighted and averaged with the ratio threshold to be updated, and the resulting weighted average can be used as the updated ratio threshold, which is then used as the ratio threshold for the next target human voice segment.

[0325] The ratio threshold that has never been updated can be set as the initial value. The ratio threshold can also be set with upper and lower limits based on a large number of experiments to keep the ratio threshold within a reasonable range.

[0326] Since the energy ratio of low and mid frequencies in each user's snoring is different, and the snoring of the same user may also be different at different times, the above processing can adaptively adjust the low and mid frequency ratio threshold when the starting frame of the target voice segment is determined to be the starting frame of the snoring segment. This makes the ratio threshold match the current snoring, improves the accuracy of determining the starting frame of the snoring segment, and thus improves the accuracy of snoring segment detection.

[0327] If the starting frame of the target human voice segment is not the starting frame of the snoring segment, then two processing methods can be used thereafter:

[0328] The first processing method is to detect the end frame of the target human voice segment frame by frame and no longer detect the snoring segment;

[0329] The second processing method is to determine whether the intermediate frame is the starting frame of the snoring segment during the process of detecting the ending frame of the target human voice segment frame by frame after determining the starting frame of the target human voice segment.

[0330] In both of the above processing methods, the specific processing of detecting the end frame of the target human voice segment frame by frame is the same as step 304 above, and will not be repeated here.

[0331] For the first processing method, if the starting frame of the target human voice segment is not the starting frame of the snoring segment, it is considered that the target human voice segment does not contain snoring. Therefore, snoring segment detection will not continue in this target human voice segment. After the next human voice segment is detected, snoring segment detection will be performed on the human voice segment.

[0332] To improve the accuracy of snoring segment detection, the second processing method described above can also be used to determine whether the intermediate frame of the target voice segment is the starting frame of the snoring segment. The specific processing method can employ existing snoring segment detection methods; this embodiment does not limit the specific method for determining whether the intermediate frame of the target voice segment is the starting frame of the snoring segment. Optionally, to further improve the accuracy of snoring segment detection, this application will introduce a method for determining whether the intermediate frame of the target voice segment is the starting frame of the snoring segment in another embodiment; this embodiment will not elaborate on this method here.

[0333] Optionally, the condition for adopting the second processing method described above can be that at least one snoring signal is detected in the human voice segments detected before the target human voice segment. As an example, assuming that the starting frame is not determined to be the starting frame of the snoring segment in the detected third human voice segment, then snoring segment detection is not continued in the third human voice segment; if the starting frame is determined to be the starting frame of the snoring segment in the detected fourth human voice segment, then the corresponding snoring signal is determined in the fourth human voice segment, and the snoring count is incremented by 1; when the fifth human voice segment is detected, if the starting frame is not determined to be the starting frame of the snoring segment, then the second processing method described above can be used to determine whether the intermediate frame is the starting frame of the snoring segment based on the snoring count. The reason for adopting this optional scheme is that when no snoring signal has been detected, the probability that the intermediate frame of the human voice segment is the starting frame of the snoring segment is small. In order to reduce the consumption of computing resources, the determination of the starting frame of the snoring segment can be performed on the intermediate frame after the snoring signal is detected.

[0334] For ease of explanation, in this embodiment, the current frame involved in the detection of the start frame of the snoring segment is referred to as the first current frame, and the current frame involved in the detection of the end frame of the snoring segment is referred to as the second current frame.

[0335] Step 503: If the starting frame of the snoring segment is determined, the ending frame of the snoring segment is determined starting from the next frame after the starting frame of the snoring segment.

[0336] Specifically, the processing in step 503 can be as follows:

[0337] Starting from the next frame after the beginning of the snoring segment, the second current frame is processed as follows:

[0338] Determine whether the second current frame is the end frame of the target human voice segment;

[0339] If the second current frame is the end frame of the target human voice segment, then the second current frame is taken as the end frame of the snoring segment;

[0340] If the second current frame is not the end frame of the target human voice segment, then determine whether the second current frame is the end frame of the snoring segment. If it is, then the second current frame is used as the end frame of the snoring segment; if not, then the above detection process is performed on the next frame of the second current frame.

[0341] In one possible implementation, after determining the start frame of the snoring segment, it can be determined whether the next frame is the end frame. The specific processing is the same as step 304 above, and will not be repeated here. If an end frame is determined, it is considered that the snoring has also ended, and the second current frame (i.e., the end frame) can be used as the end frame of the snoring segment.

[0342] If the second current frame is not determined as the end frame, then determine whether the second current frame is the end frame of the snoring segment.

[0343] Optionally, it can be determined whether the second current frame is the end frame of the snoring segment based on the Mel cepstral coefficient reference value and the Mel cepstral coefficient of the second current frame.

[0344] The Mel cepstral coefficient reference value refers to the Mel cepstral coefficient reference value of snoring.

[0345] In one possible implementation, the Mel-Cepstral Coefficients of the second current frame can be calculated, and the cross-correlation coefficient between the Mel-Cepstral Coefficients and a reference value can be calculated. It can then be determined whether the cross-correlation coefficient is less than a corresponding threshold. If the cross-correlation coefficient is less than the corresponding threshold, the second current frame can be considered not to belong to snoring, and thus the second current frame is designated as the end frame of the snoring segment.

[0346] In another possible implementation, the Mel-frequency cepstral coefficients (MFCCs) of each frame between the start frame of the snoring segment and the second current frame can be calculated, and their average value can be taken as the average MFCC from the start frame to the second current frame. The cross-correlation coefficient between this average MFCC and the baseline MFCC is calculated, and it is determined whether the cross-correlation coefficient is less than a corresponding threshold. If the cross-correlation coefficient is less than the corresponding threshold, the second current frame can be considered not to belong to snoring, and this second current frame is then taken as the end frame of the snoring segment. Compared with the previous implementation, the end frame of the snoring segment can be determined after a certain delay of a certain number of frames. Based on this, it can be ensured that the snoring signal contains a larger amount of snoring information.

[0347] If the cross-correlation coefficient is not less than the corresponding threshold, then the processing of step 503 above continues for the next frame.

[0348] Alternatively, if the aforementioned cross-correlation coefficient is not less than the corresponding threshold, then it is further determined whether the number of frames from the start frame of the snoring segment to the second current frame is greater than the snoring frame count threshold. If the number of frames is greater than the snoring frame count threshold, then the second current frame can be used as the end frame of the snoring segment; if the number of frames is not greater than the snoring frame count threshold, then the processing of step 503 above continues for the next frame.

[0349] Optionally, the aforementioned Mel-Cepstral coefficient reference values ​​can be updated adaptively. This embodiment provides two update methods as follows:

[0350] Method 1: When it is determined that the starting frame of the target vocal segment is the starting frame of the snoring segment, the above-mentioned Mel-Cepstral Coefficient baseline value is updated based on the Mel-Cepstral Coefficients of each vocal segment in at least two vocal segments preceding the target vocal segment and the Mel-Cepstral Coefficients of the starting frame of the target vocal segment.

[0351] In one possible implementation, when the starting frame of the target vocal segment is determined to be the starting frame of the snoring segment, the Mel-spectral coefficients (MSCs) of each frame in the at least two vocal segments can be calculated, and the MSC of the starting frame (i.e., the starting frame of the snoring segment) can also be calculated. The average MSC of the at least two vocal segments and the starting frame is then obtained. This average MSC is used as the reference value for the MSCs used in the target vocal segment.

[0352] Method 2: If, after determining that the starting frame of the target human voice segment is not the starting frame of the snoring segment, the intermediate frame of the target human voice segment is determined to be the starting frame of the snoring segment, then the Mel-Cepstral Coefficient Baseline Value is updated based on the mean Mel-Cepstral Coefficient between the starting frame of the target human voice segment and the starting frame of the snoring segment.

[0353] In one possible implementation, when it is determined that the intermediate frame of the target vocal segment is the starting frame of the snoring segment, the Mel-frequency cepstral coefficients of each frame from the starting frame of the target vocal segment to the starting frame of the snoring segment can be calculated, and the average value of the Mel-frequency cepstral coefficients can be obtained. Furthermore, the reference value of the Mel-frequency cepstral coefficients can be replaced with this average value.

[0354] For the detection of the snoring segment of the next target human voice segment, if the starting frame of the snoring segment is determined, the reference value of the Mel-Cepstral Coefficient used by the target human voice segment can also be determined based on the above processing, thereby realizing the adaptive update of the reference value of the Mel-Cepstral Coefficient.

[0355] Since the snoring signals of different target human voice segments may have different Mel-Cepstral Coefficients, through the above-mentioned update process, the reference value of the Mel-Cepstral Coefficient used for each target human voice segment can be adapted to the snoring of that target human voice segment, thereby improving the accuracy of determining the end frame of the snoring segment and thus improving the accuracy of snoring segment detection.

[0356] Step 504: Use the sound signal between the start frame and end frame of the snoring segment as the snoring signal.

[0357] In one possible implementation, the start frame and end frame of the snoring segment can be recorded. When analysis of the snoring signal is required, the corresponding snoring signal can be extracted based on the start and end frames of the snoring segment. This embodiment does not limit the subsequent processing of the snoring signal.

[0358] This embodiment can achieve the following beneficial effects:

[0359] (1) After real-time acquisition of sound signals, human voice segments can be detected in real time to determine the human voice segments in the sound signal. After determining at least two human voice segments, starting from the starting frame of the target human voice segment, snoring segment detection is performed based on the similarity of the preset characteristics of the starting frame of the at least two human voice segments and the snoring characteristics. Since the computational cost of the above processing is relatively small compared to machine learning algorithms such as Gaussian mixture models, hidden Markov models, clustering models, and support vector machines, real-time snoring segment detection can be implemented on embedded systems.

[0360] (2) In the process of calculating the similarity score of the starting frame, the similarity of multiple preset characteristics such as energy similarity, spectrum similarity, Mel cepstral coefficient similarity, and noise similarity is adopted. In the process of calculating the snoring characteristic score of the starting frame, multiple snoring characteristics such as snoring energy characteristics, snoring fundamental frequency characteristics, and snoring zero-crossing rate characteristics are adopted, which improves the detection accuracy and improves the noise resistance.

[0361] (3) Adaptively adjust the ratio threshold of low frequency and mid frequency so that the ratio threshold is adapted to the current snoring and improves the accuracy of determining the starting frame of the snoring segment.

[0362] (4) Adaptively adjust the reference value of the Mel cepstral coefficient so that the reference value of the Mel cepstral coefficient used for each target human voice segment can be matched with the snoring of the target human voice segment, thereby improving the accuracy of determining the end frame of the snoring segment.

[0363] The following will refer to Figure 6 The flowchart shown illustrates a method for determining snoring signals, illustrating the method provided in this embodiment. This embodiment primarily relates to a method for determining whether the intermediate frame of a target human voice segment is the starting frame of a snoring segment.

[0364] like Figure 3 As shown, the method for determining snoring signals provided in this embodiment includes the following steps 601-604.

[0365] Step 601: Acquire the sound signal, perform human voice segment detection on the sound signal, and detect at least two human voice segments before the target human voice segment.

[0366] In one possible implementation, the device may be equipped with an audio acquisition device that can acquire sound signals in real time. The acquired sound signals are then processed through signal framing, pre-emphasis, impulse noise removal, and mean removal, and then human voice segments are detected in each frame of the sound signal to identify the human voice segments within the sound signal.

[0367] The specific method for determining the human voice segment can be found in the above embodiments. Figure 7The flowchart of the method for determining the human voice segment shown is similar to the above embodiment, and will not be repeated in this embodiment.

[0368] After determining the end frame of the human voice segment, the processing steps 301-304 above can be performed on the next frame after the end frame to determine each human voice segment in the sound signal, and snoring segment detection can be performed in the human voice segment based on the following steps 602-604.

[0369] The method for determining snoring signals provided in this embodiment can detect at least two human voice segments before detecting snoring segments in the target human voice segment. Therefore, if no human voice segments are detected in the sound signal, the end frame can be detected after the start frame is detected; if at least two human voice segments are detected in the sound signal, the following step 602 is entered after the start frame is detected to determine whether the start frame is the start frame of the snoring segment.

[0370] Step 602: If the starting frame of the target human voice segment is detected in the sound signal, and the starting frame is not the starting frame of the snoring segment, then determine whether the intermediate frame of the target human voice segment is the starting frame of the snoring segment.

[0371] In one possible implementation, when two voice segments are detected in the sound signal, snoring segment detection can begin from the third voice segment. Optionally, to ensure the accuracy of snoring segment detection, in at least two voice segments preceding the target voice segment, the length of each voice segment meets a preset length condition (e.g., within the snoring length range), and the interval between two adjacent voice segments is greater than a preset interval duration.

[0372] In this embodiment, the human voice segment for which snoring is detected is referred to as the target human voice segment, and the same applies below.

[0373] For ease of explanation, in this embodiment, the current frame involved in the detection of the start frame of the snoring segment is referred to as the first current frame, and the current frame involved in the detection of the end frame of the snoring segment is referred to as the second current frame.

[0374] The first current frame includes two cases: the first case is that the first current frame is the starting frame of the target human voice segment; the second case is that the first current frame is any intermediate frame after the starting frame of the target human voice segment.

[0375] For the first scenario described above, the specific processing for determining whether the starting frame is the starting frame of the snoring segment can employ existing snoring segment detection methods, and this embodiment does not limit the specific method. Alternatively, to further improve the accuracy of snoring segment detection, the method for determining whether the starting frame of the target human voice segment is the starting frame of the snoring segment, as described in any of the above embodiments, can be used, and this embodiment will not elaborate on it here.

[0376] This embodiment improves the method for determining whether the intermediate frame of the target human voice segment is the starting frame of the snoring segment in the second case described above. Specifically:

[0377] If the first current frame in the target human voice segment is determined to satisfy any one or more of the following snoring initiation conditions, then the first current frame is determined to be the snoring segment initiation frame. The above snoring initiation conditions include: snoring duration condition, Mel-frequency cepstral coefficient condition, no-noise condition, zero-crossing rate condition, spectrum condition, and fundamental frequency condition.

[0378] Optionally, the condition for adopting the second scenario described above could be that at least one snoring signal is detected in the human voice segments detected before the target human voice segment. As an example, assuming that the starting frame is not determined to be the starting frame of the snoring segment in the detected third human voice segment, then snoring segment detection is not continued in the third human voice segment; if the starting frame is determined to be the starting frame of the snoring segment in the detected fourth human voice segment, then the corresponding snoring signal is determined in the fourth human voice segment, and the snoring count is incremented by 1; when the fifth human voice segment is detected, if the starting frame is not determined to be the starting frame of the snoring segment, then the snoring count can be used to enter the second processing method described above to determine whether the intermediate frame is the starting frame of the snoring segment. The reason for adopting this optional scheme is that when no snoring signal has been detected, the probability that the intermediate frame of the human voice segment is the starting frame of the snoring segment is small. To reduce the consumption of computational resources, the determination of the starting frame of the snoring segment can be performed on the intermediate frame after the snoring signal is detected.

[0379] Reference Figure 8 The flowchart of the method for determining the starting frame of the snoring segment is shown. The processing of step 602 may include the following steps 701-703.

[0380] Step 701: Determine whether the first current frame is the end frame;

[0381] Step 702: If the first current frame is not the end frame, determine whether the first current frame satisfies any one or more of the above snoring initiation conditions.

[0382] Step 703: If the condition is met, the first current frame is taken as the starting frame of the snoring segment; if the condition is not met, the next frame is processed according to steps 701-703.

[0383] In one possible implementation, if the starting frame of the target human voice segment is not the starting frame of the snoring segment, then after a certain number of frames, it is determined whether the first current frame is the ending frame of the target human voice segment. The specific processing is the same as step 304 above, and will not be repeated here.

[0384] If the first current frame is detected as the end frame, the detection of the target human voice segment ends, and the human voice segment is detected for the next frame of sound signal to determine whether it is the start frame.

[0385] If the first current frame is not detected as the end frame, the snoring segment detection continues on the first current frame (which is now the middle frame of the target human voice segment), that is, it is determined whether the first current frame meets the above-mentioned snoring start condition. If it does, it indicates that the probability of the first current frame belonging to snoring is relatively high, and the first current frame is taken as the snoring segment start frame; if it does not meet, the above process is repeated for the next frame to determine whether it is the snoring segment start frame. Until the snoring segment start frame or the end frame is determined, the above process of determining the snoring segment start frame can be ended.

[0386] The following sections will describe the conditions for the onset of snoring as described above.

[0387] Determining that the first current frame meets the snoring duration condition means: determining that the number of frames between the starting frame of the target voice segment and the first current frame is within the snoring duration range, and / or determining that the duration between the starting frame of the target voice segment and the ending frame of the previous voice segment is greater than the snoring interval duration.

[0388] Determining that the first current frame satisfies the Mel-Cepstral Coefficient (MCC) condition means: calculating the mean of the first MMC between the starting frame and the first current frame of the target vocal segment; calculating the first cross-correlation coefficient between the mean of the first MMC and the reference value of the MMC; and determining that the first cross-correlation coefficient is greater than a first preset threshold. Specifically, the MMC can be calculated for each frame between the starting frame and the first current frame to obtain the mean of the first MMC, and then the first cross-correlation coefficient between the mean of the first MMC and the reference value of the MMC can be calculated. The reference value of the MMC refers to the reference value of the MMC for snoring. If the first cross-correlation coefficient is greater than the first preset threshold, it indicates that the probability of the first current frame belonging to snoring is relatively high.

[0389] Determining that the first current frame meets the non-noise condition means: calculating the second cross-correlation coefficient between the mean of the first Mel-Cepstral Coefficients and the Mel-Cepstral Coefficients of the preset noise, and determining that the second cross-correlation coefficient is less than a second preset threshold. If the second cross-correlation coefficient is less than the second preset threshold, it indicates that the probability of the first current frame belonging to noise is relatively low.

[0390] Determining that the first current frame meets the zero-crossing rate condition means calculating the average zero-crossing rate between the starting frame and the first current frame of the target human voice segment, and determining that the average zero-crossing rate is within a preset zero-crossing rate range. The preset zero-crossing rate range can refer to the range of zero-crossing rates obtained from statistical analysis of snoring sounds. If the average zero-crossing rate of each frame from the starting frame to the first current frame is within the preset zero-crossing rate range, it indicates that the first current frame has a high probability of being a snoring sound.

[0391] Determining that the first current frame meets the spectral condition involves: calculating the spectral mean between the starting frame of the target vocal segment and the first current frame; calculating the average low-frequency energy and average mid-frequency energy on the spectral mean; and calculating the ratio of the average low-frequency energy to the average mid-frequency energy, ensuring that this ratio is greater than a threshold. Snoring has the characteristic of higher low-frequency energy than mid-frequency energy; therefore, if the ratio of the average low-frequency energy to the average mid-frequency energy is greater than the threshold, it indicates that the first current frame is likely to be snoring.

[0392] Low frequency can refer to the frequency band from 50Hz to 400Hz, and mid frequency can refer to the frequency band from 400Hz to 4000Hz. The low and mid frequency settings are adapted to the snoring sound.

[0393] Determining that the first current frame meets the fundamental frequency condition means: determining the fundamental frequency value based on the average spectral value between the starting frame of the target human voice segment and the first current frame, and ensuring that the fundamental frequency value of the average spectral value falls within the snoring fundamental frequency range. The snoring fundamental frequency range can be obtained based on statistical analysis of snoring fundamental frequencies. If the fundamental frequency value of the average spectral value falls within the snoring fundamental frequency range, it indicates a high probability that the first current frame belongs to snoring.

[0394] Optionally, the aforementioned Mel-Cepstral coefficient reference values ​​can be updated adaptively. This embodiment provides two update methods as follows:

[0395] Method 1: When it is determined that the starting frame of the target vocal segment is the starting frame of the snoring segment, the above-mentioned Mel-Cepstral Coefficient baseline value is updated based on the Mel-Cepstral Coefficients of each vocal segment in at least two vocal segments preceding the target vocal segment and the Mel-Cepstral Coefficients of the starting frame of the target vocal segment.

[0396] In one possible implementation, when the starting frame of the target vocal segment is determined to be the starting frame of the snoring segment, the Mel-spectral coefficients (MSCs) of each frame in the aforementioned at least two vocal segments can be calculated, and the MSC of the starting frame (which is also the starting frame of the snoring segment) can be calculated. The average value of the MSCs of the aforementioned at least two vocal segments and the starting frame is then obtained. This average value of the MSCs is then used as the reference value of the MSCs used for the target vocal segment.

[0397] Method 2: When it is determined that the middle frame of the target human voice segment is the starting frame of the snoring segment, the Mel-Cepstral Coefficient baseline value is updated based on the mean Mel-Cepstral Coefficient between the starting frame of the target human voice segment and the starting frame of the snoring segment.

[0398] In one possible implementation, when it is determined that the intermediate frame of the target vocal segment is the starting frame of the snoring segment, the Mel-frequency cepstral coefficients of each frame from the starting frame of the target vocal segment to the starting frame of the snoring segment can be calculated, and the average value of the Mel-frequency cepstral coefficients can be obtained. Furthermore, the reference value of the Mel-frequency cepstral coefficients can be replaced with this average value.

[0399] For the detection of the snoring segment of the next target human voice segment, if the starting frame of the snoring segment is determined, the reference value of the Mel-Cepstral Coefficient used by the target human voice segment can also be determined based on the above processing, thereby realizing the adaptive update of the reference value of the Mel-Cepstral Coefficient.

[0400] Since the snoring signals of different target human voice segments may have different Mel-Cepstral Coefficients, through the above-mentioned update process, the reference value of the Mel-Cepstral Coefficient used for each target human voice segment can be adapted to the snoring of that target human voice segment, thereby improving the accuracy of determining the end frame of the snoring segment and thus improving the accuracy of snoring segment detection.

[0401] Optionally, the ratio threshold in the above-mentioned spectral conditions can also be updated adaptively. This embodiment provides two update methods as follows:

[0402] Method 1: When the starting frame of the target human voice segment is determined to be the starting frame of the snoring segment, calculate the average low-frequency energy and average mid-frequency energy of each of the at least two human voice segments preceding the target human voice segment, and calculate the first ratio of the average low-frequency energy to the average mid-frequency energy of each human voice segment; calculate the average low-frequency energy and average mid-frequency energy of the starting frame, and calculate the second ratio of the average low-frequency energy to the average mid-frequency energy of the starting frame; based on the first ratio and the second ratio, update the ratio threshold in the spectrum conditions.

[0403] In one possible implementation, when it is determined that the starting frame of the target human voice segment is the starting frame of the snoring segment, a weighted average can be performed based on the first ratio of at least two human voice segments preceding the target human voice segment, the second ratio of the target human voice segment, and the ratio threshold to be updated. The resulting weighted average is used as the updated ratio threshold, which is then used as the ratio threshold for the next target human voice segment.

[0404] If the method provided in any of the above embodiments determines that the starting frame of the human voice segment is the starting frame of the snoring segment, the ratio threshold in the spectral conditions can be updated using the calculated first ratio and second ratio, without having to recalculate the first ratio and second ratio.

[0405] Method 2: When it is determined that the middle frame of the target human voice segment is the starting frame of the snoring segment, the ratio threshold is updated based on the ratio of the average low-frequency energy and the average mid-frequency energy of the above-mentioned spectrum average.

[0406] In one possible implementation, when it is determined that the middle frame of the target human voice segment is the starting frame of the snoring segment, the above-mentioned mean ratio can be weighted and averaged with the ratio threshold to be updated, and the resulting weighted average can be used as the updated ratio threshold, which is then used as the ratio threshold for the next target human voice segment.

[0407] The ratio threshold that has never been updated can be set as the initial value. The ratio threshold can also be set with upper and lower limits based on a large number of experiments to keep the ratio threshold within a reasonable range.

[0408] Since the energy ratio of low and mid frequencies in each user's snoring is different, and the snoring of the same user may also be different at different times, the above processing can adaptively adjust the low and mid frequency ratio threshold when the starting frame of the target voice segment is determined to be the starting frame of the snoring segment. This makes the ratio threshold match the current snoring, improves the accuracy of determining the starting frame of the snoring segment, and thus improves the accuracy of snoring segment detection.

[0409] Step 603: If the starting frame of the snoring segment is determined, the ending frame of the snoring segment is determined starting from the next frame after the starting frame of the snoring segment.

[0410] Specifically, the processing in step 603 can be as follows:

[0411] Starting from the next frame after the beginning of the snoring segment, the second current frame is processed as follows:

[0412] Determine whether the second current frame is the end frame of the target human voice segment;

[0413] If the second current frame is the end frame of the target human voice segment, then the second current frame is taken as the end frame of the snoring segment;

[0414] If the second current frame is not the end frame of the target human voice segment, then determine whether the second current frame is the end frame of the snoring segment. If it is, then the second current frame is used as the end frame of the snoring segment; if not, then the above detection process is performed on the next frame of the second current frame.

[0415] In one possible implementation, after determining the start frame of the snoring segment, it can be determined whether the next frame is the end frame. The specific processing is the same as step 304 above, and will not be repeated here. If an end frame is determined, it is considered that the snoring has also ended, and the second current frame (i.e., the end frame) can be used as the end frame of the snoring segment.

[0416] If the second current frame is not determined as the end frame, then determine whether the second current frame is the end frame of the snoring segment.

[0417] Optionally, it can be determined whether the second current frame is the end frame of the snoring segment based on the Mel cepstral coefficient reference value and the Mel cepstral coefficient of the second current frame.

[0418] The Mel cepstral coefficient reference value can be the same as the Mel cepstral coefficient reference value used in the process of determining the starting frame of the snoring segment.

[0419] In one possible implementation, the Mel-Cepstral Coefficients of the second current frame can be calculated, and the cross-correlation coefficient between the Mel-Cepstral Coefficients and a reference value can be calculated. It can then be determined whether the cross-correlation coefficient is less than a corresponding threshold. If the cross-correlation coefficient is less than the corresponding threshold, the second current frame can be considered not to belong to snoring, and thus the second current frame is designated as the end frame of the snoring segment.

[0420] In another possible implementation, the Mel-frequency cepstral coefficients (MFCCs) of each frame between the start frame of the snoring segment and the second current frame can be calculated, and their average value can be taken as the average MFCC from the start frame to the second current frame. The cross-correlation coefficient between this average MFCC and the baseline MFCC is calculated, and it is determined whether the cross-correlation coefficient is less than a corresponding threshold. If the cross-correlation coefficient is less than the corresponding threshold, the second current frame can be considered not to belong to snoring, and this second current frame is then taken as the end frame of the snoring segment. Compared with the previous implementation, the end frame of the snoring segment can be determined after a certain delay of a certain number of frames. Based on this, it can be ensured that the snoring signal contains a larger amount of snoring information.

[0421] If the cross-correlation coefficient is not less than the corresponding threshold, then the processing of step 603 above continues for the next frame.

[0422] Alternatively, if the aforementioned cross-correlation coefficient is not less than the corresponding threshold, then it is further determined whether the number of frames from the start frame of the snoring segment to the second current frame is greater than the snoring frame count threshold. If the number of frames is greater than the snoring frame count threshold, then the second current frame can be used as the end frame of the snoring segment; if the number of frames is not greater than the snoring frame count threshold, then the processing of step 603 above continues for the next frame.

[0423] Step 604: The sound signal between the start frame and the end frame of the snoring segment is taken as the snoring signal.

[0424] In one possible implementation, the start frame and end frame of the snoring segment can be recorded. When analysis of the snoring signal is required, the corresponding snoring signal can be extracted based on the start and end frames of the snoring segment. This embodiment does not limit the subsequent processing of the snoring signal.

[0425] This embodiment can achieve the following beneficial effects:

[0426] (1) After real-time acquisition of sound signals, human voice segments can be detected in real time to determine the human voice segments in the sound signal. After determining at least two human voice segments, starting from the starting frame of the target human voice segment, the starting frame and ending frame of the snoring segment are detected frame by frame. Since the computational cost of the above processing is relatively small compared to machine learning algorithms such as Gaussian mixture models, hidden Markov models, clustering models, and support vector machines, real-time snoring segment detection can be implemented on embedded systems.

[0427] (2) In determining whether the middle frame of the target human voice segment is the starting frame of the snoring segment, a combination of snoring start conditions such as snoring duration condition, Mel cepstral coefficient condition, non-noise condition, zero-crossing rate condition, spectrum condition, and fundamental frequency condition are used to make a comprehensive judgment, which improves the detection accuracy and noise resistance performance.

[0428] (3) Adaptively adjust the ratio threshold of low frequency and mid frequency so that the ratio threshold is adapted to the current snoring and improves the accuracy of determining the starting frame of the snoring segment.

[0429] (4) Adaptively adjust the reference value of the Mel cepstral coefficient so that the reference value of the Mel cepstral coefficient used for each target human voice segment can be matched with the snoring of the target human voice segment, thereby improving the accuracy of determining the start frame and end frame of the snoring segment.

[0430] This invention also provides an apparatus for determining snoring signals, which is used to implement any of the methods described above for determining snoring signals. Figure 9 The schematic block diagram shown illustrates that the device 800 for determining snoring signals includes: an acquisition module 801, a first determination module 802, and a second determination module 803.

[0431] The acquisition module 801 is used to acquire sound signals, detect human voice segments in the sound signals, and detect at least two human voice segments before the target human voice segment.

[0432] The first determining module 802 is used to determine whether the first current frame in the target human voice segment is the starting frame of the snoring segment if the starting frame of the target human voice segment is detected in the sound signal.

[0433] The second determining module 803 is used to determine the snoring segment end frame starting from the next frame after the snoring segment start frame if the snoring segment start frame is determined; and to take the sound signal between the snoring segment start frame and the snoring segment end frame as the snoring signal.

[0434] Optionally, when the first current frame is the starting frame of the target human voice segment, the first determining module 802 is configured to:

[0435] In the target human voice segment, based on the similarity of the preset characteristics of the starting frame of the at least two human voice segments and the target human voice segment and / or the snoring characteristics, it is determined whether the starting frame of the target human voice segment is the starting frame of the snoring segment.

[0436] Optionally, the first determining module 802 is used to:

[0437] Based on the similarity of the preset characteristics of the starting frames of the at least two human voice segments and the target human voice segment, a similarity score is calculated for the starting frame of the target human voice segment; based on the similarity score of the starting frame of the target human voice segment, it is determined whether the starting frame of the target human voice segment is the starting frame of a snoring segment; or

[0438] Based on the snoring characteristics of the starting frames of the at least two human voice segments and the target human voice segment, a snoring characteristic score of the starting frame of the target human voice segment is calculated; based on the snoring characteristic score of the starting frame of the target human voice segment, it is determined whether the starting frame of the target human voice segment is a snoring segment starting frame.

[0439] Optionally, the first determining module 802 is used to:

[0440] Based on the similarity of the preset characteristics of the starting frames of the at least two human voice segments and the target human voice segment, a similarity score of the starting frame of the target human voice segment is calculated.

[0441] Based on the snoring characteristics of the starting frames of the at least two human voice segments and the target human voice segment, a snoring characteristic score of the starting frame of the target human voice segment is calculated.

[0442] Based on the similarity score and the snoring characteristic score, it is determined whether the starting frame of the target human voice segment is the starting frame of the snoring segment.

[0443] Optionally, the similarity of the preset characteristics includes any one or more of the following: energy similarity, spectral similarity, Mel-frequency cepstral coefficient similarity, and noise similarity;

[0444] The snoring characteristics include any one or more of the following: snoring energy characteristics, snoring fundamental frequency characteristics, and snoring zero-crossing rate characteristics.

[0445] Optionally, the first determining module 802 is used to:

[0446] For energy similarity, the average energy of each voice segment in the at least two voice segments is calculated, and the difference ratio between the average energy of each voice segment is calculated. If the difference ratio is within a preset ratio range, the corresponding first similarity score is added to the similarity score of the starting frame of the target voice segment.

[0447] For spectral similarity, determine the number of first peaks in the spectrum of the at least two human voice segments that meet the condition of overlapping peak positions. If the number of first peaks is greater than the first threshold, then add a corresponding second similarity score to the similarity score of the starting frame of the target human voice segment.

[0448] For Mel-Cepstral Coefficient Similarity, the Mel-Cepstral Coefficient of each of the at least two human voice segments and the Mel-Cepstral Coefficient of the starting frame of the target human voice segment are calculated. The first cross-correlation coefficient between any two Mel-Cepstral Coefficients in the at least two human voice segments and the starting frame of the target human voice segment is calculated respectively. If the first cross-correlation coefficient is greater than the first preset threshold, the corresponding third similarity score is added to the similarity score of the starting frame of the target human voice segment.

[0449] For noise similarity, a second cross-correlation coefficient is calculated between any Mel-Cepstral Coefficient in the starting frame of the at least two human voice segments and the target human voice segment and the Mel-Cepstral Coefficient of the preset noise. If any of the second cross-correlation coefficients is greater than a second preset threshold, the corresponding fourth similarity score is subtracted from the similarity score of the starting frame of the target human voice segment. If all the second cross-correlation coefficients are less than a third preset threshold, the corresponding fifth similarity score is added to the similarity score of the starting frame of the target human voice segment.

[0450] Optionally, the first determining module 802 is further configured to:

[0451] Regarding spectral similarity, if the number of the first peaks is greater than a preset threshold, then in the spectrum of the starting frame of the target human voice segment, determine the number of second peaks that meet the condition of peak position overlap with the at least two human voice segments. If the number of the second peaks is greater than a second threshold, then add a corresponding sixth similarity score to the similarity score of the starting frame of the target human voice segment.

[0452] Optionally, the first determining module 802 is used to:

[0453] Regarding the energy characteristics of snoring:

[0454] Calculate the average low-frequency energy and average mid-frequency energy of each of the at least two human voice segments, and calculate the first ratio of the average low-frequency energy to the average mid-frequency energy of each human voice segment. If the first ratio is greater than the ratio threshold, add the corresponding first snoring characteristic score to the snoring characteristic score of the starting frame of the target human voice segment.

[0455] Calculate the average low-frequency energy and average mid-frequency energy of the starting frame of the target human voice segment, and calculate a second ratio of the average low-frequency energy to the average mid-frequency energy of the starting frame of the target human voice segment. If the second ratio is greater than the ratio threshold, add a corresponding second snoring characteristic score to the snoring characteristic score of the starting frame of the target human voice segment.

[0456] Regarding the fundamental frequency characteristics of snoring:

[0457] Calculate the fundamental frequency of each of the at least two human voice segments, and calculate the fundamental frequency difference between each human voice segment. If the fundamental frequency of each human voice segment is within the snoring fundamental frequency range, and the fundamental frequency difference is within the fundamental frequency difference range, then add the corresponding third snoring characteristic score to the snoring characteristic score of the starting frame of the target human voice segment.

[0458] Calculate the fundamental frequency of the starting frame of the target human voice segment. If the fundamental frequency of the starting frame of the target human voice segment is within the range of the snoring fundamental frequency, then add a corresponding fourth snoring characteristic score to the snoring characteristic score of the starting frame of the target human voice segment.

[0459] Regarding the zero-crossing rate characteristic of snoring:

[0460] Calculate the average zero-crossing rate of each voice segment in the at least two voice segments. If the average zero-crossing rate of each voice segment is within the range of snoring zero-crossing rate, add the corresponding fifth snoring characteristic score to the snoring characteristic score of the starting frame of the target voice segment.

[0461] Calculate the zero-crossing rate of the starting frame of the target human voice segment. If the zero-crossing rate of the starting frame of the target human voice segment is within the range of the snoring zero-crossing rate, then add the corresponding sixth snoring characteristic score to the snoring characteristic score of the starting frame of the target human voice segment.

[0462] Optionally, the device further includes an update module, the update module being used for:

[0463] When it is determined that the starting frame of the target human voice segment is the starting frame of the snoring segment, the ratio threshold is updated based on the first ratio and the second ratio; and / or

[0464] If, after determining that the starting frame of the target human voice segment is not the starting frame of the snoring segment, the intermediate frame of the target human voice segment is determined to be the starting frame of the snoring segment, then the ratio threshold is updated based on the ratio of the average low-frequency energy and the average mid-frequency energy of the average spectrum between the starting frame of the target human voice segment and the starting frame of the snoring segment.

[0465] Optionally, when the first current frame is any intermediate frame after the starting frame of the target human voice segment, the first determining module 802 is configured to:

[0466] If an intermediate frame of the target human voice segment is determined to satisfy any one or more of the following snoring initiation conditions, then the intermediate frame of the target human voice segment is determined to be the snoring initiation frame. The snoring initiation conditions include: snoring duration condition, Mel-frequency cepstral coefficient condition, non-noise condition, zero-crossing rate condition, spectral condition, and fundamental frequency condition.

[0467] Optionally, determining that the intermediate frame of the target human voice segment meets the snoring duration condition means: determining that the number of frames between the starting frame and the intermediate frame of the target human voice segment is within the snoring duration range, and / or determining that the duration between the starting frame of the target human voice segment and the ending frame of the previous human voice segment is greater than the snoring interval duration.

[0468] Determining that the intermediate frame of the target vocal segment satisfies the Mel-Cepstral Coefficient condition means: calculating the first Mel-Cepstral Coefficient mean between the starting frame of the target vocal segment and the intermediate frame of the target vocal segment, calculating the first cross-correlation coefficient between the first Mel-Cepstral Coefficient mean and the Mel-Cepstral Coefficient reference value, and determining that the first cross-correlation coefficient is greater than a first preset threshold.

[0469] Determining that the intermediate frames of the target human voice segment meet the non-noise condition means: calculating the second cross-correlation coefficient between the mean of the first Mel-Cepstral Coefficients and the Mel-Cepstral Coefficients of the preset noise, and determining that the second cross-correlation coefficient is less than the second preset threshold;

[0470] Determining that the intermediate frame of the target human voice segment satisfies the zero-crossing rate condition means: calculating the average zero-crossing rate between the starting frame of the target human voice segment and the intermediate frame of the target human voice segment, and determining that the average zero-crossing rate is within a preset zero-crossing rate range;

[0471] Determining that the intermediate frame of the target human voice segment meets the spectral condition means: calculating the spectral mean between the starting frame of the target human voice segment and the intermediate frame of the target human voice segment, calculating the low-frequency energy mean and the mid-frequency energy mean on the spectral mean, calculating the ratio of the mean of the low-frequency energy mean and the mean of the mid-frequency energy mean, and determining that the ratio of the mean is greater than the ratio threshold.

[0472] Determining that the intermediate frame of the target human voice segment meets the fundamental frequency condition means: determining the fundamental frequency value on the mean of the spectrum, and determining that the fundamental frequency value of the mean of the spectrum is within the range of the fundamental frequency of snoring.

[0473] Optionally, the update module is used for:

[0474] When it is determined that the starting frame of the target vocal segment is the starting frame of the snoring segment, the average low-frequency energy and average mid-frequency energy of each vocal segment in at least two vocal segments preceding the target vocal segment are calculated, and a first ratio of the average low-frequency energy to the average mid-frequency energy of each vocal segment is calculated; the average low-frequency energy and average mid-frequency energy of the starting frame of the target vocal segment are calculated, and a second ratio of the average low-frequency energy to the average mid-frequency energy of the starting frame of the target vocal segment is calculated; based on the first ratio and the second ratio, the ratio threshold is updated; and / or

[0475] When it is determined that the middle frame of the target human voice segment is the starting frame of the snoring segment, the ratio threshold is updated based on the ratio of the average low-frequency energy and the average mid-frequency energy of the average spectrum between the starting frame of the target human voice segment and the starting frame of the snoring segment.

[0476] Optionally, at least one snoring signal is detected in at least two of the at least two voice segments.

[0477] Optionally, the second determining module 803 is used for:

[0478] Starting from the next frame after the starting frame of the snoring segment, the second current frame is processed as follows:

[0479] Determine whether the second current frame is the end frame of the target human voice segment;

[0480] If the second current frame is the end frame of the target human voice segment, then the second current frame is taken as the end frame of the snoring segment;

[0481] If the second current frame is not the end frame of the target human voice segment, then determine whether the second current frame is the end frame of the snoring segment. If yes, then the second current frame is used as the end frame of the snoring segment; if no, then the detection process is performed on the next frame of the second current frame.

[0482] Optionally, the second determining module 803 is used for:

[0483] Based on the Mel-Cepstral Coefficient baseline value and the Mel-Cepstral Coefficient of the second current frame, it is determined whether the second current frame is the end frame of the snoring segment.

[0484] Optionally, the update module is used for:

[0485] When it is determined that the starting frame of the target vocal segment is the starting frame of the snoring segment, the Mel-Cepstral Coefficient (MCC) reference value is updated based on the MMC of each of the at least two vocal segments preceding the target vocal segment and the MMC of the starting frame of the target vocal segment; and / or

[0486] When it is determined that the middle frame of the target human voice segment is the starting frame of the snoring segment, the Mel-Cepstral Coefficient Reference Value is updated based on the mean Mel-Cepstral Coefficient between the starting frame of the target human voice segment and the starting frame of the snoring segment.

[0487] In this embodiment of the invention, after real-time acquisition of the sound signal, human voice segments can be detected in real time to determine the human voice segments in the sound signal. After determining at least two human voice segments, starting from the start frame of the target human voice segment, the start frame and end frame of the snoring segment are detected frame by frame. Since the computational cost of the above processing is relatively small compared to machine learning algorithms such as Gaussian mixture models, hidden Markov models, clustering models, and support vector machines, real-time snoring segment detection can be implemented on embedded systems.

[0488] An exemplary embodiment of the present invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the electronic device to perform a method according to an embodiment of the present invention.

[0489] An exemplary embodiment of the present invention also provides a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method according to an embodiment of the present invention.

[0490] An exemplary embodiment of the present invention also provides a computer program product, including a computer program, wherein, when executed by a computer's processor, the computer program is used to cause the computer to perform a method according to an embodiment of the present invention.

[0491] refer to Figure 9 The present invention will now be described in the form of a structural block diagram of an electronic device 900 that can serve as a server or client of the present invention, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0492] like ​As shown, the electronic device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. The RAM 903 may also store various programs and data required for the operation of the device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0493] Multiple components in electronic device 900 are connected to I / O interface 905, including: input unit 906, output unit 907, storage unit 908, and communication unit 909. Input unit 906 can be any type of device capable of inputting information to electronic device 900. Input unit 906 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 907 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 908 may include, but is not limited to, disk and optical disk. Communication unit 909 allows electronic device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0494] The computing unit 901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above. For example, in some embodiments, the method for determining snoring signals described above can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 900 via ROM 902 and / or communication unit 909. In some embodiments, the computing unit 901 can be configured to perform the method for determining snoring signals described above by any other suitable means (e.g., by means of firmware).

[0495] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0496] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0497] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0498] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0499] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0500] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

Claims

1. A method for determining snoring signals, characterized in that, The method includes: Acquire sound signals, perform human voice segment detection on the sound signals, and detect at least two human voice segments before the target human voice segment for snoring segment detection; If the starting frame of the target human voice segment is detected in the sound signal, then it is determined whether the first current frame in the target human voice segment is the starting frame of the snoring segment, wherein the first current frame is the starting frame of the target human voice segment, or the first current frame is any intermediate frame after the starting frame of the target human voice segment. When the first current frame is the starting frame of the target human voice segment, determining whether the first current frame in the target human voice segment is the starting frame of the snoring segment includes: In the target human voice segment, based on the similarity of the preset characteristics of the starting frame of the at least two human voice segments and the target human voice segment, a similarity score of the starting frame of the target human voice segment is calculated, wherein the similarity of the preset characteristics includes any one or more of the following: energy similarity, spectral similarity, Mel-frequency cepstral coefficient similarity, and noise similarity. Based on the snoring characteristics of the starting frames of the at least two human voice segments and the target human voice segment, a snoring characteristic score of the starting frame of the target human voice segment is calculated, wherein the snoring characteristics include any one or more of the following: snoring energy characteristics, snoring fundamental frequency characteristics, and snoring zero-crossing rate characteristics. Based on the similarity score and the snoring characteristic score, it is determined whether the starting frame of the target human voice segment is the starting frame of the snoring segment; If the start frame of the snoring segment is determined, the end frame of the snoring segment is determined from the next frame after the start frame of the snoring segment. The sound signal between the start frame and the end frame of the snoring segment is taken as the snoring signal; The step of determining the end frame of the snoring segment from the next frame after the start frame of the snoring segment includes: Starting from the next frame after the starting frame of the snoring segment, the second current frame is processed as follows: Determine whether the second current frame is the end frame of the target human voice segment; If the second current frame is the end frame of the target human voice segment, then the second current frame is taken as the end frame of the snoring segment; If the second current frame is not the end frame of the target human voice segment, then determine whether the second current frame is the end frame of the snoring segment. If yes, then the second current frame is used as the end frame of the snoring segment; if no, then the detection process is performed on the next frame of the second current frame.

2. The method according to claim 1, characterized in that, Regarding energy similarity, the calculation of the similarity score of the starting frame of the target human voice segment includes: calculating the average energy of each human voice segment in the at least two human voice segments, and calculating the difference ratio of the average energy between each human voice segment. If the difference ratio is within a preset ratio range, a corresponding first similarity score is added to the similarity score of the starting frame of the target human voice segment. Regarding spectral similarity, the calculation of the similarity score of the starting frame of the target human voice segment includes: determining the number of first peaks in the spectrum of the at least two human voice segments that meet the condition of overlapping peak positions; if the number of first peaks is greater than the first threshold, then a corresponding second similarity score is added to the similarity score of the starting frame of the target human voice segment. For Mel-Cepstral Coefficient Similarity, the calculation of the similarity score of the starting frame of the target vocal segment includes: calculating the Mel-Cepstral Coefficient of each vocal segment in the at least two vocal segments and the Mel-Cepstral Coefficient of the starting frame of the target vocal segment; calculating the first cross-correlation coefficient between any two Mel-Cepstral Coefficients in the at least two vocal segments and the starting frame of the target vocal segment; if the first cross-correlation coefficient is greater than a first preset threshold, then adding a corresponding third similarity score to the similarity score of the starting frame of the target vocal segment. Regarding noise similarity, calculating the similarity score of the starting frame of the target human voice segment includes: calculating a second cross-correlation coefficient between any Mel-Cepstral Coefficient in the starting frame of the at least two human voice segments and the target human voice segment and the Mel-Cepstral Coefficient of the preset noise; if any of the second cross-correlation coefficients is greater than a second preset threshold, then the corresponding fourth similarity score is subtracted from the similarity score of the starting frame of the target human voice segment; if all the second cross-correlation coefficients are less than a third preset threshold, then the corresponding fifth similarity score is added to the similarity score of the starting frame of the target human voice segment.

3. The method according to claim 2, characterized in that, Regarding spectral similarity, calculating the similarity score of the starting frame of the target human voice segment further includes: If the number of the first peaks is greater than a preset threshold, then in the spectrum of the starting frame of the target voice segment, determine the number of second peaks that meet the peak position overlap condition compared with the at least two voice segments. If the number of the second peaks is greater than a second threshold, then add a corresponding sixth similarity score to the similarity score of the starting frame of the target voice segment.

4. The method according to claim 1, characterized in that, Regarding the snoring energy characteristics, the calculation of the snoring characteristic score of the starting frame of the target vocal segment includes: Calculate the average low-frequency energy and average mid-frequency energy of each of the at least two human voice segments, and calculate the first ratio of the average low-frequency energy to the average mid-frequency energy of each human voice segment. If the first ratio is greater than the ratio threshold, add the corresponding first snoring characteristic score to the snoring characteristic score of the starting frame of the target human voice segment. Calculate the average low-frequency energy and average mid-frequency energy of the starting frame of the target human voice segment, and calculate a second ratio of the average low-frequency energy to the average mid-frequency energy of the starting frame of the target human voice segment. If the second ratio is greater than the ratio threshold, add a corresponding second snoring characteristic score to the snoring characteristic score of the starting frame of the target human voice segment. Regarding the fundamental frequency characteristics of snoring, the calculation of the snoring characteristic score of the starting frame of the target human voice segment includes: Calculate the fundamental frequency of each of the at least two human voice segments, and calculate the fundamental frequency difference between each human voice segment. If the fundamental frequency of each human voice segment is within the snoring fundamental frequency range, and the fundamental frequency difference is within the fundamental frequency difference range, then add the corresponding third snoring characteristic score to the snoring characteristic score of the starting frame of the target human voice segment. Calculate the fundamental frequency of the starting frame of the target human voice segment. If the fundamental frequency of the starting frame of the target human voice segment is within the range of the snoring fundamental frequency, then add a corresponding fourth snoring characteristic score to the snoring characteristic score of the starting frame of the target human voice segment. Regarding the snoring zero-crossing rate characteristic, the calculation of the snoring characteristic score of the starting frame of the target human voice segment includes: Calculate the average zero-crossing rate of each voice segment in the at least two voice segments. If the average zero-crossing rate of each voice segment is within the range of snoring zero-crossing rate, add the corresponding fifth snoring characteristic score to the snoring characteristic score of the starting frame of the target voice segment. Calculate the zero-crossing rate of the starting frame of the target human voice segment. If the zero-crossing rate of the starting frame of the target human voice segment is within the range of the snoring zero-crossing rate, then add the corresponding sixth snoring characteristic score to the snoring characteristic score of the starting frame of the target human voice segment.

5. The method according to claim 4, characterized in that, The method further includes: When it is determined that the starting frame of the target vocal segment is the starting frame of the snoring segment, a weighted average is calculated based on the first ratio of at least two vocal segments preceding the target vocal segment, the second ratio of the target vocal segment, and the ratio threshold to be updated. The resulting weighted average is used as the updated ratio threshold; and / or If, after determining that the starting frame of the target human voice segment is not the starting frame of the snoring segment, the intermediate frame of the target human voice segment is determined to be the starting frame of the snoring segment, then a weighted average is calculated based on the ratio of the average low-frequency energy and the average mid-frequency energy of the spectral mean between the starting frame of the target human voice segment and the starting frame of the snoring segment, and the ratio threshold to be updated. The resulting weighted average is then used as the updated ratio threshold.

6. The method according to claim 1, characterized in that, When the first current frame is any intermediate frame after the start frame of the target human voice segment, determining whether the first current frame in the target human voice segment is the start frame of the snoring segment includes: If the intermediate frame of the target human voice segment is determined to satisfy any one or more of the following snoring initiation conditions, then the intermediate frame of the target human voice segment is determined to be the snoring initiation frame. The snoring initiation conditions include: snoring duration condition, Mel-frequency cepstral coefficient condition, non-noise condition, zero-crossing rate condition, spectrum condition, and fundamental frequency condition. in, Determining that the intermediate frame of the target human voice segment meets the snoring duration condition means: determining that the number of frames between the starting frame and the intermediate frame of the target human voice segment is within the snoring duration range, and / or determining that the duration between the starting frame of the target human voice segment and the ending frame of the previous human voice segment is greater than the snoring interval duration. Determining that the intermediate frame of the target vocal segment satisfies the Mel-Cepstral Coefficient condition means: calculating the first Mel-Cepstral Coefficient mean between the starting frame of the target vocal segment and the intermediate frame of the target vocal segment, calculating the first cross-correlation coefficient between the first Mel-Cepstral Coefficient mean and the Mel-Cepstral Coefficient reference value, and determining that the first cross-correlation coefficient is greater than a first preset threshold. Determining that the intermediate frames of the target human voice segment meet the non-noise condition means: calculating the second cross-correlation coefficient between the mean of the first Mel-Cepstral Coefficients and the Mel-Cepstral Coefficients of the preset noise, and determining that the second cross-correlation coefficient is less than the second preset threshold; Determining that the intermediate frame of the target human voice segment satisfies the zero-crossing rate condition means: calculating the average zero-crossing rate between the starting frame of the target human voice segment and the intermediate frame of the target human voice segment, and determining that the average zero-crossing rate is within a preset zero-crossing rate range; Determining that the intermediate frame of the target human voice segment meets the spectral condition means: calculating the spectral mean between the starting frame of the target human voice segment and the intermediate frame of the target human voice segment, calculating the low-frequency energy mean and the mid-frequency energy mean on the spectral mean, calculating the ratio of the mean of the low-frequency energy mean and the mean of the mid-frequency energy mean, and determining that the ratio of the mean is greater than the ratio threshold. Determining that the intermediate frame of the target human voice segment meets the fundamental frequency condition means: determining the fundamental frequency value on the mean of the spectrum, and determining that the fundamental frequency value of the mean of the spectrum is within the range of the fundamental frequency of snoring.

7. The method according to claim 6, characterized in that, The method further includes: When the starting frame of the target vocal segment is determined to be the starting frame of the snoring segment, the average low-frequency energy and average mid-frequency energy of each vocal segment in at least two vocal segments preceding the target vocal segment are calculated, and a first ratio of the average low-frequency energy to the average mid-frequency energy of each vocal segment is calculated; the average low-frequency energy and average mid-frequency energy of the starting frame of the target vocal segment are calculated, and a second ratio of the average low-frequency energy to the average mid-frequency energy of the starting frame of the target vocal segment is calculated; a weighted average is calculated based on the first ratio of the at least two vocal segments preceding the target vocal segment, the second ratio of the target vocal segment, and a ratio threshold to be updated, and the resulting weighted average is used as the updated ratio threshold; and / or When it is determined that the middle frame of the target human voice segment is the starting frame of the snoring segment, a weighted average is calculated based on the ratio of the average low-frequency energy and the average mid-frequency energy of the average spectrum between the starting frame of the target human voice segment and the starting frame of the snoring segment to the ratio threshold to be updated, and the resulting weighted average is used as the updated ratio threshold.

8. The method according to claim 7, characterized in that, At least one snoring signal was detected in at least two of the at least two voice segments.

9. The method according to claim 1, characterized in that, Determining whether the second current frame is the end frame of the snoring segment includes: Based on the Mel-Cepstral Coefficient baseline value and the Mel-Cepstral Coefficient of the second current frame, it is determined whether the second current frame is the end frame of the snoring segment.

10. The method according to any one of claims 6 or 9, characterized in that, The method further includes: When the starting frame of the target vocal segment is determined to be the starting frame of the snoring segment, the Mel-spectral coefficients of each frame in at least two vocal segments preceding the target vocal segment are calculated, along with the Mel-spectral coefficients of the starting frame of the target vocal segment. The average of these average Mel-spectral coefficients for the at least two vocal segments and the starting frame is then used as the updated Mel-spectral coefficient baseline value; and / or When it is determined that the middle frame of the target human voice segment is the starting frame of the snoring segment, the Mel-Cepstral Coefficients of each frame from the starting frame of the target human voice segment to the starting frame of the snoring segment are calculated, and the mean value of the Mel-Cepstral Coefficients is obtained after taking the mean value, which is used as the updated reference value of the Mel-Cepstral Coefficients.

11. A device for determining snoring signals, characterized in that, The device includes: The acquisition module is used to acquire sound signals, detect human voice segments in the sound signals, and detect at least two human voice segments before the target human voice segment for snoring segment detection; A first determining module is configured to, if the starting frame of a target human voice segment is detected in the sound signal, determine whether a first current frame in the target human voice segment is the starting frame of a snoring segment, wherein the first current frame is the starting frame of the target human voice segment, or the first current frame is any intermediate frame following the starting frame of the target human voice segment; when the first current frame is the starting frame of the target human voice segment, calculate the similarity of the starting frame of the target human voice segment to a preset characteristic of the at least two human voice segments and the starting frame of the target human voice segment. Similarity score, wherein the similarity of the preset characteristics includes any one or more of the following: energy similarity, spectral similarity, Mel-frequency cepstral coefficient similarity, noise similarity; based on the snoring characteristics of the starting frames of the at least two human voice segments and the target human voice segment, a snoring characteristic score of the starting frame of the target human voice segment is calculated, wherein the snoring characteristics include any one or more of the following: snoring energy characteristics, snoring fundamental frequency characteristics, snoring zero-crossing rate characteristics; based on the similarity score and the snoring characteristic score, it is determined whether the starting frame of the target human voice segment is the starting frame of the snoring segment; The second determining module is configured to, if a snoring segment start frame is determined, determine a snoring segment end frame starting from the next frame of the snoring segment start frame; and use the sound signal between the snoring segment start frame and the snoring segment end frame as a snoring signal; wherein, determining the snoring segment end frame starting from the next frame of the snoring segment start frame includes: starting from the next frame of the snoring segment start frame, performing the following detection processing on the second current frame: determining whether the second current frame is the end frame of the target human voice segment; if the second current frame is the end frame of the target human voice segment, then using the second current frame as the snoring segment end frame; if the second current frame is not the end frame of the target human voice segment, then determining whether the second current frame is the snoring segment end frame, and if so, using the second current frame as the snoring segment end frame; if not, then performing the detection processing on the next frame of the second current frame.

12. An electronic device, comprising: processor; as well as Stored program memory, The program includes instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1-10.

13. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-10.

Images