Respiratory sound analysis device, respiratory sound analysis method, and computer program

Abstract

To classify and output a plurality of sound types included in respiratory sound.SOLUTION: A respiratory sound analysis device includes: classification means for classifying respiratory sound into a plurality of sound types; input means for receiving an input for selecting a type of the respiratory sound to be output; output means for outputting the respiratory sound of the sound type selected according to the input received by the input means, as a spectral image; and change means capable of changing a color of the spectral image for every sound type selected according to the input received by the input means.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to the technical field of a respiratory sound analysis device, a respiratory sound analysis method, and a computer program for analyzing respiratory sounds including a plurality of sound types.

Background Art

[0002] As this type of device, there is known one that discriminates between normal respiratory sounds and abnormal respiratory sounds for the respiratory sounds of a living body detected by an electronic stethoscope or the like. For example, in Patent Document 1, a technique of outputting the analysis result of an abnormal sound by a three-dimensional display device has been proposed. In Patent Document 2, a technique of detecting a rattling sound as an abnormal sound included in a respiratory sound has been proposed. In Patent Document 3, a technique of dividing a sound waveform into a plurality of sections and discriminating sound types has been proposed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the techniques described in Patent Documents 1 to 3 above have a technical problem that a plurality of sound types included in a respiratory sound cannot be arbitrarily output.

[0005] Examples of the problems to be solved by the present invention include those described above. An object of the present invention is to provide a respiratory sound analysis device, a respiratory sound analysis method, and a computer program capable of separating and outputting a plurality of sound types included in a respiratory sound.

Means for Solving the Problem

[0006] The respiratory sound analysis apparatus for solving the above problems includes a separating means for separating respiratory sounds into a plurality of sound types, an input means for receiving an input for selecting the type of respiratory sound to be output, an output means for outputting the respiratory sound of the sound type selected according to the input received by the input means as a spectral image, and a changing means capable of changing the color of the spectral image for each sound type selected according to the input received by the input means.

[0007] The respiratory sound analysis method for solving the above problems includes a separating step for separating respiratory sounds into a plurality of sound types, an input step for receiving an input for selecting the type of respiratory sound to be output, an output step for outputting the respiratory sound of the sound type selected according to the input received by the input means as a spectral image, and a changing step capable of changing the color of the spectral image for each sound type selected according to the input received by the input means.

[0008] The computer program for solving the above problems causes a computer to execute a separating step for separating respiratory sounds into a plurality of sound types, an input step for receiving an input for selecting the type of respiratory sound to be output, an output step for outputting the respiratory sound of the sound type selected according to the input received by the input means as a spectral image, and a changing step capable of changing the color of the spectral image for each sound type selected according to the input received by the input means.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Figure 37

Figure 38

Mode for Carrying Out the Invention

[0010] <1> The respiratory sound analysis device according to this embodiment includes a classification means for classifying respiratory sounds into normal sounds and abnormal sounds, and an output means for outputting any one of the classified respiratory sounds.

[0011] According to the respiratory sound analysis device according to this embodiment, during its operation, first, respiratory sounds are classified into normal sounds and abnormal sounds. The respiratory sounds classified by the classification means do not necessarily have to be only two types, normal sounds and abnormal sounds, and each of the normal sounds and abnormal sounds may be further classified into a plurality of sound types. For example, abnormal sounds may be classified into whistling sounds, snoring-like sounds, crackling sounds, etc. respectively. Note that the specific classification method is not particularly limited, as long as it can be classified in a state where any respiratory sound described later can be output.

[0012] When the respiratory sounds are classified, any one of the classified respiratory sounds is output. That is, a plurality of classified respiratory sounds are selectively output. Thereby, for example, it is possible to output only the respiratory sounds desired by the user. More specifically, for example, only the whistling sound included in the respiratory sound can be output, or only the whistling sound and the snoring-like sound can be output. Note that the output mode is not particularly limited, and it may be output as sound or image, or in other modes.

[0013] By classifying and outputting respiratory sounds as described above, for example, diagnosis of health conditions can be easily performed. Specifically, for example, when it sounds like a plurality of respiratory sounds are mixed, it is difficult for even a skilled doctor to distinguish and listen to each sound type. However, if any respiratory sound can be output, it becomes possible to easily determine the sound types included in the respiratory sound. In this way, if it is possible to make it easier to listen to only specific sound types included in the respiratory sound, it can also be utilized in the education and research of doctors.

[0014] As described above, according to the respiratory sound analysis device according to this embodiment, since respiratory sounds can be classified and any respiratory sound can be output, it is possible to suitably analyze respiratory sounds including a plurality of sound types.

[0015] <2> In one aspect of the breath sound analysis device according to the present embodiment, the output means outputs a plurality of the separated breath sounds simultaneously.

[0016] According to this aspect, since a plurality of breath sounds can be output simultaneously as arbitrary breath sounds (in other words, in a superimposed state), desired breath sounds can be output by appropriately combining them.

[0017] <3> In another aspect of the breath sound analysis device according to the present embodiment, the output means outputs the arbitrary breath sound as voice or a spectral image.

[0018] According to this aspect, any of the separated breath sounds is output as voice using, for example, a speaker or headphones. Alternatively, any breath sound is output as a spectral image using a display such as a liquid crystal monitor. Therefore, the output arbitrary breath sound can be suitably used.

[0019] <4> In another aspect of the breath sound analysis device according to the present embodiment, the device further includes a changing means capable of changing the output state of the arbitrary breath sound for each sound type.

[0020] According to this aspect, since the output state (for example, output volume, image display mode, etc.) of an arbitrary breath sound can be changed for each sound type, the output arbitrary breath sound can be used more suitably. For example, by increasing the volume only for a specific abnormal sound included in the breath sound and outputting it, even in a state where a plurality of breath sounds are mixed, the specific abnormal sound can be made easier to hear. Also, even if it is made easy to hear once and then returned to the normal volume again, it is considered that it will be easier to hear thereafter. Therefore, it can be effectively utilized for training inexperienced doctors, etc.

[0021] <5> In the aspect further including the above-described changing means, the changing means may be capable of changing the output volume of the arbitrary breath sound for each sound type.

[0022] In this case, since the output volume can be changed for each sound type, the convenience can be improved by making only the desired breathing sound easier to hear, for example.

[0023] <6> In the mode where the output volume described above can be changed for each sound type, the changing means may be capable of changing the output volume of the arbitrary breathing sound for each predetermined frequency band.

[0024] In this case, since the change in the output volume for each sound type can be performed in more detail, for example, only the output volume in a predetermined frequency band can be increased to make the desired breathing sound even easier to hear. The predetermined frequency band may be set according to the characteristics of the sound types to be separated, etc.

[0025] <7> Alternatively, in an aspect further including changing means, the changing means may be capable of performing predetermined image processing for each sound type on the image indicating the arbitrary breathing sound.

[0026] In this case, for an image indicating an arbitrary breathing sound (for example, a spectrogram from which only one type of breathing sound is extracted), etc., predetermined image processing can be performed to realize a state that is visually easy to recognize. Examples of the predetermined image processing include color change (for example, RGB adjustment), binarization, edge detection, etc.

[0027] <8> In the mode where the above-described image processing can be performed for each sound type, the changing means may be capable of outputting, by overlapping, the images obtained by performing the predetermined image processing for each sound type with a plurality of sound types.

[0028] In this case, since a plurality of images for each sound type that have been separately subjected to image processing can be overlapped and displayed, a plurality of sound types displayed in an appropriate manner by the image processing can be collectively recognized in one image, and for example, comparison between a plurality of sound types can be suitably performed.

[0029] <9> In another aspect of the breath sound analysis apparatus according to the present embodiment, the classification means includes an acquisition means for acquiring information regarding a frequency corresponding to a predetermined feature of the spectrum of the breath sound, a shift means for shifting a plurality of reference spectra serving as criteria for classifying the breath sound according to the information regarding the frequency to obtain frequency-shifted reference spectra, and a ratio output means for outputting a ratio of the plurality of reference spectra included in the breath sound based on the breath sound and the frequency-shifted reference spectra.

[0030] According to this aspect, in the classification means, first, information regarding a frequency corresponding to a predetermined feature of the spectrum of the breath sound is acquired. Here, the "predetermined feature" means a feature that occurs at a specific frequency according to the sound type included in the spectrum of the biological sound, and is, for example, a peak appearing in the frequency-analyzed signal. Further, the "information regarding the frequency" is not limited to information directly indicating the frequency, but includes information that can indirectly derive the frequency.

[0031] When the information regarding the frequency is acquired, a plurality of reference spectra serving as criteria for classifying the breath sound are shifted according to the information regarding the frequency, and frequency-shifted reference spectra are acquired. Here, the "reference spectrum" is a spectrum preset according to each sound type in order to classify a plurality of sound types (for example, normal breath sound, continuous rattling sound, crackling sound, etc.) included in the breath sound. The reference spectrum is frequency-shifted according to, for example, the peak position, which is a predetermined feature obtained from the breath sound, to obtain a frequency-shifted reference spectrum.

[0032] When the frequency-shifted reference spectra are acquired, a ratio of a plurality of reference spectra included in the breath sound is output based on the breath sound and the frequency-shifted reference spectra. Specifically, it is calculated what ratio of sound types corresponding to the plurality of reference spectra is included in the breath sound to be analyzed, and the result is output. More specifically, for example, an operation based on a plurality of reference spectra is performed on the spectrum of the breath sound, and the ratio of the reference spectra is calculated as a coupling coefficient.

[0033] As a result of the above, according to the sorting means according to the present embodiment, breath sounds including a plurality of sound types can be suitably sorted. In particular, in the present embodiment, even when a plurality of breath sounds are mixed on the same frequency axis, the ratio of each sound type can be suitably sorted.

[0034] <10> In the mode of using the above-described frequency shift reference spectrum, the predetermined feature may be a maximum value.

[0035] In this case, for example, frequency analysis such as fast Fourier transform (FFT) is performed on a signal indicating a breath sound, and information regarding the frequency corresponding to the maximum value (i.e., peak) of the analysis result is obtained. Note that the information regarding the frequency is obtained as corresponding to the position of the maximum value, but may be obtained as information regarding the frequency corresponding to a position near the maximum value even if it is not the frequency that completely matches the position of the maximum value.

[0036] As described above, by using the maximum value as a predetermined feature of the spectrum of the breath sound, information regarding the frequency can be obtained more easily and accurately.

[0037] <11> The breath sound analysis method according to the present embodiment includes a sorting step of sorting breath sounds into normal sounds and abnormal sounds, and an output step of outputting an arbitrary breath sound among the sorted breath sounds.

[0038] According to the breath sound analysis method according to the present embodiment, similar to the breath sound analysis apparatus according to the present embodiment described above, breath sounds including a plurality of sound types can be suitably analyzed.

[0039] Note that also in the breath sound analysis method according to the present embodiment, it is possible to adopt various modes similar to the various modes in the breath sound analysis apparatus according to the present embodiment described above.

[0040] <12> The computer program according to this embodiment causes a computer to execute a classification step of classifying breathing sounds into normal sounds and abnormal sounds, and an output step of outputting any one of the classified breathing sounds.

[0041] According to the computer program according to this embodiment, since the computer can execute the same processing as the breathing sound analysis method according to the above-described embodiment, breathing sounds including a plurality of sound types can be suitably analyzed.

[0042] Note that also in the computer program according to this embodiment, various aspects similar to the various aspects in the breathing sound analysis apparatus according to the above-described embodiment can be adopted.

[0043] <13> The recording medium according to this embodiment stores the above-described computer program.

[0044] According to the recording medium according to this embodiment, by causing a computer to execute the above-described computer program, it is possible to suitably analyze breathing sounds including a plurality of sound types.

[0045] The actions and other advantages of the breathing sound analysis apparatus and the breathing sound analysis method according to this embodiment, as well as the computer program and the recording medium, will be described in more detail in the examples shown below.

Examples

[0046] Hereinafter, examples of the breathing sound analysis apparatus, the breathing sound analysis method, the computer program, and the recording medium will be described in detail with reference to the drawings.

[0047] <Overall Configuration> First, the overall configuration of the breathing sound analysis apparatus according to this embodiment will be described with reference to FIG. 1. Here, FIG. 1 is a block diagram showing the overall configuration of the breathing sound analysis apparatus according to this embodiment.

[0048] In FIG. 1, the breath sound analysis apparatus according to the present embodiment includes, as main components, a biological sound sensor 110, a signal storage unit 120, a signal processing unit 125, a voice output unit 130, a base holding unit 140, a display unit 150, an input unit 160, and a processing unit 200.

[0049] The biological sound sensor 110 is a sensor configured to be able to detect the breath sound of a living body. The biological sound sensor 110 is composed of, for example, an ECM (Electret Condenser Microphone), a microphone using a piezo, a vibration sensor, or the like.

[0050] The signal storage unit 120 is configured as a buffer such as a RAM (Random Access Memory), and temporarily stores a signal indicating the breath sound detected by the biological sound sensor 110 (hereinafter, appropriately referred to as a "breath sound signal"). The signal storage unit 120 is configured to be able to output the stored signal to the voice output unit 130 and the processing unit 200, respectively.

[0051] The signal processing unit 125 processes the sound acquired by the biological sound sensor 110 and outputs it to the voice output unit 130. The signal processing unit 125 functions as, for example, an equalizer or a filter, and processes the acquired sound into a state that is easy for a person to hear.

[0052] The voice output unit 130 is configured as, for example, a speaker or headphones, and outputs the breath sound detected by the biological sound sensor 110 and processed by the signal processing unit 125.

[0053] The base holding unit 140 is configured as, for example, a ROM (Read Only Memory) or the like, and stores a base corresponding to a predetermined sound type that may be included in the breath sound. The base according to the present embodiment is an example of the "reference spectrum" of the present invention.

[0054] The display unit 150 is configured as a display such as a liquid crystal monitor, and displays the image data output from the processing unit 200.

[0055] The input unit 160 is a device that receives input from a user, and is configured as, for example, a keyboard, a mouse, a touch panel, various switches, etc. The input unit 160 is configured to be capable of performing an input operation for selecting at least the breath sound to be output.

[0056] The processing unit 200 is configured to include a plurality of arithmetic circuits, a memory, and the like. The processing unit 200 includes a frequency analysis unit 210, a frequency peak detection unit 220, a basis set generation unit 230, a coupling coefficient calculation unit 240, a signal strength calculation unit 250, an image generation unit 260, and a breath sound selection unit 270.

[0057] The operations of each part of the processing unit 200 will be described in detail later.

[0058] <Operation Explanation> Next, the operation of the breath sound analysis device according to the present embodiment will be described with reference to FIG. 2. Here, FIG. 2 is a flowchart showing the operation of the breath sound analysis device according to the present embodiment. Here, a simple explanation will be given to grasp the overall flow of the processing executed by the breath sound analysis device according to the present embodiment. Details of each process will be described later.

[0059] In FIG. 2, when the breath sound analysis device according to the present embodiment operates, first, a breath sound is detected by the biological sound sensor 110, and a breath sound signal is acquired by the processing unit 200 (step S101).

[0060] When the breath sound signal is acquired, frequency analysis (for example, fast Fourier transform) is executed by the frequency analysis unit 210 (step S102). Also, the frequency peak detection unit 220 detects a peak (maximum value) using the frequency analysis result.

[0061] Subsequently, a basis set is generated in the basis set generation unit 230 (step S103). Specifically, the basis set generation unit 230 generates a basis set using the basis stored in the basis holding unit 140. At this time, the basis set generation unit 230 shifts the basis based on the peak position (i.e., the corresponding frequency) obtained from the frequency analysis result.

[0062] When the basis set is generated, the coupling coefficient calculation unit 240 calculates the coupling coefficient based on the frequency analysis result and the basis set (step S104).

[0063] When the coupling coefficient is calculated, the signal strength calculation unit 250 calculates the signal strength according to the coupling coefficient (step S105). In other words, the ratio of each sound type included in the breath sound signal is calculated.

[0064] When the signal strength is calculated, the image generation unit 260 generates image data indicating the signal strength. The generated image data is displayed as an analysis result on the display unit 150 (step S106).

[0065] After the analysis result is displayed, when the sound type to be output is input by the user (step S107: YES), the breath sound selection unit 270 selects the breath sound to be output, and the selected sound type is output to the voice output unit 130 or the display unit 150 (step S108).

[0066] Thereafter, a determination is made as to whether to continue the analysis process (step S109). If it is determined to continue the analysis process (step S109: YES), the process from step S101 is executed again. If it is determined not to continue the analysis process (step S109: NO), the series of processes ends.

[0067] <Specific example of breath sound signal> Next, a specific example of the breath sound signal analyzed by the breath sound analysis device according to this embodiment will be described with reference to FIGS. 3 and 4. Here, FIG. 3 is a spectrogram showing the frequency analysis result of the breath sound including crackles, and FIG. 4 is a spectrogram showing the frequency analysis result of the breath sound including wheezes.

[0068] In the example shown in FIG. 3, in addition to the spectrogram pattern corresponding to the normal breath sound, a spectrogram pattern corresponding to crackles, which is one of the abnormal breath sounds, is observed. The spectrogram pattern corresponding to crackles is close to a rhombus shape as shown in the enlarged portion in the figure.

[0069] In the example shown in FIG. 4, in addition to the spectrogram pattern corresponding to the normal breath sound, a spectrogram pattern corresponding to wheezes, which is one of the abnormal breath sounds, is observed. The spectrogram pattern corresponding to wheezes is shaped like a swan's neck as shown in the enlarged portion in the figure.

[0070] Thus, there are multiple sound types in the abnormal breath sound, and they are observed as spectrogram patterns with different shapes depending on the sound type. However, as can be seen from the figure, the normal breath sound and the abnormal breath sound are detected in a mixed state. The breath sound analysis device according to this embodiment performs an analysis for separating such a plurality of mixed sound types.

[0071] <Approximation method of breath sound signal> Next, the analysis method by the breath sound analysis device according to this embodiment will be briefly described with reference to FIGS. 5 to 8. Here, FIG. 5 is a graph showing the spectrum of the breath sound including crackles at a predetermined timing, and FIG. 6 is a conceptual diagram showing the approximation method of the spectrum of the breath sound including crackles. FIG. 7 is a graph showing the spectrum of the breath sound including wheezes at a predetermined timing, and FIG. 8 is a conceptual diagram showing the approximation method of the spectrum of the breath sound including wheezes.

[0072] In FIG. 5, for a breath sound signal including a crackling sound (see FIG. 3), when the spectrum is extracted at the timing when the spectrogram pattern corresponding to the crackling sound appears strongly, the results shown in the figure are obtained. This spectrum is considered to include normal breath sounds and crackling sounds.

[0073] In FIG. 6, the spectrum corresponding to the normal breath sound and the spectrum corresponding to the crackling sound can be estimated in advance by experiments or the like. Therefore, by using the estimated pattern in advance, it is possible to know in what ratio the component corresponding to the normal breath sound and the component corresponding to the crackling sound are included in the above-described spectrum.

[0074] In FIG. 7, for a breath sound signal including a whistling sound (see FIG. 4), when the spectrum is extracted at the timing when the spectrogram pattern corresponding to the whistling sound appears strongly, the results shown in the figure are obtained. This spectrum is considered to include normal breath sounds and whistling sounds.

[0075] In FIG. 8, similar to the normal breath sound and the crackling sound described above, the spectrum corresponding to the whistling sound can also be estimated in advance by experiments or the like. Therefore, by using the estimated pattern in advance, it is possible to know in what ratio the component corresponding to the normal breath sound and the component corresponding to the whistling sound are included in the above-described spectrum.

[0076] Hereinafter, each process for realizing such analysis will be described more specifically.

[0077] <Frequency Analysis> The frequency analysis of the breath sound signal and the detection of peaks in the analysis results will be described in detail with reference to FIGS. 9 to 11. Here, FIG. 9 is a graph showing an example of a frequency analysis method, FIG. 10 is a diagram showing an example of a frequency analysis result, and FIG. 11 is a conceptual diagram showing the peak detection result of the spectrum.

[0078] In FIG. 9, for the acquired breath sound signal, first, frequency analysis is performed. The frequency can be obtained using existing technologies such as fast Fourier transform. In this embodiment, the amplitude value for each frequency (i.e., the amplitude spectrum) is used as the frequency analysis result. Note that the sampling frequency, window size, and window function (e.g., Hanning window, etc.) at the time of data acquisition may be determined as appropriate.

[0079] As shown in FIG. 10, the frequency analysis result is obtained as being composed of n values. Note that "n" is a value determined by the window size, etc. in the frequency analysis.

[0080] In FIG. 11, for the spectrum obtained by frequency analysis, peak detection is performed. In the example shown in the figure, peaks p1 to p4 are detected at positions of 100 Hz, 130 Hz, 180 Hz, and 320 Hz. Note that for the peak detection process, since it only needs to be known at which frequency a peak exists, a simple process may be sufficient. However, it is preferable that the parameters for peak detection are set so that even small peaks are not missed.

[0081] In this embodiment, the points that take the maximum value are obtained, and then the maximum N (N is a predetermined value) are detected in order from those with a small second derivative value (i.e., a large absolute value) of that point. The maximum value is obtained from the point where the sign of the difference switches from positive to negative. The second derivative value is approximated by the difference of the differences. Those with this value smaller than a predetermined threshold (negative value) are selected as the maximum N in order from the smallest, and their positions are memorized.

[0082] <Generation of the basis set> Next, the generation of the basis set will be described in detail with reference to FIGS. 12 to 16. Here, FIG. 12 is a graph showing the normal alveolar breath sound basis. FIG. 13 is a graph showing the crackling sound basis, FIG. 14 is a graph showing the continuous rattling sound basis, FIG. 15 is a graph showing the white noise basis. FIG. 16 is a graph showing the frequency-shifted continuous rattling sound basis.

[0083] As shown in FIGS. 12 to 15, the bases corresponding to each sound type have unique shapes. Each base is composed of the same n numerical values (i.e., amplitude values for each frequency) as the frequency analysis result. Each base is normalized so that the area enclosed by the line indicating the amplitude value for each frequency and the frequency axis becomes a predetermined value (e.g., 1).

[0084] Incidentally, here, four bases, namely the normal alveolar breath sound base, the crackle sound base, the continuous "ra" sound base, and the white noise base, are shown. However, the analysis can be executed even if there is only one base. Also, bases other than those listed here can be used. If a base corresponding to a heart sound or a bowel sound is used instead of the base corresponding to the breath sound listed here, it becomes possible to execute the analysis of the heart sound or the bowel sound.

[0085] In FIG. 16, among the above-described bases, the base corresponding to the continuous "ra" sound is frequency-shifted in accordance with the peak positions detected from the result of the frequency analysis. Here, an example in which the continuous "ra" sound base is frequency-shifted in accordance with each of the peaks p1 to p4 shown in FIG. 11 is shown. Note that bases other than the base corresponding to the continuous "ra" sound may be frequency-shifted.

[0086] As a result of the above, the base set is generated as a set of the normal alveolar breath sound base, the crackle sound base, the continuous "ra" sound bases corresponding to the number of detected peaks, and the white noise base.

[0087] <Calculation of coupling coefficient> Next, the calculation of the coupling coefficient will be described in detail with reference to FIGS. 17 to 19. Here, FIG. 17 is a diagram showing the relationship between the spectrum, the bases, and the coupling coefficient, FIG. 18 is a diagram showing an example of the observed spectrum and the bases used for approximation. FIG. 19 is a diagram showing the approximation result by non-negative matrix factorization.

[0088] The relationship among the spectrum y to be analyzed, the base h(f), and the coupling coefficient u can be expressed by the following mathematical formula (1).

[0089]

Number

[0090] As shown in FIG. 17, the spectrum y and each basis h(f) have n values. On the other hand, the coupling coefficients have m values. Note that "m" is the number of bases included in the basis set.

[0091] In the breath sound analysis apparatus according to this embodiment, non - negative matrix factorization is used to calculate the coupling coefficients of each basis included in the basis set. Specifically, it is only necessary to obtain u (where each component value of u is non - negative) that minimizes the optimization criterion function D represented by the following mathematical formula (2).

[0092]

Number

[0093] Note that general non - negative matrix factorization is a method of calculating both a basis matrix representing a set of basis spectra and an activation matrix representing coupling coefficients. However, in this embodiment, the basis matrix is fixed and only the coupling coefficients are calculated.

[0094] Incidentally, as a means for calculating the coupling coefficients, an approximation method other than non - negative matrix factorization may be used. However, even in this case, the condition of being non - negative is desirable. Below, the reason for using the non - negative approximation method will be explained with specific examples. As shown in FIG. 18, consider the case where the observed spectrum is approximated by four bases, bases A to D, to calculate the coupling coefficients. Note that the expected coupling coefficient u under the condition of being non - negative is 1 for the one corresponding to basis A, 1 for the one corresponding to basis B, 0 for the one corresponding to basis C, and 0 for the one corresponding to basis D. That is, under the condition of being non - negative, the observed spectrum is approximated as the spectrum obtained by adding the one multiplied by 1 for basis A and the one multiplied by 1 for basis B.

[0095] On the other hand, when not conditioned to be non - negative, the expected coupling coefficient u has a value of 0 for the basis corresponding to basis A, 0 for the basis corresponding to basis B, 1 for the basis corresponding to basis C, and - 0.5 for the basis corresponding to basis D. That is, when not conditioned to be non - negative, the observed spectrum is approximated as the spectrum obtained by adding the spectrum multiplied by 1 for basis C and the spectrum multiplied by - 0.5 for basis D.

[0096] When comparing the above two examples, in some cases, a higher approximation accuracy can be obtained when not conditioning to be non - negative than when conditioning to be non - negative. However, since the coupling coefficient u here represents the component amount for each spectrum, it must be obtained as a non - negative value. In other words, when the coupling coefficient u is obtained as a negative value, it cannot be interpreted as a component amount. On the contrary, if approximation is performed with the non - negative condition imposed, the coupling coefficient u corresponding to the component amount can be calculated.

[0097] In FIG. 19, in the biological analysis apparatus according to the present embodiment, as described above, in order to calculate the coupling coefficient u using the basis set consisting of the normal alveolar breath sound basis, the crackling sound basis, the four continuous r - sound bases, and the white noise basis, the coupling coefficient u is calculated as having seven values from u 1 to u 7

[0098] Here, it can be said that the coupling coefficient u 1 corresponding to the normal alveolar breath sound basis represents the ratio of the normal alveolar breath sound to the breath sound. Similarly, the coupling coefficient u 2 corresponding to the crackling sound basis, the coupling coefficient u 3 corresponding to the white noise basis, the coupling coefficient u 4 corresponding to the continuous r - sound basis shifted to 100 Hz, the coupling coefficient u 5 corresponding to the continuous r - sound basis shifted to 130 Hz, the coupling coefficient u 6 corresponding to the continuous r - sound basis shifted to 180 Hz, and the coupling coefficient u 7For each of them, it can be said that it is a value indicating the ratio of each sound type to the breathing sound. Therefore, the signal intensity of each sound type can be calculated from the coupling coefficient u.

[0099] As described above, in this embodiment, a plurality of bases corresponding to each sound type are used to distinguish a plurality of sound types included in the breathing sound. However, the above-described discrimination method is merely an example, and other discrimination methods may be used to distinguish a plurality of sound types.

[0100] <Modification example of the discrimination method> Hereinafter, some examples will be given and described for discrimination methods other than the discrimination method using a plurality of bases already described.

[0101] <First modification example> First, the discrimination method according to the first modification example will be described with reference to FIGS. 20 and 21. Here, FIGS. 20 and 21 are conceptual diagrams showing the discrimination method of the continuous "ra" sound according to the first modification example, respectively.

[0102] In the discrimination method according to the first modification example, the breathing sound is discriminated into a continuous "ra" sound and other sounds. Specifically, when the peak frequency detected from the frequency analysis result of the breathing sound signal varies within a predetermined range, it is determined that it is a continuous "ra" sound.

[0103] As shown in FIG. 20, for the flute sound and the snore-like sound which are continuous "ra" sounds, the positions of the peaks continuously detected on the time axis vary so as to fall within a predetermined range. In other words, the peak frequency changes so as to have temporal continuity. Therefore, when the continuous peak positions are within a predetermined range, it can be determined that the sound is a continuous "ra" sound.

[0104] On the other hand, as shown in FIG. 21, for the sounds other than the continuous "ra" sound, the positions of the peaks continuously detected on the time axis vary so as not to fall within a predetermined range. In other words, the peak frequency changes discretely without having temporal continuity. Therefore, when the continuous peak positions are not within a predetermined range, it can be determined that the sound is not a continuous "ra" sound.

[0105] Note that, for the determination of the continuous "ra" sound, the determination results of multiple times can also be used. Specifically, when the number of times the positions of the peaks continuously detected on the time axis vary within a predetermined range and continues for a predetermined number of times or more, it may be determined that the sound is a continuous "ra" sound.

[0106] <Second Modification Example> Next, a classification method according to the second modification example will be described with reference to FIG. 22. Here, FIG. 22 is a graph showing the threshold values used for the discrimination between the flute sound and the snore-like sound according to the second modification example.

[0107] In the classification method according to the second modification example, the continuous "ra" sound is classified into the flute sound and the snore-like sound. Here, the flute sound and the snore-like sound can be discriminated by the height of the sound (i.e., frequency), such that the flute sound is called a high-pitched continuous "ra" sound and the snore-like sound is called a low-pitched continuous "ra" sound. However, the peak frequencies of the flute sound and the snore-like sound change over time. For this reason, if a single threshold value for the peak frequency (i.e., one threshold value whose value does not fluctuate) is used to determine the flute sound and the snore-like sound, the determination result may change over time. For example, if the peak frequency changes so as to cross the determination threshold value, what has been accurately determined until then will be determined as an incorrect sound type. Therefore, in the second modification example, the determination threshold value is varied according to the peak frequency.

[0108] As shown in FIG. 22, in the classification method according to the second modification example, the threshold value varies such that the ratio of determining the flute sound and the ratio of determining the snore-like sound change smoothly according to the peak frequency. For example, when the peak frequency is 200 Hz, it is determined that 7% is the flute sound and 93% is the snore-like sound. When the peak frequency is 250 Hz, it is determined that 50% is the flute sound and 50% is the snore-like sound. When the peak frequency is 280 Hz, it is determined that 78% is the flute sound and 22% is the snore-like sound. Note that the specific numerical values here are merely examples, and different values may be set. Also, it may have different variation characteristics depending on the gender, age, height, weight, etc. of the living body to be measured.

[0109] By using the above-described variable threshold value, it is possible to suitably prevent misjudgment caused by fluctuations in the peak frequency. That is, in the discrimination method according to the second modification example, since the threshold value for determining the flute sound and the snore-like sound varies to an appropriate value according to the peak frequency, more accurate discrimination can be performed as compared with the case of using a single fixed threshold value, for example.

[0110] <Third Modification Example> Next, the discrimination method according to the third modification example will be described with reference to FIGS. 23 to 25. Here, FIG. 23 is a graph showing the initial value of the threshold value used for discriminating the flute sound and the snore-like sound according to the third modification example. FIGS. 24 and 25 are graphs showing the adjusted values of the threshold value used for discriminating the flute sound and the snore-like sound according to the third modification example, respectively.

[0111] The discrimination method according to the third modification example is also a method of discriminating the continuous "ra" sound into the flute sound and the snore-like sound, similar to the second modification example already described. Also, the point of making a determination using the threshold value for the peak frequency obtained from the frequency analysis result is the same as that of the second modification example.

[0112] As shown in FIG. 23, in the discrimination method according to the third modification example, it is set such that the determination result changes with the threshold value of 250 Hz as a boundary. Specifically, when the peak frequency is 250 Hz or more, it is determined that the continuous "ra" sound contains 100% of the flute sound component and does not contain the snore-like sound. On the other hand, when the peak frequency is less than 250 Hz, it is determined that the continuous "ra" sound contains 100% of the snore-like sound component and does not contain the flute sound.

[0113] As shown in FIG. 24, in the discrimination method according to the third modification example, when it is determined in the previous determination that the sound contains 100% of the flute sound component, the threshold value is lowered from 250 Hz to 220 Hz. Therefore, it becomes easier to be determined as containing 100% of the flute sound component. Specifically, considering the case where the peak frequency is 230 Hz, according to the initial threshold value (see FIG. 23), it would be determined as the snore-like sound, but according to the adjusted threshold value (see FIG. 24), it is determined as the flute sound.

[0114] As shown in FIG. 25, in the classification method according to the third modification, when it is determined in the immediately preceding determination that the snore-like sound component is 100%, the threshold value is increased from 250 Hz to 280 Hz. Therefore, it becomes easier to be determined as a sound containing 100% of the snore-like sound component. Specifically, considering the case where the peak frequency is 270 Hz, according to the initial threshold value (see FIG. 23), it will be determined as a whistling sound, but according to the adjusted threshold value (see FIG. 25), it will be determined as a snore-like sound.

[0115] As described above, by adjusting the threshold value, it is possible to suitably prevent misjudgment caused by fluctuations in the peak frequency. That is, in the classification method according to the third modification, since the threshold values for determining the whistling sound and the snore-like sound are adjusted to appropriate values based on the past determination results, more accurate determination can be performed compared to the case of using a single unadjusted threshold value, for example.

[0116] Note that the adjustment of the threshold value may be performed based on not only the immediately preceding determination result but also a plurality of past determination results. Further, when using a plurality of past determination results, weighting may be performed for each determination result. For example, weighting may be performed so that the influence becomes smaller as the past determination result is older. Also, as the initial value of the threshold value to be adjusted, the smooth threshold value of the second modification may be used (see FIG. 22).

[0117] <Fourth Modification> Next, the classification method according to the third modification will be described with reference to FIGS. 26 to 26. Here, FIG. 26 is a spectrogram of a breath sound including a whistling sound, and FIG. 27 is a graph showing the peak frequency and the number of peaks of the whistling sound. FIG. 28 is a spectrogram of a breath sound including a snore-like sound, and FIG. 29 is a graph showing the peak frequency and the number of peaks of the snore-like sound.

[0118] The classification method according to the fourth modification is also a method of separating the continuous L sound into a whistling sound and a snore-like sound, similar to the second and third modifications already described.

[0119] In FIG. 26, the breath sound including the whistling sound is detected as a spectrum waveform having a predetermined peak. To detect the peak frequency F and the number of peaks N from this, first, a frequency-amplitude graph corresponding to a single time of the spectrum waveform (that is, the region surrounded by the white frame in the figure) is created.

[0120] From the graph shown in FIG. 27, the peak frequency F1 and the number of peaks N1 of the whistling sound can be detected. It is known that the distribution of the peak frequency of the whistling sound is about 180 to 900 Hz. Also, as can be seen from the figure, the number of peaks N1 of the whistling sound is 1.

[0121] In FIG. 28, the breath sound including the snore-like sound is detected as a spectrum waveform having a predetermined peak different from the whistling sound. To detect the peak frequency F and the number of peaks N from this, similarly, a frequency-amplitude graph corresponding to a single time of the spectrum waveform is created.

[0122] From the graph shown in FIG. 29, the peak frequency F2 and the number of peaks N2 of the snore-like sound can be detected. It is known that the distribution of the peak frequency of the snore-like sound is about 100 to 260 Hz. That is, the peak frequency F2 of the snore-like sound is distributed in a region lower than the peak frequency F1 of the whistling sound. Also, as can be seen from the figure, the number of peaks N2 of the snore-like sound is, for example, 3. That is, the number of peaks N2 of the snore-like sound is not 1 like the number of peaks N1 of the whistling sound, but plural.

[0123] In the discrimination method according to the fourth modification, the determination is made using the differences in the characteristics of the whistling sound and the snore-like sound described above. Specifically, based on each of the peak frequency F and the number of peaks N, the whistling sound and the snore-like sound are discriminated. In this way, for example, compared with the case of discriminating the whistling sound and the snore-like sound using only the peak frequency F, more accurate discrimination can be performed.

[0124] <Display of analysis results> Next, the display of the analysis results will be described in detail with reference to FIG. 30. Here, FIG. 30 is a plan view showing an example of display on the display unit.

[0125] As shown in FIG. 30, the analysis results are displayed as a plurality of images in the display area 155 of the display unit 150. Specifically, the waveform of the acquired breath sound is displayed in the area 155a. The spectrum of the acquired breath sound is displayed in the area 155b. The spectrogram of the acquired breath sound is displayed in the area 155c. A graph representing the time-series change in the component amounts of the classified sound types (here, five sound types: normal breath sound, snore-like sound, whistling sound, crackling sound, and bubbling sound) is displayed in the area 155d. The ratio of each classified sound type is displayed as a radar chart in the area 155e.

[0126] Note that such a display mode of the analysis results is merely an example, and the analysis results may be displayed in other display modes. For example, the ratio of each classified sound type may be displayed as a bar graph or a pie chart, or may be displayed numerically.

[0127] <Selection and Output of Sound Types> Next, the selection of sound types by the user and the output for each selected sound type will be described with reference to FIGS. 31 to 33. Here, FIG. 31 is a spectrogram diagram showing the extraction results for each sound type. FIGS. 32 and 33 are conceptual diagrams showing examples of voice output for each classified sound type, respectively.

[0128] As shown in FIG. 31, the spectrogram displayed in the area 155c of the display unit 150 may be displayed for each sound type selected by the user. That is, instead of the spectrogram of the original (acquired original breath sound) shown in FIG. 31(a), the spectrogram of the normal breath sound shown in FIG. 31(b), the spectrogram of the snore-like sound shown in FIG. 31(c), the spectrogram of the whistling sound shown in FIG. 31(d), the spectrogram of the crackling sound shown in FIG. 31(e), and the spectrogram of the bubbling sound shown in FIG. 31(f) may be displayed. Also, a plurality of these spectrograms for each sound type may be arranged and displayed.

[0129] As shown in FIGS. 32 and 33, a graph for each sound type displayed in the area 155d of the display unit 150 may be selected so that only the selected sound type is output as sound volume. For example, in the example of FIG. 32, only the normal breathing sound is selected, and none of the other snoring-like sounds, whistling sounds, hair rubbing sounds, and bubbling sounds are selected. Therefore, only the normal breathing sound is output from the voice output unit 130. Also, in the example of FIG. 33, the normal breathing sound is not selected only, and all of the other snoring-like sounds, whistling sounds, hair rubbing sounds, and bubbling sounds are selected. Therefore, a voice synthesized from the snoring-like sound, whistling sound, hair rubbing sound, and bubbling sound is output from the voice output unit 130.

[0130] <Change of Output Mode> Next, a method for changing the output mode for each sound type will be specifically described with reference to FIGS. 34 to 38. Here, FIG. 34 is a conceptual diagram showing a method for adjusting the volume for each separated sound type, and FIG. 35 is a conceptual diagram showing a method for adjusting the volume for each frequency band. Also, FIG. 36 is a conceptual diagram showing an example of image processing performed for each sound type, and FIG. 37 is a plan view showing an example of an image generated by overlapping images obtained by performing image processing for each sound type. FIG. 38 is a conceptual diagram showing a method for adjusting the display color tone for each separated sound type.

[0131] As shown in FIG. 34, the output volume of each separated sound type may be adjustable for each sound type. In the operation screen shown in the figure, the ON / OFF for each sound type can be switched by a check box, and the volume for each sound type can be adjusted by a slider. In the example shown in the figure, the snoring-like sound and the whistling sound are each output, and the whistling sound is output at a volume larger than that of the snoring-like sound.

[0132] As shown in FIG. 35, in addition to adjusting the output volume for each sound type, it may be possible to adjust the output volume for each frequency band. In the example shown in the figure, the gain can be adjusted in each frequency band of 125 Hz, 250 Hz, 500 Hz, 1 kHz, and 2 kHz.

[0133] As shown in FIG. 36, image processing (for example, binarization, edge detection, etc.) may be performed on the spectrogram extracted for each sound type. By doing so, what was difficult to recognize in the extracted state can be displayed in a more visually understandable state. Note that the image processing may be a combination of a plurality of processes. Also, different image processing may be performed depending on the sound type.

[0134] As shown in FIG. 37, the images for each sound type (see FIG. 36) subjected to image processing may be superimposed and displayed. By doing so, since the spectrograms for each sound type can be recognized as one image, visual grasping can be suitably performed. Here, an example of superimposing and displaying the images of normal breathing sounds and flute sounds is shown, but the sound types to be superimposed and displayed are selectable, and only desired sound types can be appropriately selected and displayed.

[0135] As shown in FIG. 38, the color of the image may be adjustable for each sound type. In the example shown in the figure, the RGB values can be adjusted for each sound type by adjusting the sliders corresponding to R (red), G (green), and B (blue), respectively. By doing so, it becomes possible to display a plurality of sound types in different colors, and a display in a state that is easier to visually grasp can be realized.

[0136] As described above, according to the breathing sound analysis device according to the present embodiment, after separating the breathing sounds, they can be appropriately selected and output. Also, since the output mode can be changed for each sound type, the data for each separated sound type can be suitably used.

[0137] The present invention is not limited to the above-described embodiments, and can be appropriately changed within a range not contrary to the gist or idea of the invention read from the claims and the entire specification. The breathing sound analysis device and breathing sound analysis method accompanied by such changes, as well as the computer program and recording medium, are also included in the technical scope of the present invention.

Explanation of Reference Numerals

[0138] 110 Biosound sensor 120 Signal memory unit 125 Signal processing unit 130 Voice output unit 140 Baseline holding unit 150 Display unit 155 Display area 160 Input unit 200 Processing unit 210 Frequency analysis unit 220 Frequency peak detection unit 230 Basis set generation unit 240 Coupling coefficient calculation unit 250 Signal strength calculation unit 260 Image generation unit 270 Breath sound selection unit y spectrum h(f) basis u coupling coefficient

Claims

1. Classification means for classifying breath sounds into a plurality of sound types, Input means for receiving an input for selecting the type of breath sound to be output, Output means for outputting, as a spectral image, the breath sound of the sound type selected according to the input received by the input means and corresponding to a plurality of reference spectra that serve as criteria for classifying the breath sound, Changing means for changing the color of the spectral image for each sound type corresponding to the plurality of reference spectra, the sound type being selected according to the input received by the input means, Comprising The classification means Acquisition means for acquiring information regarding a frequency corresponding to a predetermined feature of the spectrum of the breath sound, Shifting means for shifting the plurality of reference spectra according to the information regarding the frequency to obtain frequency-shifted reference spectra, And ratio output means for outputting the ratio of the plurality of reference spectra included in the breath sound based on the breath sound and the frequency-shifted reference spectra A breath sound analysis apparatus characterized by the above.

2. Acquire information regarding a frequency corresponding to a predetermined feature of the spectrum of the breath sound, Shift a plurality of reference spectra that serve as criteria for classifying the breath sound according to the information regarding the frequency to obtain frequency-shifted reference spectra, Output the ratio of the plurality of reference spectra included in the breath sound based on the breath sound and the frequency-shifted reference spectra Thereby, a classification step of classifying the breath sound into a plurality of sound types, An input step of receiving an input for selecting the type of breath sound to be output, An output step of outputting, as a spectral image, the breath sound of the sound type selected according to the input received in the input step and corresponding to the plurality of reference spectra, A changing step of changing the color of the spectral image for each sound type corresponding to the plurality of reference spectra, the sound type being selected according to the input received in the input step, A computer program characterized by causing a computer to execute the above.

Images