Apparatus and method for determining state of pulmonary congestion by using artificial intelligence analysis

Abstract

Disclosed are an apparatus and a method for determining the state of pulmonary congestion by using artificial intelligence analysis. The apparatus for determining the state of pulmonary congestion by using artificial intelligence analysis, according to an embodiment, may comprise: a data reception unit that receives speech data from a user; a data preprocessing unit that extracts, from the received speech data, partial speech data corresponding to a portion of the speech data by using windowing processing, and converts the extracted partial speech data into a spectrogram; and a pulmonary congestion state determination unit that outputs, by using the spectrogram and a neural network-based pulmonary congestion state determination model, determination data including pulmonary congestion state information about the user.

Description

Device and method for identifying pulmonary congestion using artificial intelligence analysis

[0001] The following disclosure relates to a device and method for identifying pulmonary congestion using artificial intelligence analysis.

[0002] Techniques to differentiate pulmonary congestion due to heart failure include blood biomarkers (e.g., BNP, NT-proBNP, ST2), lung imaging (e.g., chest X-ray, chest CT), and CadioMEMS, which monitors pulmonary artery pressure via a sensor implanted in the pulmonary artery. Fluid overload due to acute heart failure can cause swelling in the lungs, larynx, and articulatory organs (e.g., vocal cords, tongue, palate, mouth muscles), which are the main organs responsible for producing voice through airflow.

[0003] A device for determining a pulmonary congestion condition using artificial intelligence analysis according to one embodiment may include a data receiving unit that receives a user's voice data, a data preprocessing unit that extracts partial voice data corresponding to a part of the voice data from the received voice data through windowing processing, and converts the extracted partial voice data into a spectrogram, and a pulmonary congestion condition determination unit that outputs determination data including information on the user's pulmonary congestion condition using a neural network-based pulmonary congestion condition determination model and the spectrogram.

[0004] The above pulmonary congestion condition discrimination model may be a densenet including a convolution layer, a dense layer, a transition layer, and a classification layer.

[0005] The above pulmonary congestion condition discrimination model may be a dense net, and a hidden feature may be extracted through a skip connection of the dense net, and the pulmonary congestion condition information may be determined using the extracted hidden feature.

[0006] The output of the above pulmonary congestion condition discrimination model may include at least one of a probability value corresponding to a state in which the user's pulmonary congestion has improved and a probability value corresponding to a state in which the user's pulmonary congestion has not improved.

[0007] The above pulmonary congestion condition discrimination model is a vision transformer, and the above pulmonary congestion condition discrimination model may include a projection unit that transforms the dimension of the feature vector of the spectrogram, a transformer encoder, and a classification layer.

[0008] The above data preprocessing unit divides the spectrogram into a plurality of partial spectrograms, and at least one of the divided partial spectrograms can be input to the vision transformer.

[0009] The above vision transformer can perform vector projection on the segmented partial spectrogram, calculate attention of the vector-projected partial spectrogram, and determine pulmonary congestion status information based on the calculated attention.

[0010] The user's voice data may be voice data of a patient suffering from pulmonary congestion due to acute heart failure.

[0011] The above spectrogram may be a mel-spectrogram, which is a graph in which the user's voice data is converted into frequency information corresponding to the voice data by Fourier transform and the scale of the frequency information is converted into a mel-scale.

[0012] In the above pulmonary congestion condition discrimination model, the spectrogram may be input, and at least one of the user information and the user's blood test information may be further input.

[0013] A method for determining a pulmonary congestion state performed by a device for determining a pulmonary congestion state according to one embodiment may include an operation of receiving a user's voice data through a data receiving unit, an operation of extracting partial voice data corresponding to a part of the voice data from the received voice data through windowing processing, an operation of converting the extracted partial voice data into a spectrogram, and an operation of obtaining determination data including information on the user's pulmonary congestion state using a pulmonary congestion state determination model based on a neural network and the spectrogram.

[0014] The method for determining a pulmonary congestion condition may further include an operation of dividing the spectrogram into a plurality of partial spectrograms and inputting at least one of the divided partial spectrograms into the vision transformer.

[0015] FIG. 1 is a block diagram illustrating the configurations of a pulmonary congestion condition detection device according to one embodiment.

[0016] FIG. 2 is a diagram for explaining preprocessing of voice data through a data preprocessing unit according to one embodiment.

[0017] FIG. 3 is a diagram for explaining a pulmonary congestion condition discrimination model according to one embodiment.

[0018] FIG. 4 is a diagram for explaining a pulmonary congestion condition discrimination model according to one embodiment.

[0019] FIG. 5 is a flowchart illustrating operations of a method for identifying a pulmonary congestion condition according to one embodiment.

[0020] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Therefore, the actual implementation is not limited to the specific embodiments disclosed, and the scope of this specification includes modifications, equivalents, or alternatives within the technical concepts described in the embodiments.

[0021] Although terms such as "first" or "second" may be used to describe various components, these terms should be interpreted solely to distinguish one component from another. For example, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

[0022] When it is said that a component is "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but there may also be other components in between.

[0023] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this document, phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can each include any one of the items listed together in that phrase, or all possible combinations thereof. In this specification, it should be understood that the terms "comprises" or "has" and the like are intended to specify the presence of a described feature, number, step, operation, component, part, or combination thereof, but do not exclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0024] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art. Terms defined in commonly used dictionaries should be interpreted to have a meaning consistent with their meaning in the context of the relevant technology, and will not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0025] The term "~part" as used in this document refers to a software or hardware component such as an FPGA or ASIC, and the "~part" performs certain roles. However, the "~part" is not limited to software or hardware. The "~part" may be configured to reside on an addressable storage medium and may be configured to execute one or more processors. For example, the "~part" may include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided within the components and "~parts" may be combined into a smaller number of components and "~parts" or further separated into additional components and "~parts." Furthermore, the components and "~parts" may be implemented to execute one or more CPUs within a device or a secure multimedia card. Additionally, '~bu' may include one or more processors.

[0026] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are assigned the same reference numerals regardless of the drawing numbers, and redundant descriptions thereof will be omitted.

[0027]

[0028] FIG. 1 is a block diagram illustrating the configurations of a pulmonary congestion condition detection device according to one embodiment.

[0029] Referring to FIG. 1, the pulmonary congestion status identification device (100) is a device that identifies a user's pulmonary congestion status using artificial intelligence analysis. Pulmonary congestion indicates an increase in blood volume in the pulmonary blood vessels and congestion in the pulmonary circulation. The pulmonary congestion status identification device (100) according to one embodiment may include a data receiving unit (110), a data preprocessing unit (120), and a pulmonary congestion status identification unit (130).

[0030] The data receiving unit (110) can receive the user's voice data. The user's voice data received by the data receiving unit (110) may be voice data of a patient suffering from pulmonary congestion due to acute heart failure. For example, the user's voice data may be the patient's pronunciation of "Long Live the Republic of Korea," "Ah," "Eh," "I," "Oh," and "U." Pulmonary congestion due to acute heart failure indicates pulmonary congestion caused by structural or functional abnormalities of the heart.

[0031] The data preprocessing unit (120) can preprocess the user's voice data. In one embodiment, the data preprocessing unit (120) can extract partial voice data corresponding to a portion of the voice data received through windowing processing. The data preprocessing unit (120) can convert the extracted partial voice data into a spectrogram. A spectrogram is a graph that visualizes the sound spectrum. The preprocessing of the user's voice data by the data preprocessing unit (120) is described in detail in FIG. 2.

[0032] The pulmonary congestion state discrimination unit (130) can output discrimination data including pulmonary congestion state information. For example, the pulmonary congestion state discrimination unit (130) can output discrimination data including the user's pulmonary congestion state information using a pulmonary congestion state discrimination model based on a neural network and a spectrogram. The pulmonary congestion state discrimination unit (130) outputting discrimination data using the pulmonary congestion state discrimination model based on a neural network will be described in detail with reference to FIGS. 3 and 4.

[0033] According to one embodiment, a pulmonary congestion status identification device (100) can distinguish the pulmonary congestion status of a heart failure patient through a pulmonary congestion identification model. The pulmonary congestion status identification device (100) can identify and monitor the pulmonary congestion status by utilizing a negative biomarker for the pulmonary congestion status of a heart failure patient. Unlike conventional pulmonary congestion diagnosis and monitoring technologies, the pulmonary congestion status identification device (100) does not require blood analysis or photography using x-rays. Unlike the conventional CadioMEMS method, the pulmonary congestion status identification device (100) does not require a visit to a hospital and does not require continuous monitoring and management by medical staff. Therefore, the pulmonary congestion status identification device (100) has superior advantages in terms of cost, convenience, safety, and management compared to the CadioMEMS method.

[0034] Pulmonary congestion due to heart failure can cause respiratory distress and lead to hospitalization or death. Furthermore, pulmonary congestion can increase medical expenses and reduce the patient's quality of life. The device (100) for assessing pulmonary congestion according to the embodiments can improve the patient's quality of life by reducing mortality, readmission rates, and medical expenses due to heart failure. Furthermore, the device (100) for assessing pulmonary congestion is superior to existing technologies in that it can diagnose and monitor pulmonary congestion due to heart failure at low cost by noninvasively recording one's own voice anywhere.

[0035]

[0036] FIG. 2 is a diagram for explaining how a data preprocessing unit according to one embodiment preprocesses voice data.

[0037] Referring to FIG. 2, the data preprocessing unit (120) can preprocess voice data (210) to extract partial voice data (220) corresponding to a portion (211) of the voice data. The data preprocessing unit (120) according to one embodiment can output partial voice data (220) through windowing processing. For example, the data preprocessing unit (120) can apply a window having an interval of 1 second (or N (a natural number greater than or equal to 2) seconds) to the voice data (210) to extract partial voice data (220) corresponding to the area of ​​the window.

[0038] According to one embodiment, the data preprocessing unit (120) can convert partial speech data (220) into data in the frequency domain. For example, the data preprocessing unit (120) can convert the partial speech data (220) into frequency information by performing a Fourier transform (230) on the partial speech data (220).

[0039] The data preprocessing unit (120) can perform mel-scale transformation (240) on the frequency information of partial voice data converted into frequency domain data to generate a mel-spectrogram (250). The mel-scale is a scale that reflects the standard by which a person perceives sound, and the mel-spectrogram (250) is a spectrogram that includes the frequency information of the mel-scale.

[0040] In one embodiment, a Mel-Spectrogram (250) preprocessed by a data preprocessing unit (120) may be input to a pulmonary congestion condition discrimination model. According to one embodiment, the pulmonary congestion condition discrimination model may include a ResNet (e.g., resnet-50, resneXt), a DenseNet (e.g., densenet) (310), a ConvNet (e.g., ConvnetXt), an EfficientNet (e.g., efficientnet-b7), and / or a vision transformer (ViT).

[0041] A pulmonary congestion state discrimination model according to one embodiment can output a binary classification value (e.g., a binary classification value (320) of FIG. 3, a binary classification value (440) of FIG. 4) based on an input mel-spectrogram (250). For example, the pulmonary congestion state discrimination model can output a binary classification value including at least one of a probability value corresponding to an improved state of the user's pulmonary congestion and a probability value corresponding to an unimproved state of the user's pulmonary congestion by using a preprocessed mel-spectrogram (250).

[0042] According to one embodiment, a pulmonary congestion status identification model may further input at least one of user information and blood test information of the user in addition to a spectrogram (e.g., Mel-spectrogram (250)). The user information may be personal information about a user using a pulmonary congestion status identification device, and may include, for example, information on age, gender, medical history, and / or lifestyle habits. The blood test information may be medical information about the user obtained by testing the user's blood, and may include, for example, information on BNP (B-type natriuretic peptide), NT-proBNP (N-terminal pro-BNT), and / or ST2 (growth stimulation expressed gene 2).

[0043]

[0044] FIG. 3 is a diagram illustrating how a density net according to one embodiment outputs a binary classification value.

[0045] Referring to FIG. 3, a pulmonary congestion condition discrimination device (e.g., a pulmonary congestion condition discrimination device (100) of FIG. 1) can discriminate a pulmonary congestion condition using a dense network (310) as a pulmonary congestion condition discrimination model. The dense network (310) may include a plurality of layers. The dense network (310) may include, for example, a convolution layer (311), dense layers (312, 314, 316, 318), transition layers (313, 315, 317), and a classification layer (319). Each of the convolution layer, dense layer, transition layer, and classification layer included in the dense network (310) may be one or more.

[0046] A convolution layer (311) according to one embodiment can extract a feature vector of input data. For example, the convolution layer (311) can extract a feature vector from an input mel-spectrogram (e.g., the mel-spectrogram (250) of FIG. 2).

[0047] According to one embodiment, dense layers (312, 314, 316, 318) represent densely connected layers. For example, features extracted from a convolutional layer (311) can be input to transition layers (313, 315, 317) through dense layers (312, 314, 316, 318). DenseNet (310) according to one embodiment can extract features related to pulmonary congestion status information through skip connections. For example, DenseNet (310) can extract hidden features through skip connections between layers. Skip connections refer to a method in which an input layer skips at least one of the intermediate layers and is directly connected to the output layer.

[0048] According to one embodiment, the transition layers (313, 315, 317) can reduce the size of a feature map. For example, the transition layers (313, 315, 317) can reduce the size of a feature map of features extracted through a dense layer.

[0049] According to one embodiment, a classification layer (319) may output pulmonary congestion status information based on the extracted features. The pulmonary congestion status information may include a binary classification value (320) including at least one of a probability value corresponding to an improved pulmonary congestion state and a probability value corresponding to an unimproved pulmonary congestion state of the user.

[0050]

[0051] FIG. 4 is a diagram illustrating a vision transformer according to one embodiment outputting a binary classification value.

[0052] Referring to FIG. 4, a pulmonary congestion condition discrimination device (e.g., the pulmonary congestion condition discrimination device (100) of FIG. 1) can discriminate a pulmonary congestion condition using a vision transformer (430) as a pulmonary congestion condition discrimination model. The vision transformer (430) according to one embodiment may include a projection unit (431) that converts the dimension of a feature vector of a partial spectrogram, a transformer encoder (432), and a classification layer (433). Each of the projection unit (431), the transformer encoder (432), and the classification layer (433) included in the vision transformer (430) may be one or more.

[0053] A data preprocessing unit (e.g., data preprocessing unit (120) of FIG. 1) according to one embodiment may split a mel-spectrogram (250) into a plurality of partial spectrograms (420) (or patches) through cropping (410). The data preprocessing unit may input at least one of the split partial spectrograms (420) to a vision transformer (430).

[0054] According to one embodiment, the projection unit (431) can perform vector projection to transform the dimension of the feature vectors of the partial spectrograms (420). For example, the projection unit (431) can transform the dimension of the feature vectors of the partial spectrograms (420) into a dimension that can be processed by the transformer encoder (432).

[0055] A transformer encoder (432) according to one embodiment can calculate attention of vector-projected partial spectrograms. The transformer encoder (432) can perform normalization and multi-head attention on feature vectors of vector-projected partial spectrograms and calculate attention by passing them through a fully connected layer.

[0056] According to one embodiment, the classification layer (433) may output pulmonary congestion status information based on the calculated attention. The pulmonary congestion status information output by the classification layer (433) may include a binary classification value (440) including at least one of a probability value corresponding to an improved pulmonary congestion state and a probability value corresponding to an unimproved pulmonary congestion state of the user.

[0057] In one embodiment, a pulmonary congestion discrimination model can be trained using a machine learning process. The machine learning process enables the pulmonary congestion discrimination model to recognize input data patterns and make predictions using given data. This machine learning process may include the following steps: (1) data preparation, (2) model initialization, (3) forward computation, (4) loss calculation, (5) backpropagation, and (6) parameter update. The data preparation process collects and preprocesses the training data from which the pulmonary congestion discrimination model will learn. The preprocessing process refines the training data and, if necessary, performs standardization, normalization, and feature selection to create a format suitable for the pulmonary congestion discrimination model. The model initialization process sets the initial parameters of the pulmonary congestion discrimination model. For example, if the pulmonary congestion discrimination model is a neural network, this may include initializing the weights and biases. The forward computation process inputs the prepared training data into the pulmonary congestion discrimination model and calculates a predicted value for estimating the pulmonary congestion status. During this process, the pulmonary congestion discrimination model processes the training data using its current parameters and generates a predicted value as an output. The loss calculation process calculates the difference between the predicted value of the pulmonary congestion discrimination model and the actual correct answer (label) using a loss function. The loss function is a function that allows us to calculate a value that indicates how accurate (or inaccurate) the prediction of the pulmonary congestion discrimination model is. The backpropagation process is the process of adjusting the parameters of the pulmonary congestion discrimination model to reduce the loss derived from the loss function. The backpropagation algorithm differentiates the value of the loss function (gradient) to calculate how much each parameter of the pulmonary congestion discrimination model contributed to the loss, and the parameters of the pulmonary congestion discrimination model can be updated based on this value. The parameter update process is the process of updating the parameters of the pulmonary congestion discrimination model using the calculated gradient.Parameter updates typically use gradient descent or its variants. Through this process, the pulmonary congestion classification model can learn to make increasingly accurate predictions. The above processes (forward computation, loss calculation, backpropagation, parameter update) can be repeated multiple times on a large number of training data sets, and training can proceed multiple times until the pulmonary congestion classification model is sufficiently trained. Through this learning process, the pulmonary congestion classification model can learn patterns from the given data and gain the ability to make predictions on new data.

[0058]

[0059] Figure 5 is a flowchart illustrating the operations of a method for determining a pulmonary congestion condition according to one embodiment. The operations of the method for determining a pulmonary congestion condition may be performed by a pulmonary congestion condition determination device.

[0060] In operation (510), a pulmonary congestion condition identification device (e.g., a pulmonary congestion condition identification device (100) of FIG. 1) can receive a user's voice data. The pulmonary congestion condition identification device can receive the user's voice data through a data receiving unit (e.g., a data receiving unit (110) of FIG. 1) of FIG. 1). For example, the pulmonary congestion condition identification device can receive voice data of a patient suffering from pulmonary congestion due to acute heart failure.

[0061] In operation (520), the pulmonary congestion condition identification device can extract partial speech data through windowing processing. For example, the pulmonary congestion condition identification device can extract partial speech data corresponding to a portion of the speech data from the speech data through windowing processing in units of 1 second.

[0062] In operation (530), the pulmonary congestion condition detection device may convert partial voice data into a spectrogram. For example, the spectrogram may be a Mel-spectrogram, which is a graph in which the user's voice data is converted into frequency information corresponding to the voice data through a Fourier transform and the scale of the frequency information is converted into a Mel-scale.

[0063] In operation (540), the pulmonary congestion state identification device can obtain identification data using a pulmonary congestion state identification model and a spectrogram. The pulmonary congestion state identification device can obtain identification data including information on the user's pulmonary congestion state using a neural network-based pulmonary congestion state identification model and a spectrogram. The spectrogram is input into the pulmonary congestion state identification model, and at least one of user information and blood test information of the user can be further input. The user information is personal information about a user using the pulmonary congestion state identification device, and may include, for example, age, gender, medical history, and lifestyle information. The blood test information is medical information about the user obtained by testing the user's blood, and may include, for example, BNP, NT-proBNP, and ST2.

[0064] The pulmonary congestion detection model may include, for example, a resnet (e.g., resnet-50, resneXt), a densenet (e.g., densenet) (310), a convnet (e.g., ConvnetXt), an efficientnet (e.g., efficientnet-b7), and / or a vision transformer (ViT).

[0065] According to one embodiment, a dense network (e.g., a dense network (310) of FIG. 3) may be a dense network including a convolution layer, a dense network, a transition layer, and a classification layer. The pulmonary congestion detection device may extract hidden features through skip connections of the dense network and determine pulmonary congestion status information using the extracted hidden features. The pulmonary congestion status information output by the dense network according to one embodiment may include at least one of a probability value corresponding to an improved state of the user's pulmonary congestion and a probability value corresponding to an unimproved state of the user's pulmonary congestion. Through the pulmonary congestion status information, it may be possible to estimate whether the user's pulmonary congestion has improved and by how much it has improved from the user's voice data.

[0066] A model for identifying a pulmonary congestion condition according to one embodiment may be a vision transformer (e.g., vision transformer (430) of FIG. 5). The vision transformer may include a projection unit that transforms the dimension of a feature vector of a spectrogram, a transformer encoder, and a classification layer.

[0067] A device for identifying a pulmonary congestion condition may segment a spectrogram and input it to a vision transformer. For example, the device for identifying a pulmonary congestion condition may segment a spectrogram into a plurality of partial spectrograms (or patches) and input at least one of the segmented partial spectrograms to the vision transformer.

[0068] According to one embodiment, a device for determining a pulmonary congestion condition may perform vector projection on a segmented partial spectrogram through a vision transformer, calculate attention of the vector-projected partial spectrogram, and determine pulmonary congestion condition information based on the calculated attention. The pulmonary congestion condition information determined by the vision transformer may include at least one of a probability value corresponding to an improved condition and a probability value corresponding to a state in which the user's pulmonary congestion has not improved. Through the pulmonary congestion condition information, it may be possible to estimate whether the user's pulmonary congestion has improved and to what extent the improvement has been made from the user's voice data.

[0069]

[0070] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination. The program commands recorded on the medium may be those specially designed and configured for the embodiment or may be those known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of the program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiment, and vice versa.

[0071] Software may include computer programs, codes, instructions, or a combination of one or more of these, and may configure a processing device to perform a desired operation or, independently or collectively, command the processing device. The software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium, or device for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over networked computer systems and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

[0072] The term "module" as used herein may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or portion of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0073] The term "~part" as used in this document refers to a software or hardware component such as an FPGA or ASIC, and the "~part" performs certain roles. However, the "~part" is not limited to software or hardware. The "~part" may be configured to reside on an addressable storage medium and may be configured to execute one or more processors. For example, the "~part" may include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided within the components and "~parts" may be combined into a smaller number of components and "~parts" or further separated into additional components and "~parts." Furthermore, the components and "~parts" may be implemented to execute one or more CPUs within a device or a secure multimedia card. Additionally, '~bu' may include one or more processors.

[0074] Although the embodiments described above have been described with limited drawings, those skilled in the art will appreciate that various technical modifications and variations can be applied based on the described embodiments. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and / or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents.

[0075] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims described below.

Claims

1. In a device for identifying pulmonary congestion using artificial intelligence analysis, A data receiving unit that receives the user's voice data; A data preprocessing unit that extracts partial voice data corresponding to a part of the voice data from the received voice data through windowing processing and converts the extracted partial voice data into a spectrogram; and A pulmonary congestion condition discrimination model based on a neural network and a pulmonary congestion condition discrimination unit that outputs discrimination data including information on the user's pulmonary congestion condition using the spectrogram. A device for differentiating pulmonary congestion conditions, including:

2. In paragraph 1, The above pulmonary congestion condition discrimination model is, A densenet that includes a convolution layer, a dense layer, a transition layer, and a classification layer. Device for differentiating pulmonary congestion.

3. In paragraph 2, The above pulmonary congestion condition discrimination model is, It's Densnet, Extracting hidden features through the skip-connection of the above-mentioned density net, and determining the pulmonary congestion status information using the extracted hidden features. Device for differentiating pulmonary congestion.

4. In paragraph 1, The output of the above pulmonary congestion condition discrimination model is: Including at least one of a probability value corresponding to a state in which the user's pulmonary congestion is improved and a probability value corresponding to a state in which the user's pulmonary congestion is not improved. Device for differentiating pulmonary congestion.

5. In paragraph 1, The above pulmonary congestion condition discrimination model is a vision transformer, The above pulmonary congestion condition discrimination model is, A projection unit that converts the dimension of the feature vector of the above spectrogram, a transformer encoder, and a classification layer, Device for differentiating pulmonary congestion.

6. In paragraph 5, The above data preprocessing unit, Divide the above spectrogram into multiple partial spectrograms, Inputting at least one of the above segmented partial spectrograms into the vision transformer, Device for differentiating pulmonary congestion.

7. In paragraph 5, The above vision transformer, Performing vector projection on the above-described segmented partial spectrogram, calculating attention of the vector-projected partial spectrogram, and determining pulmonary congestion status information based on the calculated attention. Device for differentiating pulmonary congestion.

8. In paragraph 1, The user's voice data is, This is voice data from a patient with pulmonary congestion due to acute heart failure. Device for differentiating pulmonary congestion.

9. In paragraph 1, The above spectrogram is, The user's voice data is converted into frequency information corresponding to the voice data by Fourier transform, and the scale of the frequency information is converted into a mel-scale, which is a mel-spectrogram represented as a graph. Device for differentiating pulmonary congestion.

10. In paragraph 1, In the above pulmonary congestion condition discrimination model, The above spectrogram is input, At least one of the user information of the user and the user's blood test information is further input. Device for differentiating pulmonary congestion.

11. In a method for identifying a pulmonary congestion condition performed by a pulmonary congestion condition identification device, An action of receiving a user's voice data through a data receiving unit; An operation of extracting partial voice data corresponding to a part of the voice data from the received voice data through windowing processing and converting the extracted partial voice data into a spectrogram; and An operation of obtaining discrimination data including information on the user's pulmonary congestion status using a neural network-based pulmonary congestion status discrimination model and the spectrogram. A method for differentiating pulmonary congestion including:

12. In paragraph 11, The above pulmonary congestion condition discrimination model is, It includes convolutional layers, dense layers, transition layers, and classification layers. How to differentiate pulmonary congestion.

13. In paragraph 12, The above pulmonary congestion condition discrimination model is, It's Densnet, Extracting hidden features through the skip connection of the above-mentioned density net, and determining the pulmonary congestion status information using the extracted hidden features. How to differentiate pulmonary congestion.

14. In paragraph 11, The output of the above pulmonary congestion condition discrimination model is: Including at least one of a probability value corresponding to a state in which the user's pulmonary congestion is improved and a probability value corresponding to a state in which the user's pulmonary congestion is not improved. How to differentiate pulmonary congestion.

15. In paragraph 11, The above pulmonary congestion condition discrimination model is a vision transformer, The above pulmonary congestion condition discrimination model is, It includes a projection unit that transforms the dimension of the feature vector of the above spectrogram, a transformer encoder, and a classification layer. How to differentiate pulmonary congestion.

16. In paragraph 15, The spectrogram is divided into a plurality of partial spectrograms, and at least one of the divided partial spectrograms is input to the vision transformer. How to differentiate pulmonary congestion.

17. In paragraph 15, The above vision transformer, Performing vector projection on the above-described segmented partial spectrogram, calculating attention of the vector-projected partial spectrogram, and determining pulmonary congestion status information based on the calculated attention. How to differentiate pulmonary congestion.

18. In paragraph 11, The user's voice data is, This is voice data from a patient with pulmonary congestion due to acute heart failure. How to differentiate pulmonary congestion.

19. In paragraph 11, The above spectrogram is, The user's voice data is converted into frequency information corresponding to the voice data by Fourier transform, and the scale of the frequency information is converted into a mel-scale graph, which is a mel-spectrogram. How to differentiate pulmonary congestion.

20. In paragraph 11, In the above pulmonary congestion condition discrimination model, The above spectrogram is input, At least one of the user information of the user and the user's blood test information is further input. How to differentiate pulmonary congestion.

Images