Apparatus and method for distinguishing lung congestion condition using artificial intelligence analysis

Abstract

An apparatus and method for distinguishing a pulmonary congestion state using artificial intelligence analysis are disclosed. An apparatus for distinguishing a pulmonary congestion state using artificial intelligence analysis according to one embodiment may include a data receiving unit for receiving voice data of a user, a data preprocessing unit for extracting partial voice data corresponding to a part of the voice data from the received voice data through windowing processing and converting the extracted partial voice data into a spectrogram, and a pulmonary congestion state distinguishing unit for outputting distinguishing data including information on the user's pulmonary congestion state using a neural network-based pulmonary congestion state distinguishing model and a spectrogram.

Description

Technology Field

[0001] The following disclosure relates to an apparatus and method for differentiating pulmonary congestion using artificial intelligence analysis. Background Technology

[0003] Generally, techniques for differentiating pulmonary congestion caused by heart failure include blood biomarkers (e.g., BNP, NT-proBNP, ST2), lung imaging (e.g., chest X-ray, chest CT), and CadioMEMS, which monitors pulmonary artery pressure through sensors implanted in the pulmonary artery. Fluid overload caused by acute heart failure can cause edema in the lungs, larynx, and articulatory organs (e.g., vocal cords, tongue, palate, mouth muscles), which are the primary organs that produce voice through airflow. However, conventional techniques such as those mentioned above often relied on chest X-rays, ultrasounds, blood tests, or invasive sensors to diagnose or assess the course of pulmonary congestion, and these methods had limitations, such as difficulty in repeated measurements or the need for medical resources. Meanwhile, although disease diagnosis techniques using voice data have been proposed in the past, they were primarily focused on determining the presence of psychiatric or neurological disorders or classifying voice and swallowing disorders themselves, and thus had the problem of being difficult to apply to distinguishing temporal changes, such as the improvement or worsening of pulmonary congestion caused by heart failure.

[0004] delete Prior art literature

[65535] Republic of Korea Registered Patent No. 10-2891208, "Self-Lung Function Test System Using Computer Vision Technology" means of solving the problem

[0005] A device for distinguishing a pulmonary congestion state using artificial intelligence analysis according to one embodiment of the present invention may include a data receiving unit for receiving voice data of a user, a data preprocessing unit for extracting partial voice data corresponding to a part of the voice data from the received voice data through windowing processing and converting the extracted partial voice data into a spectrogram, a neural network-based pulmonary congestion state discrimination model, and a pulmonary congestion state discrimination unit for outputting discrimination data including information on the pulmonary congestion state of the user using the spectrogram.

[0006] The above-mentioned pulmonary congestion differential model may be a dense net comprising a convolution layer, a dense layer, a transition layer, and a classify layer.

[0007] The above-mentioned pulmonary congestion state differentiation model may be a dense net, and hidden features may be extracted through skip connections of the dense net, and the pulmonary congestion state information may be determined using the extracted hidden features.

[0008] The output of the above-described pulmonary congestion state differentiation model may include at least one of a probability value corresponding to a state in which the user's pulmonary congestion is improved and a probability value corresponding to a state in which the user's pulmonary congestion is not improved.

[0009] The above-mentioned pulmonary congestion state differentiation model is a vision transformer, and the above-mentioned pulmonary congestion state differentiation model may include a projection unit that transforms the dimension of the feature vector of the spectrogram, a transformer encoder, and a classify layer.

[0010] The data preprocessing unit divides the spectrogram into a plurality of partial spectrograms, and at least one of the divided partial spectrograms can be input to the vision transformer.

[0011] The vision transformer above can perform a vector projection on the segmented partial spectrogram, calculate the attention of the vector-projected partial spectrogram, and determine pulmonary congestion state information based on the calculated attention.

[0012] The user's voice data may be voice data from a patient with pulmonary congestion caused by acute heart failure.

[0013] The spectrogram above may be a Mel-spectrogram in which the user's voice data is converted into frequency information corresponding to the voice data by a Fourier transform, and the scale of the frequency information is converted into a Mel-scale graph.

[0014] In the above-mentioned pulmonary congestion condition differentiation model, the above-mentioned spectrogram is input, and at least one of the user's user information and the user's blood test information may be further input.

[0015] A method for determining a pulmonary congestion state performed by a device for determining a pulmonary congestion state according to one embodiment may include: receiving voice data of a user through a data receiving unit; extracting partial voice data corresponding to a part of the voice data from the received voice data through windowing processing; converting the extracted partial voice data into a spectrogram; and obtaining determination data including information on the user's pulmonary congestion state using a neural network-based pulmonary congestion state determination model and the spectrogram.

[0016] The above method for distinguishing a pulmonary congestion state may further include the operation of dividing the spectrogram into a plurality of partial spectrograms and inputting at least one of the divided partial spectrograms into the vision transformer. Brief explanation of the drawing

[0018] FIG. 1 is a block diagram illustrating the configurations of a device for distinguishing a pulmonary congestion state according to one embodiment. FIG. 2 is a diagram illustrating voice data preprocessing through a data preprocessing unit according to one embodiment. FIG. 3 is a diagram illustrating a model for differentiating pulmonary congestion according to one embodiment. FIG. 4 is a diagram illustrating a model for differentiating pulmonary congestion according to one embodiment. FIG. 5 is a flowchart illustrating the operations of a method for distinguishing a pulmonary congestion state according to one embodiment. Specific details for implementing the invention

[0019] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, actual implementations are not limited to the specific embodiments disclosed, and the scope of this specification includes modifications, equivalents, or substitutions included in the technical concept described by the embodiments.

[0020] Terms such as "first" or "second" may be used to describe various components, but these terms should be interpreted solely for the purpose of distinguishing one component from another. For example, the first component may be named the second component, and similarly, the second component may be named the first component.

[0021] When it is stated that a component is "connected" to another component, it should be understood that it may be directly connected to or coupled with that other component, or that there may be other components in between.

[0022] Singular expressions include plural expressions unless the context clearly indicates otherwise. In this document, phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B or C,” “at least one of A, B and C,” and “at least one of A, B, or C” may each include any one of the items listed together with the corresponding phrase, or all possible combinations thereof. In this specification, terms such as “comprising” or “having” are intended to designate the existence of the described features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0023] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0024] As used in this document, the term "part" refers to a software or hardware component, such as an FPGA or ASIC, that performs certain roles. However, "part" is not limited to software or hardware. "Part" may be configured to reside in an addressable storage medium or configured to operate one or more processors. For example, "part" may include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and "parts" may be combined into a smaller number of components and "parts" or further separated into additional components and "parts." Furthermore, components and "parts" may be implemented to operate one or more CPUs within a device or secure multimedia card. Additionally, '~part' may include one or more processors.

[0025] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are given the same reference numeral regardless of the drawing number, and redundant descriptions thereof will be omitted.

[0027] FIG. 1 is a block diagram illustrating the configurations of a device for distinguishing a pulmonary congestion state according to one embodiment.

[0028] Referring to FIG. 1, the pulmonary congestion state detection device (100) is a device that detects a user's pulmonary congestion state using artificial intelligence analysis. Pulmonary congestion indicates an increase in the blood volume within the pulmonary blood vessels and congestion in the pulmonary circulatory system. A pulmonary congestion state detection device (100) according to one embodiment may include a data receiving unit (110), a data preprocessing unit (120), and a pulmonary congestion state detection unit (130).

[0029] The data receiving unit (110) can receive voice data from a user. The voice data from a user received by the data receiving unit (110) may be voice data from a patient suffering from pulmonary congestion due to acute heart failure. For example, the voice data from a user may be a patient suffering from pulmonary congestion due to acute heart failure pronouncing "Long live Korea," "Ah," "Eh," "I," "O," and "U." Pulmonary congestion due to acute heart failure refers to pulmonary congestion caused by structural or functional abnormalities of the heart.

[0030] The data preprocessing unit (120) can preprocess the user's voice data. In one embodiment, the data preprocessing unit (120) can extract partial voice data corresponding to a part of the voice data from the received voice data through windowing processing. The data preprocessing unit (120) can convert the extracted partial voice data into a spectrogram. A spectrogram is a graph that visualizes the spectrum of sound. The preprocessing of the user's voice data by the data preprocessing unit (120) will be explained in detail in FIG. 2.

[0031] The pulmonary congestion state differentiation unit (130) can output differentiation data containing pulmonary congestion state information. For example, the pulmonary congestion state differentiation unit (130) can output differentiation data containing the user's pulmonary congestion state information using a neural network-based pulmonary congestion state differentiation model and a spectrogram. The outputting of differentiation data by the pulmonary congestion state differentiation unit (130) using a neural network-based pulmonary congestion state differentiation model will be explained in detail in FIGS. 3 and 4.

[0032] A pulmonary congestion state differentiation device (100) according to one embodiment can differentiate the pulmonary congestion state of a heart failure patient through a pulmonary congestion differentiation model. The pulmonary congestion state differentiation device (100) can differentiate and monitor the pulmonary congestion state by utilizing a negative biomarker for the pulmonary congestion state of a heart failure patient. Unlike conventional pulmonary congestion diagnosis and monitoring technologies, the pulmonary congestion state differentiation device (100) does not require blood analysis or photography through radiation exposure. Unlike the conventional CadioMEMS method, the pulmonary congestion state differentiation device (100) does not require a hospital visit and does not require continuous monitoring and management by medical staff. Therefore, the pulmonary congestion state differentiation device (100) has superior advantages in terms of cost, convenience, safety, and management compared to the CadioMEMS method.

[0033] Pulmonary congestion caused by heart failure can cause shortness of breath and lead to hospitalization or death of the patient. Additionally, pulmonary congestion can increase medical expenditure and lower the patient's quality of life. The pulmonary congestion diagnosis device (100) according to the embodiments can improve the patient's quality of life by reducing the mortality rate, re-hospitalization rate, and medical costs caused by heart failure. Furthermore, the pulmonary congestion diagnosis device (100) is superior to existing conventional technology in that it can diagnose and monitor pulmonary congestion caused by heart failure at low cost by non-invasively recording one's own voice anywhere.

[0035] FIG. 2 is a diagram illustrating a data preprocessing unit according to one embodiment preprocessing voice data.

[0036] Referring to FIG. 2, the data preprocessing unit (120) can preprocess voice data (210) to extract partial voice data (220) corresponding to a part (211) of the voice data. According to one embodiment, the data preprocessing unit (120) can output partial voice data (220) through windowing processing. For example, the data preprocessing unit (120) can apply a window with an interval of 1 second (or N (a natural number greater than or equal to 2) seconds) to the voice data (210) to extract partial voice data (220) corresponding to the area of ​​the window.

[0037] A data preprocessing unit (120) according to one embodiment can convert partial voice data (220) into frequency domain data. For example, the data preprocessing unit (120) can convert the voice data (220) into frequency information by performing a Fourier transform (230) on the partial voice data (220).

[0038] The data preprocessing unit (120) can generate a Mel-spectrogram (250) by performing a Mel-scale conversion (240) on the frequency information of partial voice data converted into frequency domain data. The Mel-scale is a scale that reflects the standard by which humans perceive sounds, and the Mel-spectrogram (250) is a spectrogram containing the frequency information of the Mel-scale.

[0039] In one embodiment, a Mel-spectrogram (250) preprocessed by a data preprocessing unit (120) may be input to a lung congestion state differentiation model. According to one embodiment, the lung congestion state differentiation model may include a ResNet (e.g., resnet-50, resneXt), a DenseNet (e.g., densenet) (310), a ConvNet (e.g., ConvnetXt), an EfficientNet (e.g., efficientnet-b7) and / or a vision transformer (ViT).

[0040] A pulmonary congestion state differentiation model according to one embodiment can output a binary classification value (e.g., binary classification value (320) of FIG. 3, binary classification value (440) of FIG. 4) based on an input Mel-spectrogram (250). For example, the pulmonary congestion state differentiation model can output a binary classification value including at least one of a probability value corresponding to a state where the user's pulmonary congestion is improved and a probability value corresponding to a state where the user's pulmonary congestion is not improved, using a preprocessed Mel-spectrogram (250).

[0041] In a pulmonary congestion state differentiation model according to one embodiment, at least one of user information and user blood test information may be input in addition to a spectrogram (e.g., Mel-spectrogram (250)). User information is personal information regarding a user using a pulmonary congestion state differentiation device and may include, for example, age, gender, medical history and / or lifestyle information. Blood test information is medical information regarding a user obtained by testing the user's blood and may include, for example, information on BNP (B-type natriuretic peptide), NT-proBNP (N-terminal pro-BNT), and / or ST2 (growth stimulation expressed gene 2).

[0043] FIG. 3 is a diagram illustrating that a dense net according to one embodiment outputs a binary classification value.

[0044] Referring to FIG. 3, a lung congestion state differentiation device (e.g., lung congestion state differentiation device (100) of FIG. 1) can differentiate a lung congestion state using a dense net (310) as a lung congestion state differentiation model. The dense net (310) may include a plurality of layers. The dense net (310) may include, for example, a convolution layer (311), dense layers (312, 314, 316, 318), transition layers (313, 315, 317), and a classification layer (319). Each of the convolution layer, dense layer, transition layer, and classification layer included in the dense net (310) may be one or more.

[0045] A convolution layer (311) according to one embodiment can extract a feature vector of input data. For example, the convolution layer (311) can extract a feature vector from an input Mel-spectrogram (e.g., the Mel-spectrogram (250) of FIG. 2).

[0046] A dense layer (312, 314, 316, 318) according to one embodiment represents a connected layer in which layers are densely packed. For example, features extracted from a convolution layer (311) can be input to transition layers (313, 315, 317) through the dense layer (312, 314, 316, 318). A dense net (310) according to one embodiment can extract features regarding pulmonary congestion status information through skip connections. For example, the dense net (310) can extract hidden features through skip connections between layers. A skip connection represents a method in which an input layer skips at least one of the intermediate layers and is directly connected to an output layer.

[0047] A transition layer (313, 315, 317) according to one embodiment can reduce the size of a feature map. For example, the transition layer (313, 315, 317) can reduce the size of a feature map of features extracted through a dense layer.

[0048] A classification layer (319) according to one embodiment may output pulmonary congestion status information based on extracted features. The pulmonary congestion status information may include a binary classification value (320) comprising at least one of a probability value corresponding to a state where pulmonary congestion is improved and a probability value corresponding to a state where the user's pulmonary congestion is not improved.

[0050] FIG. 4 is a diagram illustrating a vision transformer according to one embodiment outputting a binary classification value.

[0051] Referring to FIG. 4, a lung congestion state discrimination device (e.g., the lung congestion state discrimination device (100) of FIG. 1) can distinguish a lung congestion state using a vision transformer (430) as a lung congestion state discrimination model. A vision transformer (430) according to one embodiment may include a projection unit (431) that transforms the dimension of a feature vector of a partial spectrogram, a transformer encoder (432), and a classification layer (433). Each of the projection unit (431), transformer encoder (432), and classification layer (433) included in the vision transformer (430) may be one or more.

[0052] A data preprocessing unit according to one embodiment (e.g., the data preprocessing unit (110) of FIG. 1) can divide a Mel-spectrogram (250) into a plurality of partial spectrograms (420) (or patches) through cropping (410). The data preprocessing unit can input at least one of the divided partial spectrograms (420) into a vision transformer (430).

[0053] A projection unit (431) according to one embodiment can perform vector projection to transform the dimension of the feature vectors of the partial spectrograms (420). For example, the projection unit (431) can transform the dimension of the feature vectors of the partial spectrograms (420) into a dimension that can be processed by a transformer encoder (432).

[0054] A transformer encoder (432) according to one embodiment can calculate attention of vector-projected partial spectrograms. The transformer encoder (432) can calculate attention by performing normalization and multi-head attention on the feature vectors of the vector-projected partial spectrograms and passing them through a fully connected layer.

[0055] A classification layer (433) according to one embodiment may output pulmonary congestion status information based on calculated attention. The pulmonary congestion status information output by the classification layer (433) may include a binary classification value (440) that includes at least one of a probability value corresponding to a state where pulmonary congestion is improved and a probability value corresponding to a state where the user's pulmonary congestion is not improved.

[0057] FIG. 5 is a flowchart illustrating the operations of a method for determining a pulmonary congestion state according to one embodiment. The operations of the method for determining a pulmonary congestion state can be performed by a device for determining a pulmonary congestion state.

[0058] In operation (510), a lung congestion state detection device (e.g., a lung congestion state detection device (100) of FIG. 1) can receive voice data from a user. The lung congestion state detection device can receive voice data from a user through a data receiving unit (e.g., a data receiving unit (110) of FIG. 1). For example, the lung congestion state detection device can receive voice data from a patient with lung congestion caused by acute heart failure.

[0059] In operation (520), the lung congestion state detection device can extract partial voice data through windowing processing. For example, the lung congestion state detection device can extract partial voice data corresponding to a part of the voice data from the voice data through windowing processing at 1-second intervals.

[0060] In operation (530), the lung congestion state detection device can convert partial voice data into a spectrogram. For example, the spectrogram may be a Mel-spectrogram in which the user's voice data is converted into frequency information corresponding to the voice data by a Fourier transform, and the scale of the frequency information is converted into a Mel-scale graph.

[0061] In operation (540), the pulmonary congestion state differentiation device can obtain differentiation data using a pulmonary congestion state differentiation model and a spectrogram. The pulmonary congestion state differentiation device can obtain differentiation data including information on the user's pulmonary congestion state using a neural network-based pulmonary congestion state differentiation model and a spectrogram. A spectrogram is input into the pulmonary congestion state differentiation model, and at least one of the user's user information and the user's blood test information may be further input. User information is personal information regarding a user using the pulmonary congestion state differentiation device and may include, for example, age, gender, medical history, and lifestyle information. Blood test information is medical information regarding the user obtained by testing the user's blood and may include, for example, BNP, NT-proBNP, and ST2.

[0062] Models for differentiating pulmonary congestion may include, for example, ResNet (e.g., resnet-50, resneXt), DenseNet (e.g., densenet) (310), ConvNet (e.g., ConvnetXt), EfficientNet (e.g., efficientnet-b7) and / or a vision transformer (ViT).

[0063] A dense net according to one embodiment (e.g., the dense net (310) of FIG. 3) may be a dense net comprising a convolution layer, a dense layer, a transition layer, and a classification layer. A lung congestion discrimination device may extract hidden features through skip connections of the dense net and determine lung congestion status information using the extracted hidden features. The lung congestion status information output by the dense net according to one embodiment may include at least one of a probability value corresponding to a state where the user's lung congestion is improved and a probability value corresponding to a state where the user's lung congestion is not improved. Through the lung congestion status information, whether the user's lung congestion has improved and how much it has improved can be estimated from the user's voice data.

[0064] A pulmonary congestion state differentiation model according to one embodiment may be a vision transformer (e.g., the vision transformer of FIG. 5 ((430)). The vision transformer may include a projection unit that transforms the dimension of the feature vector of the spectrogram, a transformer encoder, and a classification layer.

[0065] A pulmonary congestion diagnosis device can divide a spectrogram and input it into a vision transformer. For example, the pulmonary congestion diagnosis device can divide the spectrogram into multiple partial spectrograms (or patches) and input at least one of the divided partial spectrograms into a vision transformer.

[0066] A lung congestion state discrimination device according to one embodiment may perform vector projection on a partial spectrogram segmented through a vision transformer, calculate the attention of the vector-projected partial spectrogram, and determine lung congestion state information based on the calculated attention. The lung congestion state information determined by the vision transformer may include at least one of a probability value corresponding to an improved state and a probability value corresponding to a state where the user's lung congestion is not improved. Through the lung congestion state information, whether the user's lung congestion has improved and by how much it has improved can be estimated from the user's voice data.

[0068] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0069] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium, or device so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0070] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can apply various technical modifications and variations based thereon. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0071] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

Claims

Claim 1 A device for distinguishing pulmonary congestion using artificial intelligence analysis, comprising: a data receiving unit for receiving voice data of a user; and a data preprocessing unit for extracting partial voice data corresponding to a part of the voice data from the received voice data through windowing processing and converting the extracted partial voice data into a spectrogram. A pulmonary congestion state discrimination unit that outputs discrimination data including information on the user's pulmonary congestion state using a neural network-based pulmonary congestion state discrimination model and the spectrogram; wherein the pulmonary congestion state discrimination model is a dense net including a convolution layer, a dense layer, a transition layer, and a classify layer; wherein the pulmonary congestion state discrimination model is a dense net, extracts hidden features through skip connections of the dense net, and determines the pulmonary congestion state information using the extracted hidden features; wherein the output of the pulmonary congestion state discrimination model includes at least one of a probability value corresponding to a state where the user's pulmonary congestion is improved and a probability value corresponding to a state where the user's pulmonary congestion is not improved; wherein the pulmonary congestion state discrimination model is a vision transformer, and transforms the dimension of the feature vector of the spectrogram. The system includes a projection unit, a transformer encoder, and a classification layer, wherein the data preprocessing unit divides the spectrogram into a plurality of partial spectrograms and inputs at least one of the divided partial spectrograms into the vision transformer, and the vision transformer performs a vector projection on the divided partial spectrogram.A device for determining a pulmonary congestion state, characterized by calculating the attention of the vector-projected partial spectrogram and determining pulmonary congestion state information based on the calculated attention, wherein the user's voice data is voice data of a patient with pulmonary congestion caused by acute heart failure, and wherein the spectrogram is a Mel-spectrogram in which the user's voice data is converted into frequency information corresponding to the voice data by a Fourier transform and the scale of the frequency information is converted to a Mel-scale. The device is further characterized in that the spectrogram is input into the pulmonary congestion state differentiation model, and at least one of the user's user information and the user's blood test information is further input. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 delete Claim 7 delete Claim 8 delete Claim 9 delete Claim 10 delete Claim 11 A method for determining a pulmonary congestion state performed by a device for determining a pulmonary congestion state, comprising: receiving voice data of a user through a data receiving unit; extracting partial voice data corresponding to a part of the voice data from the received voice data through windowing processing and converting the extracted partial voice data into a spectrogram; and obtaining determination data including information on the user's pulmonary congestion state using a neural network-based pulmonary congestion state determination model and the spectrogram.The pulmonary congestion state differentiation model comprises a convolution layer, a dense layer, a transition layer, and a classification layer; the pulmonary congestion state differentiation model is a dense net, extracts hidden features through skip connections of the dense net, and determines pulmonary congestion state information using the extracted hidden features; the output of the pulmonary congestion state differentiation model comprises at least one of a probability value corresponding to a state where the user's pulmonary congestion is improved and a probability value corresponding to a state where the user's pulmonary congestion is not improved; the pulmonary congestion state differentiation model is a vision transformer, comprises a projection unit that transforms the dimension of a feature vector of the spectrogram, a transformer encoder, and a classification layer; the spectrogram is divided into a plurality of partial spectrograms, and at least one of the divided partial spectrograms is input to the vision transformer; the vision transformer performs vector projection on the divided partial spectrogram, and the attention of the vector-projected partial spectrogram A method for determining a pulmonary congestion state, characterized in that the user's voice data is voice data of a patient with pulmonary congestion caused by acute heart failure, the spectrogram is a Mel-spectrogram in which the user's voice data is converted into frequency information corresponding to the voice data by a Fourier transform and the scale of the frequency information is converted into a Mel-scale graph, and the spectrogram is input into the pulmonary congestion state differentiation model, and at least one of the user's user information and the user's blood test information is further input. Claim 12 delete Claim 13 delete Claim 14 delete Claim 15 delete Claim 16 delete Claim 17 delete Claim 18 delete Claim 19 delete Claim 20 delete

Images