Method and system for supporting electrocardiogram signal reading on basis of language model

Abstract

According to one aspect of the present invention, a method for reading an electrocardiogram signal of a subject by using an electrocardiogram signal reading model is provided. The method comprises the steps of: converting a measured electrocardiogram signal of the subject into language-encoded data; extracting features from the converted language-encoded data; and reading the electrocardiogram signal of the subject by using the extracted features.

Description

Method and system for supporting electrocardiogram signal interpretation based on a language model

[0001] The present invention relates to a method and system for assisting in reading an electrocardiogram signal of a subject based on a language model.

[0002] With the recent development of wearable electrocardiogram devices, it has become possible to measure the electrocardiogram of a subject over a long period of time and obtain large-scale electrocardiogram signal data. As a result, the industry's demand for methods that can quickly and accurately analyze large amounts of accumulated electrocardiogram data is growing significantly.

[0003] Previous attempts have been made to analyze and interpret large-scale electrocardiogram signal data efficiently using artificial intelligence models, but all of them had limitations such as low analysis accuracy or insufficient processing speed for processing large amounts of electrocardiogram data.

[0004] The purpose of the present invention is to solve all of the problems of the above-mentioned prior art.

[0005] In addition, another purpose of the present invention is to quickly and accurately analyze and interpret a large amount of electrocardiogram signals by converting electrocardiogram signals and features included therein into language structure data that can be recognized by a language model, and using this as learning data to linguistically interpret the electrocardiogram signals through a learned language model.

[0006] A representative configuration of the present invention to achieve the above purpose is as follows.

[0007] According to one aspect of the present invention, a method for reading an electrocardiogram signal of a subject using an electrocardiogram signal reading model is provided, the method including the steps of converting a measured electrocardiogram signal of the subject into language structure data, extracting features from the converted language structure data, and reading the electrocardiogram signal of the subject using the extracted features.

[0008] According to another aspect of the present invention, a system for reading an electrocardiogram signal of a subject using an electrocardiogram signal reading model is provided, the system including a conversion unit for converting a measured electrocardiogram signal of the subject into language structure data, an extraction unit for extracting features from the converted language structure data, and a reading unit for reading the electrocardiogram signal of the subject using the extracted features.

[0009] In addition, a non-transitory computer-readable recording medium recording another method for implementing the present invention, another system, and a computer program for executing the method are further provided.

[0010] According to the present invention, an electrocardiogram signal and its contained features are converted into language structure data that can be recognized by a language model, and the electrocardiogram signal is linguistically interpreted through a learned language model using the converted data as learning data, thereby enabling rapid and accurate analysis and interpretation of a large number of electrocardiogram signals.

[0011] FIG. 1 is a diagram schematically illustrating the configuration of an entire system for reading an electrocardiogram signal of a subject using an electrocardiogram signal reading model according to one embodiment of the present invention.

[0012] FIG. 2 is a drawing showing in detail the internal configuration of a reading system according to one embodiment of the present invention.

[0013] FIG. 3 is a diagram exemplarily illustrating the structure of a sentence including one or more slots according to one embodiment of the present invention.

[0014] <Explanation of symbols>

[0015] 100: Communications network

[0016] 200: Reading System

[0017] 210: Conversion section

[0018] 220: Extraction section

[0019] 230: Reading Department

[0020] 240: Communications Department

[0021] 250: Control Unit

[0022] 300: Device

[0023] The following detailed description of the present invention refers to the accompanying drawings, which illustrate specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention, while different from each other, are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be modified and implemented from one embodiment to another without departing from the spirit and scope of the present invention. Furthermore, it should be understood that the positions or arrangements of individual components within each embodiment may also be modified without departing from the spirit and scope of the present invention. Accordingly, the following detailed description is not to be taken in a limiting sense, and the scope of the present invention is to be understood to encompass the scope of the claims and all equivalents thereof. Like reference numerals in the drawings represent the same or similar elements throughout the several aspects.

[0024] Hereinafter, various preferred embodiments of the present invention will be described in detail with reference to the attached drawings so that a person having ordinary skill in the art to which the present invention pertains can easily practice the present invention.

[0025] Composition of the entire system

[0026] FIG. 1 is a diagram schematically illustrating the configuration of an entire system for reading an electrocardiogram signal of a subject using an electrocardiogram signal reading model according to one embodiment of the present invention.

[0027] First, the communication network (100) according to one embodiment of the present invention can be configured regardless of the communication mode such as wired communication or wireless communication, and can be configured with various communication networks such as a local area network (LAN), a metropolitan area network (MAN), and a wide area network (WAN). Preferably, the communication network (100) referred to herein may be the well-known Internet or the World Wide Web (WWW). However, the communication network (100) is not necessarily limited thereto, and may include at least a portion of a well-known wired or wireless data communication network, a well-known telephone network, or a well-known wired or wireless television communication network.

[0028] For example, the communication network (100) may be a wireless data communication network that implements conventional communication methods such as WiFi communication, WiFi-Direct communication, Long Term Evolution (LTE) communication, 5G communication, Bluetooth communication (including Bluetooth Low Energy (BLE) communication), infrared communication, ultrasonic communication, etc., at least in part.

[0029] Next, the reading system (200) according to one embodiment of the present invention can perform communication with the device (300) described later through the communication network (100). In addition, the reading system (200) according to one embodiment of the present invention can perform a function of converting a measured electrocardiogram signal of a subject into language structure data, extracting features from the converted language structure data, and reading the electrocardiogram signal of the subject using the extracted features. Meanwhile, such a reading system (200) may be a digital device equipped with a memory means and a microprocessor to have a computing capability, and may be, for example, a server system operated on the communication network (100).

[0030] The configuration and function of the reading system (200) according to one embodiment of the present invention will be described in detail below.

[0031] Next, a device (300) according to one embodiment of the present invention is a digital device that includes a function for communicating after being connected to a reading system (200), and any digital device that has a memory means, a microprocessor, and a computing capability, such as a smart phone, a tablet, a smart watch, a smart band, smart glasses, a desktop computer, a notebook computer, a workstation, a PDA, a web pad, a mobile phone, etc., can be adopted as the device (300) according to the present invention.

[0032] In addition, according to one embodiment of the present invention, the device (300) may further include an application program for performing a function according to the present invention. Such an application may exist in the form of a program module within the device (300). The nature of such a program module may be generally similar to the conversion unit (210), extraction unit (220), reading unit (230), communication unit (240), and control unit (250) of the reading system (200) as described below. Here, at least a part of the application may be replaced with a hardware device or firmware device that can perform a function substantially identical to or equivalent thereto, as necessary.

[0033] Configuration of the reading system

[0034] Below, the internal configuration of the reading system (200) that performs important functions for implementing the present invention and the functions of each component will be examined.

[0035] FIG. 2 is a drawing showing in detail the internal configuration of a reading system (200) according to one embodiment of the present invention.

[0036] As illustrated in FIG. 2, a reading system (200) according to one embodiment of the present invention may include a conversion unit (210), an extraction unit (220), a reading unit (230), a communication unit (240), and a control unit (250). According to one embodiment of the present invention, at least some of the conversion unit (210), the extraction unit (220), the reading unit (230), the communication unit (240), and the control unit (250) of the reading system (200) may be program modules that communicate with an external system (not shown). These program modules may be included in the reading system (200) in the form of an operating system, an application program module, or other program modules, and may be physically stored in various known memory devices. In addition, these program modules may also be stored in a remote memory device that can communicate with the reading system (200). Meanwhile, these program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform specific tasks or execute specific abstract data types, as described later in accordance with the present invention.

[0037] Meanwhile, although the reading system (200) has been described as above, this description is exemplary, and it is obvious to those skilled in the art that at least some of the components or functions of the reading system (200) may be realized within a device (300) or a server (not shown) or included within an external system (not shown) as needed.

[0038] First, according to one embodiment of the present invention, the conversion unit (210) can perform a function of converting the measured electrocardiogram signal of the subject into language structure data.

[0039] An electrocardiogram signal according to one embodiment of the present invention refers to data that records the activity current generated in the heart in the form of waves. Specifically, in an electrocardiogram signal, waves (or beats) repeatedly appear according to the heartbeat, and one wave may include multiple waveforms (or segments). These waveforms can generally be classified into P waves, QRS waves (or QRS complexes), and T waves according to the characteristics of the electrocardiogram signal that appear differently depending on the electrical activity of the heart (e.g., signal generation location or generation time, etc.). Meanwhile, among the waves that repeatedly appear in an electrocardiogram signal, the time interval between the R wave, which is a positive (+) waveform located in the middle of the QRS wave included in one wave, and the R wave in the QRS wave included in the immediately following beat is called the RR interval. In summary, an electrocardiogram signal can have (1) characteristics of the aforementioned representative waveforms (P wave, QRS wave, T wave), and (2) characteristics including RR intervals, and by comprehensively analyzing these characteristics, it is possible to determine whether an electrocardiogram signal is normal or to detect at least one event described later in an electrocardiogram signal.

[0040] The measured electrocardiogram signal according to one embodiment of the present invention is an electrocardiogram signal measured from a subject, and is a concept meaning an electrocardiogram signal to be read through the reading system (200) or the electrocardiogram signal reading model of the present invention.

[0041] Language structure data according to one embodiment of the present invention is a term meaning data used as learning data by an electrocardiogram signal reading model (language model) according to one embodiment of the present invention, or input data analyzed through a language model, and is a concept meaning various forms of sequence data including grammatical, semantic, and syntactic characteristics of a language.

[0042] Meanwhile, language structure data according to one embodiment of the present invention may include tokens as the minimum unit. The tokens described above are a type of unit into which the entire data is divided for the language model to process and understand input data. They may generally be words, numbers, punctuation marks, or units with specific meanings.

[0043] A token according to one embodiment of the present invention can be used to convert an electrocardiogram signal into the above-described language structure data by corresponding to a plurality of electrocardiogram signal elements (listed in time series) in continuous electrocardiogram signal data.

[0044] An electrocardiogram signal element (or element) according to one embodiment of the present invention is a concept that includes both waveforms (P wave, QRS wave, and T wave) and intervals (RR intervals) as various features included in an electrocardiogram signal that can be used to describe the overall characteristics (characteristics of electrical activity) of the electrocardiogram signal, and will be described in more detail below.

[0045] Meanwhile, a token according to an embodiment of the present invention may mean a cluster generated for each ECG signal element by classifying various characteristics included in an ECG signal by ECG signal element and clustering multiple measurement values ​​for each classified ECG signal element based on a predetermined standard. Meanwhile, a set of tokens whose linguistic meaning has been learned by an ECG signal interpretation model according to an embodiment of the present invention may be referred to as a vocabulary.

[0046] The input data of the electrocardiogram signal reading model according to one embodiment of the present invention may be data including language structure data converted from the electrocardiogram signal described above through the conversion unit (210) according to one embodiment of the present invention.

[0047] Continuing, the conversion unit (210) according to one embodiment of the present invention may perform a function of classifying a measured electrocardiogram signal (or a plurality of characteristics included therein) into one or more of the aforementioned electrocardiogram signal elements, and tokenizing each of the classified elements to obtain language structure data.

[0048] An electrocardiogram signal element according to one embodiment of the present invention is a concept that includes all kinds of characteristics included in an electrocardiogram signal. Specifically, the electrocardiogram signal element may include various patterns, information, and characteristics appearing on an electrocardiogram signal, such as (1) waves (e.g., P waves, QRS waves (QRS complexes), and T waves, etc.) and (2) RR intervals. More specifically, the various characteristics of the aforementioned waves include, but are not limited to, the shape, width, amplitude, and duration of each wave, and include all kinds of characteristics that can significantly distinguish one wave from another. The conversion unit (210) according to one embodiment of the present invention may classify the characteristics included in an electrocardiogram signal according to the electrocardiogram signal elements described above.

[0049] As described above, a token according to an embodiment of the present invention is a minimum unit of language structure data, and may refer to a cluster obtained by clustering measurement values ​​for each element of one or more electrocardiogram signal elements included in an electrocardiogram signal. For example, a QRS wave, which is one of the waveforms included in one or more waves included in a measured electrocardiogram signal, may have various different shapes depending on various variables such as the subject's constitution, health status (presence or absence of arrhythmia, etc.), heart location, and the subject's posture at the time of electrocardiogram measurement, and representative waveforms (i.e., clusters) can be obtained by clustering these based on characteristic (e.g., shape) similarity, and the clusters formed as a result of the above-described clustering can be referred to as tokens (i.e., tokens for QRS waves).

[0050] Meanwhile, an electrocardiogram interpretation model according to an embodiment of the present invention can learn by using language structure data converted from an electrocardiogram signal as learning data. Specifically, the electrocardiogram interpretation model according to an embodiment of the present invention can learn the linguistic meaning of each individual token in relation to the entire electrocardiogram signal by analyzing the meaning of the aforementioned token in the entire electrocardiogram signal (i.e., large-scale electrocardiogram signal data), the frequency with which each token appears in the entire electrocardiogram signal data, and the relationship (context) of each token with other tokens included in the entire electrocardiogram signal data, thereby determining whether a measured electrocardiogram signal is normal or detecting at least one event in the measured electrocardiogram signal.

[0051] Determining whether a (measured) electrocardiogram signal is normal according to one embodiment of the present invention may mean determining whether the (measured) electrocardiogram signal is normal or indicates a specific pathological condition (e.g., arrhythmia).

[0052] An event according to one embodiment of the present invention is a concept encompassing all physiological phenomena or information that can be identified or analyzed from a (measured) electrocardiogram signal, and should be understood to include not only whether the electrocardiogram signal is normal or indicates a specific pathological condition (e.g., arrhythmia), but also all types of processed or unprocessed data that can be obtained from the electrocardiogram signal. For example, an event according to one embodiment of the present invention may be, but is not limited to, whether each token included in the measured electrocardiogram signal is normal (i.e., whether the token is a correct sentence in which the token is naturally arranged when considering the meaning and context of the previously learned token), or whether it corresponds to a symptom indicating an abnormality (i.e., whether the token is an incorrect sentence in which the token is not naturally arranged when considering the meaning and context of the previously learned token), or the like.

[0053] Meanwhile, as described above, determining that the (measured) electrocardiogram signal is normal may have the same meaning as (1) not detecting the above-described event, or (2) detecting the event of 'the electrocardiogram signal is normal' among the above-described events. That is, it should be understood that the electrocardiogram signal reading model according to one embodiment of the present invention may only determine whether the (measured) electrocardiogram signal is normal, or may provide all types of related information, including whether the (measured) electrocardiogram signal is normal, through a method of detecting the above-described event.

[0054] Meanwhile, a set of tokens learned by the electrocardiogram signal interpretation model of the present invention can be referred to as a vocabulary, and this vocabulary can be used by the electrocardiogram signal interpretation model to interpret language structure data converted from an electrocardiogram signal.

[0055] Tokenization according to one embodiment of the present invention may mean a process of classifying an electrocardiogram signal into one or more electrocardiogram signal elements and assigning the most appropriate token (cluster) to each classified element. Specifically, the process of assigning the above-described appropriate token may mean a process of comparing each element included in the electrocardiogram signal with a cluster obtained through clustering and matching the most similar cluster to each element. For example, it may mean extracting (morphological) features of a QRS complex included in an electrocardiogram signal to be analyzed (e.g., a measured electrocardiogram signal) and, based on these features, assigning the most similar cluster among a plurality of clusters for the QRS complex (i.e., a token for the QRS complex).

[0056] Meanwhile, tokens according to one embodiment of the present invention can be divided into first token to fourth token, and specific definitions of these tokens will be described in more detail below.

[0057] Next, according to one embodiment of the present invention, the extraction unit (220) can perform a function of extracting features from converted language structure data.

[0058] Continuing, the extraction unit (220) according to one embodiment of the present invention can extract features by extracting an embedding vector from the converted language structure data. Specifically, the converted language structure data includes one or more tokens corresponding to each electrocardiogram signal element included in the electrocardiogram signal, and the extraction unit (220) according to one embodiment of the present invention can convert each token into an embedding vector through embedding that numerically expresses the meaning (linguistic meaning) of the token.

[0059] Next, the reading unit (230) according to one embodiment of the present invention can perform a function of reading an electrocardiogram signal of a subject. Specifically, reading an electrocardiogram signal of a subject according to one embodiment of the present invention may mean extracting features from a plurality of electrocardiogram signal elements included in the electrocardiogram signal and analyzing the features to determine whether the measured electrocardiogram signal is normal or detecting at least one event as described above from the measured electrocardiogram signal, and more specifically, it may mean diagnosing whether the subject has signs of an abnormal symptom or an abnormal symptom, or supporting the diagnosis.

[0060] Continuing, the reading unit (230) according to one embodiment of the present invention can recognize the electrocardiogram signal characteristics unique to the subject in the language structure data converted from the measured electrocardiogram signal, and based on the recognized electrocardiogram signal characteristics unique to the subject, determine whether the measured electrocardiogram signal of the subject is normal or perform a function of detecting at least one event as described above in the measured electrocardiogram signal of the subject.

[0061] Meanwhile, even when measuring an electrocardiogram from the same body part, the characteristics of the waveform may appear differently depending on the location, posture, constitution, etc. of the heart from person to person (for example, even with the same normal QRS wave, there may be morphological differences from person to person). The electrocardiogram signal interpretation model according to one embodiment of the present invention can recognize the individual differences described above in the measured electrocardiogram signal of the subject (i.e., the electrocardiogram signal characteristics unique to the subject) and interpret the measured electrocardiogram signal based on the differences. Specifically, the electrocardiogram signal interpretation model according to one embodiment of the present invention can learn the electrocardiogram signal pattern unique to the subject (i.e., the electrocardiogram signal characteristics unique to the subject) by linguistically analyzing the measured electrocardiogram signal of the subject. This method does not simply analyze the characteristics of the ECG signal elements individually, but analyzes and learns the linguistic meaning or context that each token (such as a paragraph, sentence, or word) corresponding to each ECG signal element has in the entire sequence data (i.e., the entire ECG signal data), thereby enabling the identification of the relationship between individual waveforms and the relationship between individual waveforms and the entire measured ECG signal. As a result, it becomes possible to quickly and accurately read large-scale continuous ECG data by taking into account even the ECG signal characteristics unique to the subject.

[0062] According to one embodiment of the present invention, comparing features may mean embedding each of the aforementioned tokens to derive an embedding vector, and calculating the similarity between the derived embedding vectors. Specifically, the similarity between vectors may be obtained through cosine similarity or Euclidean distance, but is not limited thereto.

[0063] The reading unit (230) according to one embodiment of the present invention reads the electrocardiogram signal of the subject using the electrocardiogram signal reading model, and the electrocardiogram signal reading model according to one embodiment of the present invention has learned about the meaning of each token described above (i.e., based on the semantic similarity of the tokens, learning to make tokens with similar meanings have similar vectors), frequency (i.e., learning to make tokens with similar frequencies have similar vectors by learning the frequency of appearance of the tokens), and context (i.e., learning to make contextually similar tokens have similar vectors by reflecting the context in which the tokens are used), so that it is possible to determine whether the measured electrocardiogram signal is normal or to detect at least one event (e.g., various signs indicating abnormal symptoms, etc.) included in the measured electrocardiogram signal through a contextual error detection (i.e., outlier detection) process.

[0064] In addition, as described above, the reading unit (230) according to one embodiment of the present invention can recognize the unique electrocardiogram signal characteristics of the subject in the measured electrocardiogram signal, thereby taking into account all unique characteristics of the subject (i.e., individual characteristics) when reading the electrocardiogram signal, thereby maximizing the reading accuracy of the measured electrocardiogram waveform.

[0065] Next, the electrocardiogram signal reading model according to one embodiment of the present invention may be a language model (LM) that is trained using an electrocardiogram signal converted into language structure data as training data.

[0066] Meanwhile, according to one embodiment of the present invention, the above-described language model is a model used in natural language processing (NLP), and may mean a model that learns the patterns and structures of language based on given sequence data (e.g., text data, voice data, time-series data, or language structure data according to one embodiment of the present invention, etc.) to generate a new sequence or predict the probability of an input sequence. As described above, the ECG signal interpretation model according to one embodiment of the present invention is a language model, and uses language structure data generated by tokenizing an ECG signal for each element as sequence data, and can be used as learning data or input data.

[0067] Specifically, the language model described above may be a model that (1) receives large-scale (continuous) electrocardiogram signal data as input data and converts each electrocardiogram signal element into a corresponding token to convert it into language structure data, (2) generates an optimized embedding for each token based on the converted language structure data, and (3) learns to understand the meaning, context, and structure of the language structure data through the generated embedding vector. For example, language models that can be used for reading electrocardiogram signals according to one embodiment of the present invention may include, but are not limited to, a recurrent neural network (RNN), a long short-term memory (LSTM), a transformer, BERT (bidirectional encoder representations from transformers), a generative pre-trained transformer (GPT), and modified models of GPT (e.g., OpenAI's Ada version 1, Ada version 2, Babbage, Curie, Davinci, etc.).

[0068] Next, language structure data according to one embodiment of the present invention may include one or more paragraphs, the paragraphs may include one or more sentences, and the sentences may include one or more slots.

[0069] A paragraph according to one embodiment of the present invention may include one or more sentences, as one of the substructures included in the language structure data converted from the electrocardiogram signal. Specifically, a paragraph according to one embodiment of the present invention may consist of six sentences.

[0070] A sentence according to one embodiment of the present invention may include one or more slots as a substructure constituting the paragraph described above. Specifically, a single sentence according to one embodiment of the present invention may be composed of nine slots.

[0071] A slot according to one embodiment of the present invention is a concept meaning a space (or position) where a token, which is a lower unit constituting a sentence, can be located, and thus a sentence according to one embodiment of the present invention may include one or more slots, and a sentence may be completed by positioning a token in the above-described slot. Specifically, electrocardiogram signal data according to one embodiment of the present invention may have a structure in which one or more waves are sequentially listed over time as time-series data, and thus, transforming the electrocardiogram signal data into language structure data according to one embodiment of the present invention may mean a series of processes of (1) converting waves listed in chronological order into the most similar tokens (i.e., clusters for each element) for each electrocardiogram signal element, (2) obtaining a sentence by positioning the converted tokens in slots listed in the same order as the chronological order (at this time, an empty slot (or a slot indicated as 0) may be positioned between each slot, as necessary), (3) combining one or more obtained sentences to obtain a paragraph, and combining one or more paragraphs to obtain language structure data.

[0072] Continuing, each slot included in a sentence according to one embodiment of the present invention may be empty or may include one of the first to fourth tokens.

[0073] According to one embodiment of the present invention, tokens may be divided into first to fourth tokens (depending on the type of electrocardiogram signal element). According to one embodiment of the present invention, the first token may mean a P wave cluster, the second token may mean a QRS wave cluster, the third token may mean a T wave cluster, and the fourth token may mean an RR interval cluster. Specifically, the first to third tokens may be obtained by dividing a continuous electrocardiogram signal by waveform, and clustering multiple measurements by the same waveform to obtain one or more representative waveforms (for example, the P wave cluster may be composed of 15 types, the QRS wave cluster may be composed of 20 types, and the T wave cluster may be composed of 15 types).

[0074] More specifically, clustering may use, but is not limited to, the K-Shape clustering algorithm used for clustering time series data.

[0075] Meanwhile, the fourth token is a concept that includes clusters obtained by clustering RR intervals based on a predetermined criterion. For example, the fourth token may include three clusters (i.e., equal tokens, over tokens, and under tokens) distinguished based on a predetermined value obtained from the entire electrocardiogram waveform.

[0076] Specifically, the three clusters included in the fourth token described above can be obtained by (1) calculating the average RR interval from the entire electrocardiogram signal, (2) dividing the signal into three sections of slow pulse, normal pulse, and fast pulse based on the average RR interval value, (3) calculating a clustering criterion (i.e., a numerical range that serves as a criterion for clustering) by applying different standard deviations to each section, and (4) clustering the actual RR interval of the input electrocardiogram signal based on the clustering criterion described above.

[0077] For example, if the average RR interval obtained from the entire ECG signal is 200 or more (i.e., slow pulse), a standard deviation of 40 is applied, if the average RR interval is 120 to 200 (i.e., normal pulse), a standard deviation of 30 is applied, and if the average RR interval is 120 or less (i.e., fast pulse), a standard deviation of 20 is applied to calculate the clustering criteria (the numerical range described above), and correspondingly, if the measured RR interval is larger than the average 'RR interval + standard deviation value', an over token is awarded, if the measured RR interval is smaller than the 'RR interval - standard deviation value', an under token is awarded, and if the measured RR interval is between the 'RR interval + standard deviation' value and the 'RR interval - standard deviation' value, an equal token can be awarded.

[0078] As a more specific example, if we assume that the average RR interval calculated from the electrocardiogram signal is 200 or more (as described above, the standard deviation is applied as 40), the average 'RR interval + standard deviation' value becomes 240, and the 'RR interval - standard deviation' value becomes 160. Correspondingly, the actual RR intervals of the measured electrocardiogram signal can be classified into each section distinguished based on the above-described values, and the same token, excess token, and under token can be assigned accordingly (i.e., tokenization is performed for the RR intervals included in the electrocardiogram signal through this).

[0079] An electrocardiogram signal reading model according to one embodiment of the present invention can convert an electrocardiogram signal into a sentence (i.e., language structure data) using the above-described slot and tokens (first to fourth tokens) that can be positioned therein, and can use this as input data of a language model.

[0080] Referring to FIG. 3, a sentence according to one embodiment of the present invention may be composed of first to ninth slots, and a predetermined token may be positioned in each slot. For example, a fourth token (a token representing an RR interval) may be positioned in the first slot (i.e., the first slot) and the last slot (i.e., the ninth slot) constituting a sentence, a second token (a token representing a QRS wave) may be positioned in the central slot (i.e., the fifth slot), a first token (a token representing a P wave) may be positioned in a slot located between the first slot and the fifth slot, a third token (a token representing a T wave) may be positioned in a slot located between the fifth slot and the ninth slot, and a slot located between each token located in a slot may be empty without a token being positioned therein (an empty slot may be represented as 0). More specifically, the fourth token located in the first slot may be a token representing the RR interval with the previous wave, and the fourth token located in the ninth slot may be a token representing the RR interval with the subsequent wave.

[0081] Continuing with reference to FIG. 3, the sentences (or slots) illustrated in FIG. 3 are examples of sentences converted to correspond to a normal electrocardiogram signal in which the P wave, QRS wave, and T wave are all normally located in one wave of the electrocardiogram signal (i.e., a normal electrocardiogram signal in which all of the waveforms exist).

[0082] Meanwhile, a sentence (or a slot included therein) according to an embodiment of the present invention can express, in addition to a normal electrocardiogram signal in which a P wave-QRS wave-T wave appear in chronological order, an extremely abnormal electrocardiogram signal, such as a waveform in which at least one of the P wave and the T wave is absent due to a serious abnormal symptom in the subject or an error in the electrocardiogram measuring device, a waveform in which the P wave or the T wave appears more than twice in one wave, or a T wave appears temporally preceding the QRS wave, by converting it into language structure data. For example, if it is a signal in which the P wave does not appear at all, it can be expressed by leaving the slot (i.e., the third slot) in which the first token is generally positioned empty (or by masking the slot). Furthermore, in the case of an extremely abnormal electrocardiogram signal in which the T wave temporally precedes the QRS wave and is located between the P wave and the QRS wave (especially, a signal incompletely acquired due to a machine error, etc.), it can be expressed by placing the third token (a token representing the T wave) in the empty slot (i.e., the fourth slot). In this way, the ECG signal reading model according to one embodiment of the present invention can construct a robust model by expressing, learning, and adapting to ECG signals mixed with measurement errors rather than perfectly acquired ECG signals as language structure data using the above-described sentence (or slot included therein) structure.

[0083] However, the contents of the sentence structure described above and the structure of the sentence (or slot included therein) illustrated in FIG. 3 are merely examples, and they may be modified in any way to the extent that the purpose of the present invention can be achieved.

[0084] Next, tokens located in slots according to one embodiment of the present invention may be masked.

[0085] The training data for the electrocardiogram signal interpretation model according to one embodiment of the present invention can be expanded and enriched through data augmentation. Specifically, the data augmentation method described above may be a masking method.

[0086] The above-described masking method can be applied to language structure data according to one embodiment of the present invention to perform data augmentation. Specifically, masking can be performed by converting one or more of the tokens located in the above-described slots to special tokens (e.g., OMT tokens) or 0.

[0087] Specifically, masking according to one embodiment of the present invention may be performed by masking at least one of the tokens for the P wave and the T wave (i.e., the first token and the third token located in the second to fourth slots and the sixth to eighth slots), excluding the tokens for the RR interval and the tokens for the QRS wave (i.e., the fourth token located in the first slot and the ninth slot and the second token located in the fifth slot). More specifically, the tokens located in the third slot and the seventh slot (i.e., the general first token or the third token described above) may be masked by replacing them with OMT tokens, and the tokens located in any one of the second, fourth, sixth, and eighth slots as abnormal P waves or T waves as described above may be masked by deleting (or converting to 0).

[0088] Meanwhile, masking according to one embodiment of the present invention may be performed with a predetermined probability for each token (e.g., each token may be deleted, converted to 0, or converted to an OMT token with a probability of X%). This predetermined probability may be modified to various values ​​depending on the purpose of the present invention, the ECG signal interpretation model, and the characteristics of the dataset. For example, the predetermined probability described above may be 10% to 30%, 15% to 25%, or 20%.

[0089] Next, the learning data according to one embodiment of the present invention may further include morphological feature data of the electrocardiogram signal.

[0090] An electrocardiogram signal interpretation model according to one embodiment of the present invention can additionally utilize morphological feature data of the electrocardiogram signal, along with the aforementioned language structure data, as training data. The morphological features of the electrocardiogram signal described above may mean that the electrocardiogram signal is extracted as an embedding vector using a model (e.g., a convolutional neural network) that embeds the morphological characteristics of the waveform (i.e., waveform image) into a vector, without converting the electrocardiogram signal into language structure data.

[0091] Specifically, the embedding vector generated by embedding each token is taken as the first vector, and the embedding vector generated by embedding morphological features (or morphological features) through the process described above is taken as the second vector, and then the first vector and the second vector are concatenated to generate a new long vector, thereby combining the information of the two vectors to obtain a richer expression, which can be used as learning data. Accordingly, the ECG signal interpretation model according to one embodiment of the present invention learns linguistically about the relationship between each waveform and the relationship between waves from the first vector described above, and at the same time learns about the morphological characteristics of the waveform (i.e., the image of the waveform) from the second vector described above, thereby further improving the speed and accuracy of interpretation.

[0092] Meanwhile, as described above, the morphological characteristics of the ECG signal can be embedded into a vector using a convolutional neural network (CNN), but is not limited thereto, and any type of model capable of extracting features from an image and learning them, such as its modified models VGGNet, ResNet, Inception Network, DenseNet, MobileNet, and EfficientNet, can be used to embed the morphological characteristics of the ECG signal.

[0093] Next, the communication unit (240) according to one embodiment of the present invention can perform a function that enables data transmission and reception from / to the conversion unit (210), the extraction unit (220), and the reading unit (230).

[0094] Finally, the control unit (250) according to one embodiment of the present invention can perform a function of controlling the flow of data between the conversion unit (210), the extraction unit (220), the reading unit (230), and the communication unit (240). That is, the control unit (250) according to one embodiment of the present invention can control the flow of data from / to the outside of the reading system (200) or the flow of data between each component of the reading system (200), thereby controlling the conversion unit (210), the extraction unit (220), the reading unit (230), and the communication unit (240) to perform their own functions.

[0095] The embodiments of the present invention described above may be implemented in the form of program commands that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, etc., either singly or in combination. The program commands recorded on the computer-readable recording medium may be specially designed and configured for the present invention or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. Hardware devices may be changed into one or more software modules to perform processing according to the present invention, and vice versa.

[0096] Although the present invention has been described above with specific details such as specific components and limited examples and drawings, these are provided only to help a more general understanding of the present invention, and the present invention is not limited to the above examples, and those with ordinary knowledge in the technical field to which the present invention pertains can make various modifications and changes based on this description.

[0097] Therefore, the idea of ​​the present invention should not be limited to the embodiments described above, and not only the scope of the patent claims described below but also all scopes equivalent to or equivalently modified from the scope of the patent claims are considered to fall within the scope of the idea of ​​the present invention.

Claims

1. A method for reading the electrocardiogram signal of a subject using an electrocardiogram signal reading model, A step of converting the measured electrocardiogram signal of the subject into language structure data, A step of extracting features from the above-mentioned converted language structure data, and A step of reading the electrocardiogram signal of the subject using the above extracted features is included. method.

2. In paragraph 1, In the above conversion step, the measured electrocardiogram signal is classified into one or more electrocardiogram signal elements, and each of the classified electrocardiogram signal elements is tokenized to obtain language structure data. method.

3. In paragraph 1, In the above extraction step, an embedding vector is obtained from the converted language structure data. method.

4. In paragraph 1, In the above reading step, the electrocardiogram signal characteristic unique to the subject is recognized from the language structure data converted from the measured electrocardiogram signal, and based on the recognized electrocardiogram signal characteristic unique to the subject, the subject's measured electrocardiogram signal is judged to be normal or at least one event is detected from the subject's measured electrocardiogram signal. method.

5. In paragraph 1, The above ECG signal interpretation model is a language model (LM) that is trained using ECG signals converted into language structure data as learning data. method.

6. In paragraph 5, The above language structure data includes one or more paragraphs, The above paragraph contains one or more sentences, The above sentence contains one or more slots. method.

7. In paragraph 6, The above slot is empty or contains any one of the first to fourth tokens. method.

8. In paragraph 7, The token located in the above slot is masked method.

9. In paragraph 5, The above learning data further includes morphological feature data of the electrocardiogram signal. method.

10. A non-transitory computer-readable recording medium recording a computer program for executing the method according to paragraph 1.

11. A system for reading the electrocardiogram signal of a subject using an electrocardiogram signal reading model, A conversion unit that converts the measured electrocardiogram signal of the subject into language structure data, An extraction unit that extracts features from the above-mentioned converted language structure data, and It includes a reading unit that reads the electrocardiogram signal of the subject using the above extracted features. System.

12. In paragraph 11, The above-mentioned conversion unit classifies the measured electrocardiogram signal into one or more electrocardiogram signal elements, and tokenizes each of the classified electrocardiogram signal elements to obtain language structure data. System.

13. In paragraph 11, The above extraction unit obtains an embedding vector from the converted language structure data. System.

14. In paragraph 11, The above-mentioned reading unit recognizes the electrocardiogram signal characteristic unique to the subject from the language structure data converted from the measured electrocardiogram signal, and determines whether the measured electrocardiogram signal of the subject is normal based on the recognized electrocardiogram signal characteristic unique to the subject, or detects at least one event from the measured electrocardiogram signal of the subject. System.

15. In paragraph 11, The above ECG signal interpretation model is a language model (LM) that is trained using ECG signals converted into language structure data as learning data. System.

16. In paragraph 15, The above language structure data includes one or more paragraphs, The above paragraph contains one or more sentences, The above sentence contains one or more slots. System.

17. In paragraph 16, The above slot is empty or contains any one of the first to fourth tokens. System.

18. In paragraph 17, The token located in the above slot is masked System.

19. In paragraph 15, The above learning data further includes morphological feature data of the electrocardiogram signal. System.

Images