Method and apparatus for converting bioelectric signal data into numerical vectors and method and apparatus for analyzing disease using the same

Abstract

Embodiments relate to a method and apparatus for extracting numerical vector information from bioelectric signal data, particularly single-channel or multi-channel electrocardiogram signals or black-and-white electrocardiogram images or color electrocardiogram images obtained based on the electrocardiogram signals, using a deep learning algorithm, and a method and apparatus for analyzing and predicting diseases and providing diagnosis assistance information related to the diseases using the method and apparatus.

Description

Technical Field

[0001] This invention relates to a method and apparatus for converting bioelectrical signal data such as electrocardiogram data into numerical vectors, and to a method and apparatus for analyzing and predicting diseases and providing disease-related diagnostic auxiliary information using the same. Background Technology

[0002] Electrical biosignals are signals that express changes in biological signals as changes in electrical signals. Representative examples include electroencephalograms (EEG), electromyograms (EMG), electrooculograms (EOG), and electrocardiograms (ECG). These biosignals are unstructured data, making them difficult to analyze and apply.

[0003] In particular, electrocardiography (ECG) records the electrical activity of the heart muscle during a heartbeat. ECG testing, a simple examination taking less than a minute, is used to diagnose acute myocardial infarction and arrhythmias. ECG analysis is highly sophisticated in studying patient cardiac function and identifying these heart conditions; physicians have used ECG systems to monitor patients' cardiac activity for decades.

[0004] However, electrocardiograms contain a variety of information that even experts find difficult to interpret. Although this information can be used to further predict a patient's various future conditions and appropriate treatment, its current use remains limited. Summary of the Invention

[0005] Technical issues

[0006] According to one embodiment, an exemplary example of the present invention provides a method and apparatus for extracting numerical information from bioelectrical signal data such as electrocardiogram (ECG) to maximize its application scope and applying it within a clinical framework or integrating it with other patient information to provide auxiliary information for analyzing, predicting or diagnosing diseases.

[0007] Technical solution

[0008] An exemplary embodiment of the present invention provides an apparatus for converting bioelectrical signal data into a numerical vector, comprising: an acquisition unit for acquiring bioelectrical signal data; and an encoder for receiving the bioelectrical signal data and calculating the numerical vector using a deep learning algorithm.

[0009] Furthermore, another exemplary example of the present invention provides a disease analysis apparatus, comprising: the encoder described above; and an analysis unit that uses the numerical vector described above to provide auxiliary information for analyzing, predicting, or diagnosing diseases.

[0010] Furthermore, another exemplary embodiment of the present invention provides a method for converting bioelectrical signal data into numerical vectors, executed by a processor, for converting bioelectrical signal data such as electrocardiogram (ECG) into numerical vectors. The method includes the following steps: acquiring bioelectrical signal data; inputting the bioelectrical signal data into an encoder; and calculating the numerical vector using a deep learning algorithm through the encoder.

[0011] Furthermore, another exemplary example of the present invention provides a disease analysis method that not only performs the above steps, but also includes the following steps: generating analysis results using the above numerical vectors, wherein the analysis results are used to provide auxiliary information for analyzing, predicting or diagnosing diseases.

[0012] Specifically, according to one embodiment, an apparatus or disease analysis device for converting bioelectrical signal data into a numerical vector includes: an acquisition unit for acquiring bioelectrical signal data; and an encoder for receiving the bioelectrical signal data and calculating a first numerical vector using a deep learning algorithm. The first numerical vector is structured data related to features extracted from the bioelectrical signal data, including, particularly in zonal regions, at least one of anatomical (location) features and temporal features that can be extracted from the bioelectrical signal data. This first numerical vector can be effectively used for downstream tasks or machine learning, as described later.

[0013] The apparatus of one exemplary embodiment of the present invention further includes an analysis unit that uses the first numerical vector to perform disease or health-related analysis, prediction, or provide diagnostic assistance information.

[0014] In an exemplary embodiment of the apparatus of the present invention, the apparatus includes multiple downstream processing units that simultaneously process multiple tasks using the first numerical vector. As error signals from the outputs of each downstream task network converge at the end of an encoder due to backpropagation, the versatility of the first numerical vector can be improved by training an encoder.

[0015] In an exemplary embodiment of the apparatus of the present invention, the first numerical vector itself can be used as an input vector for a downstream processing unit or concatenated with other structured data information as an input vector for a downstream processing unit.

[0016] In the apparatus of an exemplary embodiment of the present invention, there may be two or more encoders, and an input value vector may be generated by concatenating multiple first value vectors output from each encoder.

[0017] In an exemplary embodiment of the apparatus of the present invention, N consecutive bioelectrical signal data can be passed through an encoder to obtain N consecutive first numerical vectors.

[0018] In an exemplary embodiment of the present invention, after dividing bioelectrical signal data according to a predetermined time interval, the device can provide relevant analysis, diagnosis or prediction of a specific disease based on the time-point weighted average of the result values ​​at each time point obtained by the encoder or the encoder and downstream processing unit through the information of each divided data interval.

[0019] In an exemplary embodiment of the apparatus of the present invention, when training the network of a downstream task, the apparatus can modify (update) the network weights of the downstream task by training after fixing the network weights of the encoder, and modify (update) the overall weights of the encoder network and the network of the downstream task by additional training.

[0020] In an exemplary embodiment of the apparatus of the present invention, the aforementioned plurality of downstream processing units may each be executed by a multi-layer perceptron (MLP) having two or more fully connected layers.

[0021] In an exemplary embodiment of the apparatus of the present invention, the MLP can be trained together with the encoder's coding network through (jointly) multi-task learning, or it can be trained separately after the encoder has been trained.

[0022] In the apparatus of an exemplary embodiment of the present invention, the MLP is capable of receiving additional structured data input information that is different from the first numerical vector. This additional structured data input information may include one or more of the following: age, gender, biosignals, numerical detection results, natural language information transformed into structured data, and structured data information converted from biosignal data of a different type than the bioelectrical signal data acquired from the acquisition unit. The biosignals may include one or more selected from the group consisting of blood pressure, pulse rate, body temperature, respiratory rate, and oxygen saturation. This additional structured data input information may be concatenated with or input separately from the first numerical vector.

[0023] The apparatus of an exemplary embodiment of the present invention may further include a display unit that, when the MLP predicts whether a specific disease will occur, simultaneously displays, according to the baseline risk probability, the marginal probability of the specific disease occurring considering the acquired bioelectrical signal data and the marginal probability of the specific disease occurring without considering the acquired bioelectrical signal data, and displays how many times the marginal probability of the specific disease occurring considering the acquired bioelectrical signal data has increased compared to the marginal probability of the specific disease occurring without considering the acquired bioelectrical signal data.

[0024] In an illustrative example of the apparatus of the present invention, the deep learning algorithm of the encoder is based on a convolutional neural network (CNN), and the apparatus may include an encoder subunit.

[0025] In an exemplary embodiment of the apparatus of the present invention, the encoder subunit includes: one or more convolutional layers; one or more fully connected layers, the fully connected layers including nonlinear activation functions; and a concentration layer that summarizes the feature sets extracted from the bioelectrical signal data of each channel to extract representative values ​​respectively, and readjusts the feature sets of each channel to reflect the contribution of the feature sets of each channel based on the representative values, the feature sets including features related to the anatomical structure of each channel, and the readjusted feature sets of each channel are more focused on the anatomical features of each channel compared to the aforementioned feature sets.

[0026] In an illustrative example of the apparatus of the present invention, the one or more convolutional layers may include depthwise-separable convolutional layers that individually convolve the bioelectrical signal data of the one or more channels.

[0027] In an exemplary embodiment of the apparatus of the present invention, the aforementioned centralized layer may perform pooling processing on the aforementioned feature set in order to aggregate the aforementioned feature set.

[0028] In the apparatus of an exemplary embodiment of the present invention, the aforementioned centralized layer enables the representative value of each channel to be used to calculate the contribution of each channel through the aforementioned fully connected layer, and the contribution of each channel is multiplied by the aforementioned feature set to readjust the feature set of each channel.

[0029] In the apparatus of an exemplary embodiment of the present invention, the aforementioned central layer can scale the representative values ​​of each channel to a value within a specific range through the result of the fully connected layer, thereby calculating the contribution of each channel.

[0030] In an exemplary embodiment of the apparatus of the present invention, the encoder subunit includes a sqeeze-excitation layer that calculates a scalar value by averaging the values ​​of each channel, where the scalar values ​​of each channel are between 0 and 1, and are scaled according to the importance of the channels. After the vector of the scalar values ​​of the channels is passed through a fully connected layer, an activation function (sigmoid / RELU) can be applied to reduce the dimensionality.

[0031] In an exemplary embodiment of the apparatus of the present invention, the encoder includes a plurality of convolutional blocks, and the sub-units may be included in the remaining convolutional blocks other than the first convolutional layer.

[0032] In the apparatus of an exemplary embodiment of the present invention, the convolutional block includes a first encoder subunit and a second encoder subunit. Compared to the second encoder subunit, the first encoder subunit is closer to the input end of the convolutional block than to its output end. In the work of summarizing the feature set to extract representative values ​​and in the work of readjusting based on the contribution of each channel, the concentrating layer focuses more on summarizing the feature set to extract representative values ​​than the second encoder subunit. Compared to the representative values ​​of the second encoder subunit, the representative values ​​of the first encoder subunit reflect more features related to the anatomical structure. Compared to the first encoder subunit, the second encoder subunit is closer to the output end of the convolutional block than to its input end. In the work of summarizing the feature set to extract representative values ​​and in the work of readjusting based on the contribution of each channel, the concentrating layer focuses more on readjusting based on the contribution of each channel than the first encoder subunit.

[0033] In an exemplary embodiment of the apparatus of the present invention, the final convolutional block of the encoder further includes a non-local network, which can achieve spatial attention by comparing the similarity between spatial points of the bioelectrical signal data.

[0034] In the apparatus of an exemplary embodiment of the present invention, the aforementioned bioelectric signal data is a single-channel or multi-channel signal, and the bioelectric signal input to the encoder can be a two-dimensional array of C×M (number of input leads (channels) × number of measurements of each channel).

[0035] In an exemplary embodiment of the apparatus of the present invention, the aforementioned bioelectric signal data is a bioelectric signal image, which shows the bioelectric signals of all channels on a two-dimensional plane. The bioelectric signal image is converted into an image including all lead channels or an image of each lead channel cropped according to each lead channel. The bioelectric signal image input to the encoder can be converted into a black and white image of one or more lead channels or a three-dimensional array with three channels: red (R), green (G), and blue (B) in a C×W×H (number of channels × number of horizontal pixels × number of vertical pixels) format.

[0036] In an exemplary embodiment of the apparatus of the present invention, the analysis results include disease prediction. When the analysis unit predicts a disease, the bioelectrical signal data is single-channel bioelectrical signal data. When the bioelectrical signal data is electrocardiogram (ECG) data, the disease may include shock, respiratory failure, cardiac arrest, acute coronary syndrome, myocardial infarction, or hyperkalemia.

[0037] In an exemplary embodiment of the apparatus of the present invention, the analysis result may include disease diagnostic aid information for determining whether the disease is improving or worsening using the first numerical vector. When the analysis unit provides the disease diagnostic aid information, the bioelectrical signal data consists of multiple bioelectrical signal data measured at predetermined intervals. These multiple bioelectrical signal data are then passed through the pooling layer of the encoder to provide diagnostic aid information for determining whether the disease is improving or worsening from the acquired first numerical vector.

[0038] In the apparatus of an exemplary embodiment of the present invention, the analysis results include providing disease diagnosis assistance information. The bioelectrical signal data are multiple bioelectrical signal data measured at predetermined or irregular time intervals. The analysis unit can also be trained in the following manner: first numerical vectors of the multiple bioelectrical signal data are arranged into sequential vectors, and the sequential vectors are concatenated along the length direction of the vectors to pass through a multilayer perceptron (MLP) network; or, the sequential vectors are concatenated along the perpendicular direction of the vector length to pass through a transformer network; or, the sequential vectors are not concatenated and are passed sequentially through a recurrent neural network (RNN) to extract a second numerical vector. Based on the extracted second numerical vector, it is determined whether the patient's specific disease improves or worsens over time. In this case, the vector input for training the analysis unit can also be a vector resulting from the concatenation of time-related information encoded values ​​and the first numerical vector.

[0039] In an exemplary embodiment of the device of the present invention, the encoder described above can be trained based on clinically defined morphological characteristics in the properties of bioelectrical signal data and through self-supervised learning.

[0040] In an exemplary embodiment of the device of the present invention, the encoder can convert bioelectrical signal data in a specific manner as training data and train it through self-supervised learning.

[0041] In an exemplary embodiment of the apparatus of the present invention, the encoder can use enhanced bioelectrical signal data as training data and train it through unsupervised learning. Enhanced bioelectrical signal data that share the original bioelectrical signal data are input into the encoder, and the encoder is adjusted so that the calculated first numerical vectors are the same or have high similarity.

[0042] In the apparatus of an exemplary embodiment of the present invention, the above-described adjustment, which makes the calculated first numerical vectors identical or highly similar, means minimizing the distance between the calculated first numerical vectors.

[0043] In one exemplary embodiment of the present invention, the device can be combined with a smartwatch, a medical device or sports equipment equipped with a bioelectric signal measuring device, or a device with a smartphone application or electronic health record system.

[0044] Preferably, in an exemplary embodiment of the present invention, the bioelectrical signal in the above device is particularly preferably an electrocardiogram (ECG).

[0045] On the other hand, according to another embodiment, a method or apparatus for converting bioelectrical signal data into a numerical vector, executed by a processor, is described. The method for analyzing diseases using deep learning based on the bioelectrical signal data includes the following steps: acquiring bioelectrical signal data; inputting the bioelectrical signal data into an encoder; and calculating a first numerical vector using a deep learning algorithm through the encoder. The first numerical vector is structured data related to features extracted from the bioelectrical signal data, including, in particular, at least one of anatomical (location) features and temporal features that can be extracted from the bioelectrical signal data. This first numerical vector can be effectively used for downstream tasks or machine learning described later.

[0046] In one exemplary embodiment of the present invention, the above method may further include the following steps: using a first numerical vector to perform disease or health-related analysis, prediction, or provide diagnostic assistance information.

[0047] In one exemplary embodiment of the present invention, the method may include the following steps: simultaneously processing multiple downstream tasks using the first numerical vector. As error signals from the outputs of each downstream task network converge at the end of an encoder due to backpropagation, the versatility of the first numerical vector can be improved by training an encoder.

[0048] In one exemplary embodiment of the method of the present invention, the first numerical vector itself may be used as an input vector for a downstream task processing step or concatenated with additional structured data information as an input vector for a downstream task processing step.

[0049] In one exemplary embodiment of the present invention, the encoders may be two or more, and an input numerical vector may be generated by concatenating multiple first numerical vectors output from each encoder.

[0050] In one exemplary embodiment of the present invention, N consecutive bioelectrical signal data can be passed through an encoder to obtain N consecutive first numerical vectors.

[0051] In an exemplary embodiment of the present invention, after dividing bioelectrical signal data according to a predetermined time interval, the method can provide relevant analysis, diagnosis or prediction of a specific disease based on the time-point weighted average of the result values ​​at each time point obtained by the encoder or the encoder and downstream task processing of the divided data intervals.

[0052] In an exemplary embodiment of the present invention, when training the network of a downstream task, the method can modify (update) the network weights of the downstream task by training after fixing the network weights of the encoder, and modify (update) the overall weights of the encoder network and the network of the downstream task by supplementing training.

[0053] In an exemplary embodiment of the present invention, the aforementioned multiple downstream task processing can be performed by a multi-layer perceptron (MLP) having two or more fully connected layers.

[0054] In one exemplary embodiment of the method of the present invention, the MLP can be trained together with the encoder's coding network through (joint) multi-task learning, or it can be trained separately after the encoder has been trained.

[0055] In an exemplary embodiment of the method of the present invention, the MLP is capable of receiving additional structured data input information that is different from the first numerical vector. This additional structured data input information may include one or more of the following: age, gender, biosignals, numerical detection results, natural language information transformed into structured data, and structured data information converted from biosignal data of a different type than the bioelectrical signal data acquired from the acquisition unit. The biosignals may include one or more selected from the group consisting of blood pressure, pulse rate, body temperature, respiratory rate, and oxygen saturation. This additional structured data input information may be concatenated with or input separately from the first numerical vector.

[0056] In an exemplary embodiment of the method of the present invention, when the MLP predicts whether a specific disease will occur, when the MLP outputs, it can simultaneously display, according to the baseline risk probability, the marginal probability of the specific disease occurring considering the acquired bioelectrical signal data and the marginal probability of the specific disease occurring without considering the acquired bioelectrical signal data, and display how many times the marginal probability of the specific disease occurring considering the acquired bioelectrical signal data has increased compared to the marginal probability of the specific disease occurring without considering the acquired bioelectrical signal data.

[0057] In one exemplary embodiment of the present invention, the deep learning algorithm of the encoder described above is based on CNN and may include encoder subunits.

[0058] In an exemplary embodiment of the method of the present invention, the encoder subunit includes: one or more convolutional layers; one or more fully connected layers, the fully connected layers including nonlinear activation functions; and a concentrating layer that summarizes the feature sets extracted from the bioelectrical signal data of each channel to extract representative values ​​respectively, and readjusts the feature sets of each channel to reflect the contribution of the feature sets of each channel based on the representative values, the feature sets including features related to the anatomical structure of each channel, and the readjusted feature sets of each channel can be more focused on the anatomical features of each channel compared to the aforementioned feature sets.

[0059] In one exemplary embodiment of the present invention, the one or more convolutional layers may include depthwise-separable convolutional layers that individually convolve the bioelectrical signal data of the one or more channels.

[0060] In one exemplary embodiment of the method of the present invention, the aforementioned centralized layer may perform pooling processing on the aforementioned feature set in order to aggregate the aforementioned feature set.

[0061] In an exemplary embodiment of the method of the present invention, the above-mentioned centralized layer enables the representative value of each channel to be used to calculate the contribution of each channel through the above-mentioned fully connected layer, and the contribution of each channel is multiplied by the above-mentioned feature set to readjust the feature set of each channel.

[0062] In an exemplary embodiment of the present invention, the aforementioned centralized layer can calculate the contribution of each channel by scaling the representative values ​​of each channel through the fully connected layer to a value within a specific range.

[0063] In an exemplary embodiment of the method of the present invention, the encoder subunit includes a sqeeze-excitation layer that calculates a scalar value by averaging the values ​​of each channel, where the scalar values ​​of each channel are between 0 and 1, and are scaled according to the importance of the channels. After the vector of the scalar values ​​of the channels is passed through a fully connected layer, an activation function (sigmoid / RELU) can be applied to reduce the dimensionality.

[0064] In one exemplary embodiment of the method of the present invention, the encoder includes a plurality of convolutional blocks, and the sub-units may be included in the remaining convolutional blocks other than the first convolutional layer.

[0065] In an exemplary embodiment of the method of the present invention, the convolutional block includes a first encoder subunit and a second encoder subunit. Compared to the second encoder subunit, the first encoder subunit is closer to the input end of the convolutional block than to its output end. In the work of summarizing the feature set to extract representative values ​​and the work of readjusting based on the contribution of each channel in the ensemble layer, the ensemble layer focuses more on summarizing the feature set to extract representative values ​​than the second encoder subunit. Compared to the representative values ​​of the second encoder subunit, the representative values ​​of the first encoder subunit reflect more features related to the anatomical structure. Compared to the first encoder subunit, the second encoder subunit is closer to the output end of the convolutional block than to its input end. In the work of summarizing the feature set to extract representative values ​​and the work of readjusting based on the contribution of each channel in the ensemble layer, the ensemble layer focuses more on readjusting based on the contribution of each channel than the first encoder subunit.

[0066] In one exemplary embodiment of the method of the present invention, the final convolutional block of the encoder further includes a non-local network, which can achieve spatial attention by comparing the similarity between spatial points of the bioelectrical signal data.

[0067] In one exemplary embodiment of the present invention, the above-mentioned bioelectric signal data is a single-channel or multi-channel signal, and the bioelectric signal input to the encoder can be a two-dimensional array of C×M (number of input leads (channels) × number of measurement values ​​of each channel).

[0068] In an exemplary embodiment of the present invention, the aforementioned bioelectric signal data is a bioelectric signal image, which shows the bioelectric signals of all channels on a two-dimensional plane. The bioelectric signal image is converted into an image including all lead channels or an image of each lead channel cropped according to each lead channel. The bioelectric signal image input to the encoder can be converted into a black and white image of one or more lead channels or a three-dimensional array with three channels: R (Red), G (Green), and B (Blue) in the form of C×W×H (number of channels × number of horizontal pixels × number of vertical pixels).

[0069] In an exemplary embodiment of the method of the present invention, the above analysis results include disease prediction. When the analysis unit predicts a disease, the above bioelectrical signal data is single-channel bioelectrical signal data. When the above bioelectrical signal data is electrocardiogram (ECG) data, the above disease may include shock, respiratory failure, cardiac arrest, acute coronary syndrome, myocardial infarction, or hyperkalemia.

[0070] In an exemplary embodiment of the method of the present invention, the analysis result may include disease diagnostic aid information for determining whether the disease is improving or worsening using the first numerical vector. When providing the disease diagnostic aid information, the bioelectrical signal data consists of multiple bioelectrical signal data measured at predetermined intervals, which can be passed through the pooling layer of the encoder to provide diagnostic aid information for determining whether the disease is improving or worsening from the acquired first numerical vector.

[0071] In an exemplary embodiment of the method of the present invention, the analysis result includes providing auxiliary information for disease diagnosis, wherein the bioelectrical signal data is a plurality of bioelectrical signal data measured at predetermined time intervals or irregular time intervals. The analysis unit can also be trained in the following manner: first numerical vectors of the plurality of bioelectrical signal data are arranged into sequential vectors, and the sequential vectors are concatenated along the length of the vectors to pass through a multilayer perceptron (MLP) network; or, the sequential vectors are concatenated along the perpendicular direction of the vector length to pass through a transformer network; or, the sequential vectors are not concatenated and are passed sequentially through an RNN, thereby extracting a second numerical vector, and determining whether the patient's specific disease improves or worsens over time based on the extracted second numerical vector. In this case, the vector input for training the analysis unit can also be a vector resulting from the concatenation of time-related information encoded values ​​and the first numerical vector.

[0072] In one exemplary embodiment of the present invention, the encoder described above can be trained based on clinically defined morphological characteristics of bioelectrical signal data and through self-supervised learning.

[0073] In one exemplary embodiment of the present invention, the encoder described above can convert bioelectrical signal data in a specific manner as training data and train it through self-supervised learning.

[0074] In an exemplary embodiment of the present invention, the encoder described above can use enhanced bioelectrical signal data as training data and be trained by unsupervised learning. The method may include the following steps: inputting enhanced bioelectrical signal data that share the original bioelectrical signal data into the encoder respectively, and adjusting them so that the calculated first numerical vectors are the same or have high similarity.

[0075] In an exemplary embodiment of the method of the present invention, the step of adjusting the calculated first numerical vectors to be identical or having high similarity can refer to minimizing the distance between the calculated first numerical vectors.

[0076] In one exemplary embodiment of the present invention, the bioelectrical signal in the above method is particularly preferably an electrocardiogram (ECG).

[0077] On the other hand, according to another embodiment, an exemplary example of the present invention provides a computer-readable recording medium that can be read by a computer and stores the aforementioned computer-executable program instructions. When the processor of the computer executes the program instructions, the processor performs the method of the aforementioned example.

[0078] The effects of the invention

[0079] According to exemplary embodiments of the present invention, structured numerical vectors can be extracted from unstructured bioelectrical signal data such as electrocardiograms and applied to various clinical conditions.

[0080] In particular, by directly applying it to existing clinical frameworks, universal numerical information can be extracted to maximize the application scope of bioelectrical signal information such as electrocardiogram (ECG) data. This universal numerical information (embedded vector) can be used not only independently but also in conjunction with other patient information. Furthermore, changes in patient condition can be easily quantified through the numericalization of bioelectrical signal data such as ECG data. Therefore, it can be applied not only to patient triage in emergency rooms or disaster relief sites but also to various medical devices. Moreover, by applying structured numerical vectors to existing artificial intelligence algorithms to easily assess and predict changes in patient condition, it can provide various diagnostic auxiliary information related to the heart, lungs, and blood flow, in addition to acute myocardial infarction or arrhythmia, especially various diagnostic auxiliary information related to bioelectrical signals such as ECG.

[0081] The effects of this invention are not limited to those mentioned above. Those skilled in the art to which this invention pertains can clearly understand other effects not mentioned through the content described in the claims. Attached Figure Description

[0082] Hereinafter, to further clarify exemplary embodiments of the present invention, the accompanying drawings required for illustrating the embodiments are briefly described. It should be understood that the following drawings are only for illustrating embodiments of this specification and are not intended to limit the scope of the invention. Furthermore, to ensure clarity of description, some elements shown in the drawings may be modified in various ways, for example, they may be enlarged or omitted.

[0083] Figure 1 This is a simplified diagram illustrating an embodiment of the present invention of a device for converting electrocardiogram data into numerical vectors to analyze diseases.

[0084] Figure 2 This is a flowchart illustrating a method for converting electrocardiogram data into numerical vectors to analyze diseases, according to an embodiment of the present invention.

[0085] Figure 3 A diagram illustrating an ECG encoder subunit according to an embodiment of the present invention.

[0086] Figure 4 A diagram illustrating an embodiment of the ECG encoder of the present invention.

[0087] Figure 5 A graph illustrating the application of another embodiment of the invention, showing a numerical vector obtained from multiple electrocardiogram data acquired through repeated measurements.

[0088] Figure 6 A diagram illustrating the application of N consecutively acquired numerical vectors in another embodiment of the present invention. Detailed Implementation

[0089] Hereinafter, some embodiments of the present invention will be described in detail with reference to the illustrative drawings. In assigning reference numerals to structural elements in the various drawings, the same structural elements may have the same reference numerals even if they are shown in different drawings. Furthermore, in describing these embodiments, detailed descriptions of known structures or functions may be omitted if it is determined that such detailed descriptions may obscure the main idea of ​​the present invention.

[0090] the term

[0091] When this specification uses terms such as "including," "having," or "consisting of," it implies the presence of other parts, unless "only" is used. Unless there is a particularly explicit description of a structural element, singular expressions may include plural expressions.

[0092] Furthermore, when describing the structural elements of the present invention, terms such as "first," "second," "A," "B," "(a)," and "(b)" may be used. Unless otherwise defined, such terms are only used to distinguish the structural element from other structural elements, and such terms do not limit the nature, importance, order, or number of the corresponding structural elements.

[0093] In this specification, the term "learning" refers to performing machine learning through computation.

[0094] In this specification, "network" refers to a neural network of a machine learning algorithm or model.

[0095] In this specification, terms such as “unit,” “module,” “device,” or “system” refer not only to hardware but also to a combination of software driven by the corresponding hardware. For example, hardware can be a data processing device including a central processing unit (CPU), a graphics processing unit (GPU), or other processors. Furthermore, software can refer to a running process, object, executable file, thread of execution, program, etc.

[0096] In this specification, numerical vectors or numerical vector information, as structured numerical coordinates, have a consistent structure and / or meaning form formed through deep learning algorithms, to be applied to one or more machine learning services or tasks, and are associated with features extracted from bioelectrical signal data (reflecting the corresponding features).

[0097] Converting specific data into a numerical vector refers to transforming unstructured data, such as electrocardiograms (ECGs), which have various formats (electrical signals themselves or their outputs) and sizes (channel types, number, and measurement time), into a numerical vector (or array) smaller than the original data, with a constant length (in the case of an array, forming a defined dimension and size), and whose elements contain a consistent meaning for each location. This consistently represents the position of a specific ECG within the vector space defined by its elements. This abstract coordinate information can be applied to various downstream tasks in multiple ways (algorithms).

[0098] In this specification, the network structures used (in particular, including compressed excitation networks and non-void network structures) have characteristics that allow such numerical vectors to be correlated with features extracted from bioelectrical signal data, which includes anatomical (locational) features and / or temporal features of various channels of bioelectrical signal data.

[0099] Taking electrocardiogram (ECG) as an example, the anatomical or anatomical spatial characteristics (the space defined by the anatomical position of the electrodes used to measure the ECG) and / or temporal or temporal spatial characteristics (the space defined by the overall time interval of the ECG signal) of the ECG signal are broadly and effectively reflected. In the above-mentioned extensive feature extraction process, the characteristics of the training method (assisted learning based on multi-task learning) are conducive to effectively extracting highly generalizable features from multiple tasks.

[0100] This not only makes training new downstream tasks very simple, but also helps to extract high-quality numerical vectors for simplifying few-shot or one-shot learning.

[0101] In this specification, the anatomical (location) features or anatomical spatial features of a bioelectric signal refer to features related to the anatomical location or structure of the bioelectric signal (the object being measured) representing the individual channel or lead.

[0102] Taking an electrocardiogram (ECG) as an example, the different electrodes (leads) on an ECG are responsible for different anatomically located parts of the heart, and the correlation analysis of the morphological features observed in each lead can also differ. For example, although a Q wave indicates myocardial infarction, normal values ​​are often observed in leads I, aVL, V5, and V6. Considering this anatomical location, it is particularly important to consider the characteristics of other locations along the choroid when analyzing the characteristics of a specific location (lead). For example, in the anterior precordial leads V1, V2, and V3, when the ST segment shows a characteristic of rising from the baseline, in order to quantify the clinical meaning of this characteristic (myocardial infarction), the reciprocal change (ST segment change) observed in the inferior leads (II, III, aVF) should be considered to strengthen or weaken the numerical reflection of the corresponding characteristic.

[0103] This contextual consideration—that is, taking into account the characteristics of other locations (leads) in the context when analyzing the characteristics of specific locations (leads) of a bioelectrical signal—can be achieved through the squeeze-excitation network structure illustrated in the illustrative examples of this invention. At this level, the illustrative examples of this invention show, particularly in the context, the inclusion of anatomical (locational) features that can be extracted from bioelectrical signal data.

[0104] In this specification, the temporal or temporal-spatial characteristics of bioelectric signals refer to the characteristics related to the distribution of the morphological properties of bioelectric signals within a one-dimensional time interval in which the bioelectric signals are measured.

[0105] For example, in the case of electrocardiography (ECG), ECGs are measured over various time periods, ranging from several days to several hours or days. In this variable one-dimensional space, the ECG waveforms, consisting of P waves, QRS waves, and T waves, can be batched at various frequencies, either regularly or irregularly, depending on the patient's condition. Considering this batching characteristic, it is particularly important to consider (contextually) other characteristics that are further apart when clinically analyzing a particular characteristic within the corresponding one-dimensional space. For example, the diagnosis of a life-threatening arrhythmia called complete atrioventricular block can be considered. Typically, arrhythmias are diagnosed using the temporal characteristics related to the batching status of each characteristic in the aforementioned one-dimensional space. In complete atrioventricular block, the P wave and QRS wave are not synchronized, exhibiting their own velocities. In the diagnostic process, it is important to determine whether a small deflection of a particular QRS wave is a P wave overlapping with the QRS wave. Therefore, it is particularly important to determine which of the various morphological features of the P wave, which could be QRS deflection or other locations, is actually a P wave. Without considering the connections between the corresponding characteristics, the above process cannot be achieved.

[0106] In an exemplary embodiment of the invention, to address the problem of extracting temporal characteristics from contexts that are geographically distant, a non-local network structure is applied. At this level, the exemplary embodiment of the invention represents the inclusion of temporal features, particularly at the context level, that can be extracted from bioelectrical signal data.

[0107] In this specification, numerical vectors can be represented as first numerical vectors, second numerical vectors, etc. For example, a first numerical vector refers to the result calculated from the encoder using a deep learning algorithm, while a second numerical vector refers to the result calculated using the first numerical vector along with downstream tasks and additional machine learning algorithms. For example, in some figures, the sequential vectors included in the first numerical vector can be represented as vector1, vector2, vector3, etc.

[0108] In this specification, embedding refers to the process of converting unstructured data such as electrocardiograms into the aforementioned numerical vectors, or the result of its calculation (the numerical vectors themselves).

[0109] In this specification, the generality of a numerical vector means that it can be used for machine learning purposes other than a specific one, and preferably, it can be used for multiple machine learning purposes. That is, the numerical vector includes the morphological characteristics of a specific bioelectrical signal (e.g., electrocardiogram), preferably in a comprehensive and / or effective manner, thus indicating not only that it has been applied, but also that it can be effectively applied to unknown downstream tasks that may be applied in the future, preferably effectively applied to two or more downstream tasks, and more preferably effectively applied to most downstream tasks.

[0110] For example, to facilitate understanding, the following example illustrates a numerical vector lacking generality. Suppose that in a numerical vector of 100 elements, 3 elements possess characteristics effective for diagnosing specific diseases such as myocardial infarction, while the remaining 97 elements contain redundant or noise-related information. In this case, the numerical vector can be considered unapplicable to downstream tasks other than diagnosing myocardial infarction and lacks generality. To populate the elements of these vectors with meaningful information, multiple clinical diagnostic tasks could be performed simultaneously, rather than just one diagnosis. However, this only encodes characteristics of the numerical vector relevant to the trained diagnosis, making it difficult to use for unknown downstream tasks.

[0111] Conversely, in the exemplary embodiments of the present invention, compressed excitation networks and non-local networks can improve the versatility by enhancing the scope and quality of specific information that the numerical vectors may include, as described above. Furthermore, in the exemplary embodiments of the present invention, the versatility of the numerical vectors can be further increased by additionally applying 1) supervised learning based on existing clinically defined morphological features, and 2) self-supervised learning that learns ECG morphological features unrelated to clinical information. Moreover, in order to effectively batch process information within the vector space defined by the numerical vectors, the versatility of the numerical vectors can be further increased by additionally performing 3) unsupervised learning, described later.

[0112] In this specification, unstructured data, as the measured numerical dataset, refers to 1) data with a non-constant number of dimensions and / or size, 2) data with inconsistent location-based numerical analysis, or 3) data that requires simple transformation due to its large size or complexity.

[0113] In contrast, in this specification, structured data refers to data with a constant number of dimensions and size. The analysis of each value in structured data is based on a uniform location, and its size is no larger than that of unstructured data (the number of elements is not excessive) and it is simple. Compared to unstructured data, machine learning algorithms for downstream tasks can be trained with only a small amount of data. For example, an electrocardiogram (ECG), which is equivalent to converting it into numerical vectors through embedding, can include tabular data such as the patient's age, gender, blood pressure, pulse rate, respiratory rate, and body temperature.

[0114] In this specification, downstream tasks refer to one or more machine learning operations that utilize numerical vectors obtained through embedding. These may include, as described below, 1) supervised learning, 2) unsupervised learning, 3) self-supervised learning, 4) clustering, and 5) anomaly detection, etc.

[0115] In this specification, the disease analysis method or disease analysis device includes analyzing, predicting disease or health, and providing relevant diagnostic information about the disease.

[0116] In this specification, ECG refers to an electrocardiogram or electrocardiogram signal, which records the electrical activity of the heart according to a numerical waveform. More specifically, it refers to the waveform consisting of the vector sum of action potentials generated by the heart's special excitatory and conductive system. That is, it refers to the signal obtained by measuring the vector and signal of action potentials generated by various structural elements of the heart, such as the sinoatrial node (SA node), atrioventricular node (AV node), His bundle, and Purkinje fibers, from electrodes in contact with the outside of the body. Typically, two or more electrodes are used when measuring an ECG, forming electrodes in pairs. The above-mentioned ECG signals, electrocardiograms, or electrocardiogram signals can be used interchangeably, and even when used interchangeably, they can each represent different meanings.

[0117] Explanation of exemplary examples

[0118] In exemplary embodiments of the present invention, single-channel or multi-channel bioelectrical signal data can be used. In particular, a deep learning-based artificial intelligence algorithm can be applied to single-channel or multi-channel electrocardiogram signals or black-and-white or color electrocardiogram images obtained based on the aforementioned electrocardiogram signals to extract numerical vector information applicable to various clinical conditions. In particular, general numerical vector information can be extracted.

[0119] The acquired numerical vector information, especially that obtained from electrocardiograms, can be used to diagnose or predict, either individually or in a single instance, whether 1) heart rate abnormalities (tachycardia, bradycardia, various arrhythmias), 2) structural and functional abnormalities of the heart (heart failure, cardiac tamponade, valvular stenosis / failure, pulmonary hypertension, pulmonary embolism, cardiomyopathy), 3) various clinical events (shock, respiratory failure, cardiac arrest, cardiogenic syncope and sudden death, acute coronary syndrome, myocardial infarction, hyperkalemia) or whether 4) various clinical treatments (administration of vasopressors, use of mechanical ventilation, coronary angiography and blood transfusion, etc.) will occur within a specified period.

[0120] Furthermore, in addition to bioelectrical signal data such as electrocardiogram data, other structured information (age, gender, blood pressure, pulse rate, respiratory rate, body temperature, numerical test results, etc.) and unstructured information that has been structured through appropriate transformation (main symptoms, underlying diseases, text, radiographic and ultrasound image information, stethoscope sounds and other sound information, and various biological signals) can also be concatenated with corresponding numerical vectors to improve the accuracy of diagnosis.

[0121] The algorithms in exemplary embodiments of the present invention may include partial algorithms for processing additional information other than deep learning algorithm components such as CNN (convolutional neural network) and / or bioelectrical signal data such as electrocardiogram data.

[0122] Furthermore, in exemplary embodiments of the present invention, bioelectrical signal data such as electrocardiogram data can be acquired to provide auxiliary information for the analysis, prediction, and diagnosis of diseases.

[0123] In an exemplary embodiment of the present invention, an apparatus for converting bioelectrical signal data into a numerical vector may include: an acquisition unit for acquiring bioelectrical signal data; and an encoder for receiving the bioelectrical signal data and calculating a numerical vector (which may be referred to as a first numerical vector) using a deep learning algorithm.

[0124] In an exemplary embodiment of the present invention, an apparatus for analyzing diseases by converting bioelectrical signal data into numerical vectors may include: an acquisition unit for acquiring bioelectrical signal data; an encoder for receiving the bioelectrical signal data and calculating a numerical vector (which may be referred to as a first numerical vector) using a deep learning algorithm; and an analysis unit for providing disease-related analytical information, predictive information, or diagnostic assistance information using the numerical vector.

[0125] In an exemplary embodiment of the present invention, a method for converting bioelectrical signal data into a numerical vector is executed by a processor and may include the following steps: acquiring bioelectrical signal data; inputting the bioelectrical signal data into an encoder; and calculating a numerical vector (which may be referred to as a first numerical vector) using the encoder and a deep learning algorithm.

[0126] In an exemplary embodiment of the present invention, a method for analyzing diseases from bioelectrical signal data using deep learning is executed by a processor. The provided disease analysis method includes the following steps: acquiring bioelectrical signal data; inputting the bioelectrical signal data into an encoder; calculating a numerical vector (which may be referred to as a first numerical vector) using the encoder and a deep learning algorithm; and using the numerical vector to perform disease-related analysis, prediction, or provide diagnostic assistance information.

[0127] In exemplary embodiments of the present invention, the bioelectrical signal may be an electroencephalogram (EEG), an electromyogram (EMG), an electrooculogram (EOG), or an electrocardiogram (ECG), as described below, and in particular, an electrocardiogram (ECG) is preferred.

[0128] In an exemplary embodiment of the present invention, the above-mentioned numerical vector can also be used for downstream tasks.

[0129] As described above, because multiple tasks are performed simultaneously, the error signals from the outputs of each downstream task network are backpropagated and converged at the end of a single encoder, thus training an encoder. Consequently, digital vectors can be transformed into numerical vectors with improved versatility.

[0130] In one exemplary embodiment of the present invention, the first numerical vector itself can be used as an input vector for a downstream processing unit or concatenated with other structured data information as an input vector for a downstream processing unit.

[0131] The aforementioned additional structured data information may include at least one of the following: biological signals (vital signs) such as age, gender, blood pressure, pulse rate, body temperature, respiratory rate, and oxygen saturation; existing structured data information such as various numerical test results; unstructured data (images, sounds, biological signals, etc., where the corresponding biological signals are bioelectrical signals and other biological information input to the encoder to obtain the first numerical vector) converted into structured data through machine learning methods; and natural language information such as symptoms, disease names, and medical records transformed into structured data through natural language processing.

[0132] In one exemplary embodiment of the invention, the encoder described above can be two or more, and multiple first numerical vectors output from each encoder can be concatenated to generate an input numerical vector. The corresponding network can be learned by setting the corresponding input numerical vector as the input value of the downstream task network and setting the diagnosis to be predicted as the output value of the corresponding downstream task network.

[0133] In one exemplary embodiment of the invention, N consecutive bioelectrical signal data can be passed through an encoder to obtain N consecutive first numerical vectors. These N consecutive first numerical vectors can be used as input values ​​for learning downstream tasks, such as predicting whether a particular disease will improve or worsen over time, predicting the severity of a particular disease, or predicting the occurrence of clinical events.

[0134] In one exemplary embodiment of the present invention, after dividing bioelectrical signal data according to a specified time interval, the device can provide relevant analysis, diagnosis or prediction of a specific disease based on the time-point weighted average of the result values ​​at each time point obtained by the encoder or the encoder and downstream processing of the information of each divided data interval.

[0135] In one exemplary embodiment of the present invention, the aforementioned numerical vector conversion device or disease analysis device may include a downstream processing unit or processing step that processes a downstream task using numerical vectors. The downstream task may mean processing multiple tasks, each of which may execute a multi-layer perceptron (MLP) having two or more fully connected layers.

[0136] In an exemplary embodiment of the present invention, when the MLP predicts whether a specific disease will occur, when the MLP outputs, it can simultaneously display the marginal probability of the disease occurring considering bioelectrical signal data and the marginal probability of the disease occurring without considering bioelectrical signal data, according to the baseline risk probability, and display how many times the marginal probability of the disease occurring considering the bioelectrical signal data has increased compared to the marginal probability of the disease occurring without considering the bioelectrical signal data.

[0137] In one exemplary embodiment of the present invention, the MLPs for each task are trained jointly with the encoder's coding network, or the encoder is trained separately after the encoder has been trained.

[0138] In an exemplary embodiment of the present invention, the encoder is based on a CNN and includes an encoder subunit, which may include a depthwise-separable convolution layer that convolutions the bioelectric signal data separately for each channel.

[0139] In one exemplary embodiment of the present invention, the encoder subunit applies a squeeze-excitation mechanism to extract a numerical value (average or highest value) for each channel. This causes the numerical vector to pass through a network consisting of two or more fully connected layers, including non-linear activation functions such as ReLU, and then the sigmoid function is applied to obtain values ​​between 0 and 1 for each channel. These values ​​are then multiplied by the corresponding channel to recalibrate the features of each channel.

[0140] In one exemplary embodiment of the present invention, the encoder may include a plurality of convolutional blocks, the convolutional blocks including a first convolutional layer and a plurality of encoder subunits.

[0141] In one exemplary embodiment of the invention, the final convolutional block of the encoder may further include a non-local network. When this non-local network (or non-local neural network) encodes information at a specific location (in the case of an image, a spatial point; in the case of an electrocardiogram signal, a temporal point), it utilizes the characteristics of all locations (spatial / temporal points) of the input data. In this process, the contribution of each location is different, and this contribution depends on the attention mechanism.

[0142] In one exemplary embodiment of the present invention, the MLP for each of the above tasks can not only receive the numerical vector output by the ECG encoder, but also receive other additional structured data input information. This additional input information may include at least one of the following: biosignals such as age, gender, blood pressure, pulse rate, body temperature, respiratory rate, accompanying symptoms, and oxygen saturation; various numerical test results; and unstructured data (images, sound, biosignals, etc.) converted into structured numerical information.

[0143] In the exemplary embodiments of the present invention, the above-mentioned device may be a bioelectric signal measuring device or an analysis device. For example, it may be a smartwatch, a medical device equipped with a bioelectric signal measuring device, a treadmill or other exercise machine, or an EHR (electronic health records) system with an application, a camera or scanning device and an analysis algorithm, etc., but is not limited thereto.

[0144] Furthermore, in an exemplary embodiment of the present invention, a computer-readable recording medium is provided, which can be read by a computer and stores the aforementioned computer-executable program instructions. When the processor of the computer executes the aforementioned program instructions, the corresponding processor performs the aforementioned numerical vector conversion method or disease analysis method.

[0145] Description of preferred embodiments

[0146] As described above, the method and apparatus for converting bioelectrical signal data into numerical vectors, as well as the method and apparatus for analyzing diseases using the above-described method and apparatus, are effective in cases where the bioelectrical signal is an electrocardiogram (ECG). Therefore, preferred embodiments will be described in further detail with a focus on the electrocardiogram.

[0147] Figure 1 A simplified diagram is provided to illustrate an embodiment of the present invention of a device 1 (hereinafter referred to as a "disease analysis device") that converts electrocardiogram data into numerical vectors for disease analysis.

[0148] Reference Figure 1 A disease analysis apparatus 1 according to an embodiment of the present invention includes: an acquisition unit for acquiring electrocardiogram (ECG) data from an ECG measuring device; an ECG encoder for receiving the ECG data and calculating a numerical vector using deep learning; an analysis unit for generating analysis results using the numerical vector for performing disease-related prediction or diagnosis; and a downstream processing unit 16. Figure 1 Although the downstream processing unit 16 is shown separately from the analysis unit 14, the downstream processing unit 16 may also be included as part of the analysis unit 14 or replace the analysis unit 14.

[0149] The acquisition unit 10 is attached to part of the subject's body and can acquire electrocardiogram signals directly or indirectly from an electrocardiogram measuring device used to measure the electrocardiogram signals of the subject (user).

[0150] The acquisition unit 10 also receives information from the electrocardiogram measuring device for measuring the electrocardiogram signal of the subject via a sensor attached to part of the subject's body. Therefore, the acquisition unit 10 can directly acquire the electrocardiogram signal from the electrocardiogram measuring device.

[0151] A sensor is attached to a portion of the object's body and can be used to measure the object's (user's) electrocardiogram (ECG) signal. The ECG signal acquired from the sensor and the ECG signal measuring device can be converted into a digital signal via an analog-to-digital converter (ADC). Furthermore, when the touch panel determines that the user's body contact has exceeded a preset time, the ECG measuring device (not shown) can measure the biosignal. According to another embodiment, in addition to the raw ECG signal, the acquisition unit 10 can output a visualized ECG image to paper or an image based on the acquired ECG signal.

[0152] The acquisition unit 10 can receive information from the electrocardiogram measuring device that measures the electrocardiogram signal of the subject through a sensor attached to a part of the subject's body. Therefore, the acquisition unit 10 can directly acquire the electrocardiogram signal from the electrocardiogram measuring device.

[0153] Furthermore, the acquisition unit 10 can also communicate with external devices via wired / radio. Therefore, the acquisition unit 10 can acquire electrocardiogram (ECG) signal data previously acquired from or stored on an external device. The external device itself can be connected to the ECG measuring device, or ECG signal data can be acquired through other external devices connected to the ECG measuring device. Therefore, the acquisition unit 10's acquisition of ECG signal data from an external device can also be considered as indirect acquisition of ECG signals.

[0154] The ECG encoder 12, as a computing device including a processor, receives electrocardiogram (ECG) data input from the acquisition unit 10, extracts feature maps for each channel by analyzing the ECG signal, and calculates numerical vectors through various computational processes. Subsequently, the numerical vectors can be used by the analysis unit 14 or the downstream processing unit 16 to analyze diseases, predict diseases, and provide diagnostic assistance information.

[0155] In one embodiment of the present invention, for example, the ECG encoder 12 described above can be a variety of computing devices, including personal computers (PCs) or laptops, smartphones, servers, etc.

[0156] In one embodiment of the present invention, the ECG encoder 12 described above can be implemented by a server, and the ECG data input of the ECG encoder can be performed by a device (e.g., a user terminal or a signal input device) connected to the server.

[0157] In this context, the server, as a computer system or computer software implemented by a web server, can provide various types of information by building a website. The web server, which can be connected to a subordinate device capable of communicating with other web servers via computer networks such as private intranets or the Internet, refers to a computer system and computer software (web server program) that provides execution results by receiving work execution requests and performing corresponding tasks. However, depending on the circumstances, in addition to this web server program, the series of applications operating on the web server should be understood as a broad concept including various databases built internally. For example, in the case of including various databases, the ECG encoder 12 utilizes external database information such as in the cloud. In this case, the ECG encoder 12 can achieve data communication by accessing an external database server (e.g., a cloud server) through work.

[0158] In one embodiment of the present invention, the ECG encoder 12 for numerical vector calculation may include a deep learning model, which learns a large amount of unstructured ECG data in a deep neural network composed of multiple layers to automatically learn the features of each ECG data. Thus, the network for numerical vector calculation can be learned through an objective function, that is, the network for numerical vector calculation can be learned by minimizing the prediction accuracy error.

[0159] In one embodiment of the present invention, the deep learning model included in the ECG encoder 12 may utilize CNN (Convolutional Neural Network), Deep Hierarchical Network (DHN), Convolutional Deep Belief Network (CDBN), Deconvolutional Deep Network (DDN), etc. Preferably, a CNN-based deep learning model is used; however, the present invention is not limited thereto.

[0160] In one embodiment of the present invention, the modified CNN structure applied to the ECG encoder 12 of the present invention is particularly suitable for electrocardiogram analysis, for the reasons described below.

[0161] 1) Use of a squeeze excitation network: A squeeze excitation network improves the quality of the encoder and the numerical vectors it extracts by effectively reflecting the anatomical (location) information of each channel in the numerical vectors extracted by the encoder. In the input of a multi-channel electrocardiogram (ECG), the input of each channel reflects the specific anatomical location of the heart. Anatomical location information is crucial in diagnosing pathophysiology of the heart. Therefore, when extracting multiple clinically significant numerical features from an ECG, information that conforms to the specific anatomical features reflected by each value should be non-linearly integrated. Then, during the squeeze excitation process, when creating the representation for the next layer, the contribution of each channel is optimized through the aforementioned recalibration process. Specifically, when applied to the side closer to the input, feature extraction focused on specific channels is performed based on the anatomical structure; when applied to the side closer to the output, abstract clinical information is selected for non-linear integration.

[0162] 2) Use of Non-local Networks: Non-local networks effectively reflect the interaction between ECG features that are spaced apart over time by the numerical vectors extracted by the encoder, thereby improving the quality of the encoder and the numerical vectors it acquires. During the input of single-channel / multi-channel ECGs, when analyzing ECG waveforms with clinical significance at a specific time point, the ECG waveforms before and after that specific time point should be considered. Furthermore, in order to apply this information referencing and integration process to waveform information at a distance, a network capable of learning to integrate distant features appropriately into the current location information of the object being analyzed is needed; the aforementioned non-local network performs this function.

[0163] 3) Use of Skip Connections: The ECG encoder 12 in this embodiment of the invention uses a deep learning structure with multiple nonlinear transformations for the input data. In this case, gradient vanishing can occur, where the loss signal at the output cannot be fully transmitted to the input. Skip connections can effectively reduce this problem. Furthermore, skip connections allow the information at the input to be reflected at the output in a state with minimal deformation. Therefore, they can broadly reflect various features of the extracted numerical vector during the encoder's transformation process, thereby improving the quality of the numerical vector.

[0164] 4) Multi-task learning: Multi-task learning is a method for training encoder networks with the aforementioned network characteristics. During training, a numerical vector obtained through the encoder is used for multiple downstream tasks, thereby making the numerical vector universal. As described above, the output of the ECG encoder 12 of the present invention is a numerical vector shrunk to a fixed size and form through the embedding process, which is used to perform multiple downstream tasks. The numerical vector used here serves as input information for various machine learning algorithms for multiple objectives; therefore, the comprehensive clinical state of the patient should be extracted as efficiently as possible. In the embodiments of the present invention, the output vector of the ECG encoder 12 can simultaneously perform multiple tasks. Therefore, when the error signals at the output ends of each downstream task network are backpropagated, they converge at the end of one encoder, thereby training an encoder. The encoder trained in this way can generate an embedding vector with universality for achieving the aforementioned objectives.

[0165] In one embodiment of the present invention, the electrocardiogram data received by the ECG encoder 12 may include an electrocardiogram image, which shows the electrocardiogram signals of all channels on an electrocardiogram signal or a two-dimensional plane.

[0166] In one embodiment of the present invention, when the electrocardiogram data is an electrocardiogram signal, the electrocardiogram signal input to the ECG encoder 12 includes multiple channels or a single channel. In this case, the electrocardiogram data is a two-dimensional array of C×M (number of inputs (channels) of each lead × number of measured values ​​of each channel).

[0167] In one embodiment of the present invention, when the electrocardiogram data is an electrocardiogram image, the electrocardiogram image is converted into an image including all lead channels or an image of each lead channel cropped according to each lead channel. The electrocardiogram image input to the ECG encoder 12 includes a black and white image of one or more lead channels or a color image composed of R (Red), G (Green), and B (Blue) of three color channels. In this case, it can be input as a three-dimensional array of C×W×H (number of channels × number of horizontal pixels × number of vertical pixels).

[0168] In one embodiment of the present invention, the ECG encoder 12 includes a convolutional layer and a plurality of convolutional blocks thereon, each convolutional block including a plurality of consecutive ECG sub-units. As the ECG encoder 12 passes through the first convolutional layer and the plurality of convolutional blocks, electrocardiogram data can be converted into a numerical vector. Hereinafter, refer to 3 and... Figure 4 This section provides a more detailed explanation of how the ECG encoder converts electrocardiogram (ECG) data into numerical vectors.

[0169] In one embodiment of the present invention, the analysis unit 14 uses the numerical vector calculated by the ECG encoder 12 to provide information for disease-related analysis, prediction or diagnosis.

[0170] The analysis results of the analysis unit 14 include disease prediction. When predicting a disease, the electrocardiogram data can be single-channel or multi-channel data. The diseases may include shock, respiratory failure, pulmonary edema, cardiac arrest, cardiogenic syncope and sudden death, acute coronary syndrome, myocardial infarction, and hyperkalemia.

[0171] The analysis results of the above-mentioned analysis unit 14 include disease diagnosis. When diagnosing a disease, it may include heart rate abnormalities (tachycardia, bradycardia, various arrhythmias), and structural and functional abnormalities of the heart (heart failure, cardiac tamponade, valvular stenosis / failure, pulmonary hypertension, pulmonary embolism, cardiomyopathy).

[0172] In one embodiment of the present invention, the electrocardiogram (ECG) data can be multiple ECG data measured at predetermined time intervals or irregular time intervals. Each of the aforementioned ECG data is processed by the ECG encoder to obtain its respective numerical vector to obtain a diagnosis from the diagnostic unit 14. Alternatively, multiple numerical vectors can be simultaneously input into a machine learning algorithm to diagnose the aforementioned disease or to diagnose whether the disease is improving or worsening.

[0173] The analysis unit 14 described above can arrange the numerical vectors obtained from the multiple electrocardiogram data in a sequential vector order. If multiple numerical vectors are processed as input, they can be concatenated along their length to transform them into a single input and pass through a multilayer perceptron (MLP) network; or, concatenated along the vertical direction of their length to pass through a transformer network; or, without concatenation, passed sequentially through a recurrent neural network according to the order of examination. The analysis unit 14 can also learn whether a patient's specific disease improves or worsens over time based on the second numerical vector extracted from the data.

[0174] In one embodiment of the present invention, the analysis unit 14 may further consider time-related information to learn whether a specific disease is improving or worsening.

[0175] Alternatively, the time interval between the input vectors (e.g., the first numerical vector) may not be specified. In this case, an additional function can be used to encode the time-related information including the time interval to calculate the value representing the time interval, and the combined vector of the encoded value and the first numerical vector can be used as the input vector to add time-related information to the input vector. As described above, the analysis unit 14 can further consider the time interval between the input vectors to determine whether the disease is improving or worsening.

[0176] The aforementioned time-related information can also be implemented using an additional function. This function can be a function based on statistical modeling of actual electrocardiogram data or a function arbitrarily specified by the user. In this case, the function corresponding to the time-related information is incorporated into the input vector due to encoding.

[0177] At this point, the accuracy can be improved by using the function to encode time-related information and concatenating the above numerical vectors.

[0178] On the other hand, in an embodiment of the present invention, the downstream processing unit 16 applies the numerical vector calculated by the ECG encoder to process the downstream task. In one embodiment of the present invention, each of the downstream tasks may execute a multi-layer perceptron (MLP) having two or more fully connected layers.

[0179] In one embodiment of the present invention, the MLP networks for each task are trained together with the ECG encoder network, or, alternatively, the ECG encoder 12 is trained first and then trained separately. When there are multiple downstream task networks, each task network can be trained simultaneously using multi-task learning. In addition to the numerical vector output by the ECG encoder 12, the downstream task networks can receive other additional structured data input information to improve prediction accuracy. In this case, the additional structured data input information is concatenated with the numerical vector, or it can be processed by other additional input networks. The additional structured data input information may include one or more of age, gender, and biological signals different from the electrocardiogram. The biological signals different from the electrocardiogram may include at least one of blood pressure, pulse rate, body temperature, respiratory rate, accompanying symptoms, and oxygen saturation. Furthermore, the structured numerical detection results may also include unstructured detection results (image detection, etc.) after appropriate embedding and unstructured text data based on natural language as additional input.

[0180] In one embodiment of the present invention, for example, the disease analysis device 1 can be combined with an electrocardiogram measuring device (ECG) capable of providing clinical information (e.g., a 12-lead ECG, an ECG monitor, and other medical devices utilizing single-channel or multi-channel ECGs) by directly analyzing ECG signals previously or soon to be obtained from the patient in real time.

[0181] As an example of non-limitation, it can include smartwatches, learning machines equipped with electrocardiogram measuring devices, and other sports equipment, but is not limited to these.

[0182] Furthermore, in one embodiment of the present invention, the disease analysis device 1 described above can also be combined with an electrocardiogram (ECG) analysis device. The ECG analysis device can directly analyze and obtain an ECG through a local device or server and output a visualized ECG image on paper or in a photographic form to provide clinical information.

[0183] As a non-limiting example, it could be a device with an application installed, an EHR (electronic health record) system that uses a camera or scanning device and has an analysis algorithm installed, etc., but is not limited to these.

[0184] On the other hand, the method of converting electrocardiogram (ECG) data into numerical vectors to provide auxiliary information for the analysis, prediction, and diagnosis of diseases (hereinafter referred to as the "disease analysis method") is executed by a computing device including a processor. For example, the aforementioned computing device including a processor is executed by the aforementioned disease analysis device 1 or at least a portion of its structural elements (e.g., acquisition unit 10, ECG encoder 12, analysis unit 14, and / or downstream task processing unit 16) [the downstream processing unit 16 may exist separately from or be included in the analysis unit 14], or it may be executed by other computing devices. Hereinafter, to ensure clarity at the illustrative level, the present invention will be further described in detail through an embodiment in which the aforementioned disease analysis device 1 executes the aforementioned disease analysis method.

[0185] Figure 2 A flowchart illustrating a disease analysis method according to an embodiment of the present invention is provided. (Refer to...) Figure 2 The disease analysis method is executed by a processor. As a method for analyzing diseases from electrocardiogram (ECG) signals using deep learning, it includes: step S10, acquiring ECG data from an ECG measuring device (e.g., executed by the acquisition unit 10); step S121, inputting the ECG data into an ECG encoder (e.g., executed by the ECG encoder 12); step S122, calculating a numerical vector using the ECG encoder and deep learning; and step S14, performing disease-related analysis, prediction, or diagnosis using the numerical vector (e.g., executed by the analysis unit 14). For example, it also includes step S16 where a downstream task processing unit 16 processes a downstream task using the numerical vector. As part of the analysis step S14 executed by the analysis unit 14, each task can execute a multi-layer perceptron (MLP) with two or more fully connected layers.

[0186] Figure 3 A diagram illustrating an ECG encoder subunit according to an embodiment of the present invention.

[0187] Reference Figure 3 In one embodiment of the present invention, the ECG encoder 12 is based on CNN and includes multiple convolutional blocks.

[0188] The aforementioned ECG encoder 12 includes an ECG encoder subunit.

[0189] The aforementioned ECG subunit includes the remaining convolutional blocks excluding the first convolutional layer.

[0190] The aforementioned ECG encoder subunit may include a depthwise-separable convolution layer that convolutions the electrical signal data individually for each channel.

[0191] In the ECG encoder subunit constituting the ECG encoder 12 described above, electrocardiogram data (single-channel or multi-channel ECG signals or ECG images) undergoes two depthwise separable convolutions, and is then used as input data for the next convolutional layer via skip connections. After the depthwise separable convolutions, the depthwise separable convolutional layer takes the form of a layer accompanied by pointwise convolutions.

[0192] Existing convolutions consider all channels and local information to create a feature map, while depthwise separable convolutions adjust the number of output feature maps by combining depthwise separable convolutions with pointwise convolutions.

[0193] In one embodiment of the invention, depthwise separable convolution performs convolution operations with independent filters on each channel of the electrocardiogram data. It may also include a point-wise convolution operator directly connected to the depthwise separable convolution operation. Depthwise separable convolution separates the C×W×H input on a channel-by-channel basis and applies individual convolutional filters to create outputs. If the results are then merged, the convolutional filters can achieve the same output size with very small parameters. This can be advantageous when the results of each filter operation need to be independent of other filters. Point-wise convolution, acting as a 1×1 Conv filter (1× in the case of one-dimensional input), aims to extract information from each channel through shuffling. This is used to reduce or increase the total number of channels using the above method. That is, depthwise separable convolution performs convolution along the channel direction of the feature map to generate output points side-by-side. Point-wise convolution can perform convolution on each spatial point by concatnate the independent channel output values ​​using depthwise separable convolution. To improve the performance of each channel, two or more input channel output dimensions can be applied.

[0194] In one embodiment of the invention, batch normalization can be performed after depthwise separable convolution. This makes the input distribution of each layer constant, thereby improving learning efficiency. In addition to batch normalization, various nonlinear activation functions (e.g., ReLU), dropout, etc., can be used.

[0195] The aforementioned ECG encoder subunit may include a sqeeze-excitation layer, which calculates a scalar value by averaging the values ​​extracted from each channel. The scalar values ​​for each channel are between 0 and 1. As the scalar value is multiplied by each channel, the values ​​of each channel are recalibrated. This allows the deep learning algorithm to learn the importance of each channel.

[0196] The skip connection adds the existing input value to the result obtained through the above-described operation in an element-wise manner. If the number of channels in the output value becomes different from the number of channels in the input value due to the above operation, the existing input value is passed through a linear layer to match the number of channels before being added to the output value.

[0197] Figure 4 A diagram illustrating an embodiment of the ECG encoder of the present invention. (Refer to...) Figure 4 In one embodiment of the invention, the system may include a convolutional layer at the input end and four convolutional blocks connected thereto. When the input data is an electrocardiogram (ECG) signal, the first convolutional layer is a standard 7×1 convolution with 64 output channels. Subsequently, after a batch normalization layer and a max pooling layer, the system sequentially passes through four convolutional blocks. Each convolutional block includes two consecutive encoder subunits, and the final convolutional block may include a non-local network. If all convolutional blocks are passed, a global pooling process is performed at the end. The kernel size (e.g., 7×1, 5×1, 3×1), stride size, padding method, number of output channels and blocks, number of subunits in each block, and batch processing of the non-local network for all convolutional and pooling layers are appropriately determined according to the state of the ECG signal.

[0198] Reference Figure 3 In one embodiment of the present invention, each encoder subunit has a series of depthwise separable convolutional layers (e.g., stride 2), batch normalization layers, depthwise separable convolutional layers (e.g., stride 1), batch normalization layers, and compressed excitation layer structures, which may include a skip connection added to the result vector by bypassing this series of processing steps.

[0199] Squeeze-excitation is a methodology centered on the compression and recalibration of feature maps. It focuses on channel relationships to explicitly model the interdependencies between channels, thereby appropriately readjusting the feature responses of each channel.

[0200] In one embodiment of the present invention, compression operations can be applied to existential transformations (F... tr In the case of X→U, that is, transforming an input X with dimensions [H',W',C'] into an output U with dimensions [H,W,C].

[0201] Extracting information from each channel relative to the output U (F) sq ), based on its information computing (F ex The scalar weighted values ​​of each channel are multiplied by the corresponding channel (F). scale For ease of explanation, if F tr If it's a convolutional layer, then the standard convolution has C' input channels. With C output channels, it learns C'×C two-dimensional filters. The set of learned filters is V={v1,v2…v…} c}, F tr The result is U = {u1, u2, ... u} c}

[0202] Compression operations extract information from each output channel to account for channel dependencies. That is, a scalar value z representing the channel is created using the information of the entire channel.

[0203] In one embodiment of the present invention, in order to make F sq To extract only the essential information from each channel, one of the most common methodologies can be used: global average pooling. Global average pooling has the advantage that the number of parameters in the learning object does not increase.

[0204] The above compression operation can be performed using the following mathematical formula.

[0205]

[0206] Zc represents the representative value of each channel. During the compression operation, a feature map of size H×W×C is compressed to size 1×1×C using global average pooling. After summing all the eigenvalues ​​corresponding to one channel of the feature map, the result is divided by H×W to compress it to 1×1×1. The feature map has C channels; therefore, if all compression results for each channel are combined, the result is calculated as data of size 1×1×C. As described above, each channel can be compressed into one dimension and the ECG eigenvalues ​​of each channel can be summed to calculate the representative value Zc for each channel. The calculated Zc vector is used for activation operations.

[0207] If important information is squeezed, a recalibration process, called excitation, is performed to calculate channel-wise dependencies. Important aspects of this excitation process are: first, to identify non-linear relationships between channels; and second, to select and emphasize multiple channels, rather than using one-hot activation to select only one channel from C channels to increase the weighting value. To this end, in one embodiment of the invention, this can be achieved by adjusting the fully connected layer and the nonlinear function.

[0208] As a parameter of a fully connected layer or nonlinear function, the weighting value s can be represented by the following mathematical formula.

[0209] s = F ex (z, W)=σ(g(z, W))=σ(W2δ(W1z))

[0210] Where s is the weighted value of each channel based on the contribution of each ECG channel, with a value between 0 and 1. σ is the sigmoid function, and δ is the ReLU function. R is the reduction ratio, a hyperparameter used to reduce the number of parameters in the fully connected layer through dimensionality reduction. Further adjustments can be made by multiplying by the weighted values ​​s of each channel.

[0211] The final numerical vector Xc, after readjustment, is calculated based on the weighted values ​​s of each of the aforementioned channels. This final numerical vector Xc can be represented by the following mathematical formula.

[0212]

[0213] The final calculated Xc is a vector, and each element S c Values ​​between 0 and 1 are used as scalar products applied to the respective channels to assign weights to each channel. This can be learned using a deep learning algorithm to determine the importance of each channel.

[0214] In one embodiment of the present invention, the final convolutional block of the encoder may further include a non-local network. The non-local network adds a spatial attention mechanism. It calculates the inner product between the query vector of a specific spatial point in the feature map and the key vector of the overall spatial points. If the calculated inner product is normalized using a softmax operation, a scalar value, equivalent to a weighted value between 0 and 1, can be calculated at each position in the feature map. If the scalar value is multiplied by its corresponding spatial point value vector and all are summed, the value vector of the specific spatial point is converted into a weighted sum of the value vectors of the overall spatial points. The value transformed in this way is added to the initial feature map via skip connections to form the output value. The vectors corresponding to the key, query, and value can be calculated from the input feature map using independent parameter functions. In this process, when analyzing features present at specific time points in an electrocardiogram signal (equivalent to specific locations in one-dimensional input data), the overall context of the data can be more effectively determined because signals from other time points at greater distances can be considered simultaneously. Furthermore, when the input data is a two-dimensional image of a multi-channel waveform signal, there is an additional effect: waveforms from different channels can also be referenced.

[0215] Correspondingly, ordinary CNNs are limited to operating only on the local neighborhood; even with Atrous convolutions or large kernel sizes, the area that a filter can see at once is finite. Similarly, ordinary RNNs are limited to considering only previously restricted time ranges. Typically, this kind of computation, which only knows local information at the temporal or spatial level, involves repeated computations to achieve a global view. However, this repeated computation is not only inefficient and difficult to optimize, but also leads to multi-hop dependency when modeling.

[0216] This invention uses a non-local network, which allows the features of all other temporal positions in the feature map and the features of all other channels (when the input is the output image data) to improve upon the limitations of conventional CNNs by referencing the shape of the weighted sum. In embodiments of this invention, although the non-local network is used for the final convolutional block of the ECG encoder, its batch processing varies based on the input data and the intended use.

[0217] In one embodiment of the present invention, an electrocardiogram (ECG) signal can be converted from an input data block (from the first to the fourth convolutional block) into a two-dimensional matrix of N×D (the spatial dimension of the output depends on the input size and network structure) × D (channel size). This N×D matrix is ​​then expanded to an N×D dimension vector, or pooled along each channel (e.g., global average pooling) to convert it into a D dimension vector, thereby completing the overall encoding process of the ECG signal.

[0218] In an additional embodiment of the invention, when the phases of the individual electrocardiogram (ECG) signals are mismatched, an additional processing method is used. ECGs stored in medical records are typically grouped into sets (I, II, III / AVR, AVL, AVF / V1, V2, V3 / V4, V5, V6), and the measurement time is differentiated by each group. Although the phases within a group match, the phases between groups do not. Therefore, it is difficult to analyze all 12 channels simultaneously using a single network. In this case, in this embodiment of the invention, each lead group has its own separate ECG encoder network. Furthermore, one of the following methods—concatenation, addition, or attention mechanism—can be used to combine the individual output vectors into a single embedding vector.

[0219] In an additional embodiment of the invention, multiple different encoding networks trained with various settings can be combined and used together. The ECG encoder described above can use various forms of depthwise separable convolutional layers within each convolutional layer, depending on the input signal, the processing problem, and the analysis device. Furthermore, the kernel size, stride size, padding method, and output size can be set to various values ​​according to the convolutional layer. In this case, multiple embedding vectors can be extracted from an ECG signal, and their results can be integrated (e.g., through concatenation, addition, or attention mechanisms) for disease prediction and diagnosis.

[0220] In one embodiment of the invention, when the input data is an electrocardiogram (ECG) image, the basic structure remains the same as when the input data is an ECG signal, except as illustrated in the following examples. The input image is normalized by resizing and cropping each channel to a specific size. The kernels of all convolutional layers and depthwise separable convolutional layers are two-dimensional. The kernels of all pooling layers (max pooling and global average pooling) are also two-dimensional. After the final pooling (e.g., global average pooling), the output is an N×D or D-dimensional vector, identical to existing methods. Without input from individual lead channels, all channels are on a single image; therefore, it is impossible to create individual lead (or lead group) models separately.

[0221] In one embodiment of the invention, the numerical vector values ​​calculated by the ECG encoder can be used for downstream tasks. Each task executes a multi-layer perceptron (MLP) with two or more fully connected layers. The MLP for each task can be trained either 1) jointly with the ECG encoder, or 2) separately by receiving the embedding vectors output by the ECG encoder 12, which has already finished training. If trained in the second manner, only the downstream task MLP is trained with the weights of the ECG encoder fixed. After the training ends, the weights of the ECG encoder can be unfixed, and a fine-tuning process can be added by backpropagating to train the entire network.

[0222] In one embodiment of the present invention, the MLP of each task receives additional structured data input information that is different from the numerical vector calculated by the ECG encoder 12 to improve prediction accuracy. In this case, after preprocessing such as standardization, the additional input information is concatenated with the numerical vector, or it can be processed into input through additional input networks.

[0223] When dealing with multivariate regression analysis problems, the output values ​​of the above MLP (predicting multiple values) are directly used as the output value vector.

[0224] When dealing with classification problems (selecting one from multiple items), the probability of including each item is calculated using the softmax function, and the item with the highest probability is selected.

[0225] When predicting whether a specific event will occur, each output value is analyzed as an event probability using a sigmoid function. This probability can be viewed as the conditional probability of the ECG data being analyzed. When outputting, the baseline risk probability represents the marginal probability of the ECG data without considering the input data. Visual charts (e.g., bar charts) can be displayed to show the rate of increase in probability (conditional probability / marginal probability) of the ECG data.

[0226] In one embodiment of the present invention, the exemplary downstream tasks included in the present invention are clinical diagnostic or predictive tasks, including: 1) heart rate abnormalities (rapid heart rate, bradycardia, various arrhythmias - sinus tachycardia, sinus bradycardia, atrial fibrillation / flutter, junctional rhythms, supraventricular tachycardia, conduction blocks, ventricular tachycardia, ventricular fibrillation)); 2) structural and functional abnormalities of the heart (heart failure, cardiac tamponade, valvular stenosis / regurgitation, pulmonary hypertension, pulmonary embolism). 3) Various clinical events (shock, respiratory failure, cardiac arrest, cardiac syncope, sudden cardiac arrest, acute coronary syndrome, myocardial infarction, ST-segment elevation myocardial infarction, hyperkalemia) or 4) Various clinical treatments (vasopressor infusion, mechanical ventilation, coronary angiography and interventions, and transfusion, etc.)

[0227] Therefore, in addition to electrocardiograms, additional structured data information can be received. The aforementioned additional structured data input information is equivalent to age, gender, and structured biological information (blood pressure, pulse rate, respiratory rate, body temperature, numerical test results, etc.) as well as appropriately modified unstructured information that achieves structure (major symptoms, underlying diseases, text, radiographic and ultrasound image information, stethoscope sounds and other sound information, and various biological signals).

[0228] In this embodiment of the invention, in order to improve the quality of the numerical vector (embedding) of the ECG encoder, three auxiliary learning tasks (supervised learning / self-supervised learning / unsupervised learning) can be applied in the training process of the ECG encoder.

[0229] First, supervised learning can be performed in parallel by downstream tasks. This refers to calculating heart rate, PR interval, QRS duration, QT interval, corrected QT interval, P, QRS, and T wave axes, or performing calculations to determine the presence of conditions equivalent to atrial fibrillation, atrial flutter, ectopic atrial rhythm, AV block (1st, 2nd, and 3rd degree), left and right bundle branch block, intraventricular conduction delay, anterior and posterior fascicular block, ECG baseline wandering, ECG signal artifact, atrial premature beat complex, ventricular premature beat complex, pacing rhythm, and junctional rhythm. The task involves assessing the morphological characteristics of various cardiac rhythms, including bigeminy, trigeminy, supra-ventricular rhythm, ventricular rhythm, left ventricular hypertrophy, right ventricular hypertrophy, ST segment elevation, ventricular repolarization, myocardial infarction, and Q waves. This self-supervised learning-based task improves the quality of numerical vectors, ensuring that clinically defined electrocardiographic morphological characteristics are reflected in the numerical vectors.

[0230] For reference, the aforementioned morphological characteristics refer to the morphological patterns repeatedly observed from electrocardiogram waveforms and defined by medical or clinical personnel. Most of these morphological characteristics are not related to the measured lead location and cannot be considered as a final diagnosis on their own.

[0231] Second, self-supervised learning can be performed in parallel by downstream tasks. This includes: 1) transforming the original ECG data in a specific way and then deriving the type (and content) of the transformation; 2) recovering the original file using the transformed input. In method 1), the transformations used relative to the original signal include: i) adding noise conforming to a standard distribution; ii) scaling the original input value by multiplying it by a specific range; iii) reversing the waveform by multiplying the original input value by -1; iv) converting the time batch processing of the original input value into a time flip of the domain; and v) dividing the original data into specified time intervals and then mixing the intervals of each random batch processing. This transformation can be one of these or applied repeatedly. The transformation (or a combination thereof) used is determined as the primary function, and training can be performed as needed to derive the specific content of the transformation (e.g., scaling amount, order of interval mixing, etc.). In method 2), the transformations used relative to the original data include: i) adding noise conforming to a standard distribution; ii) replacing a specific time interval with 0 (only recovering the corresponding interval); and iii) adding a random nonlinear curve (continuous and differentiable). In this case, the task is to derive the entire original data or a portion of the original data (e.g., the original values ​​of the intervals that have been replaced with 0). This self-supervised learning-based task can extract high-quality numerical vectors, making the numerical vectors more effective in reflecting the morphological characteristics of the electrocardiogram.

[0232] Third, unsupervised learning can be parallelized by downstream tasks. The unsupervised learning content applied in this invention is as follows. The network training process of this invention is the same as that of other deep learning networks, and a data augmentation process that deforms the input data in a specific way can be applied. In this process, N deformed ECG input data are created from one ECG. In this case, if there are M original ECGs, then M×N ECG input values ​​are created. When two ECGs are extracted from these M×N ECGs, if the two ECGs are the same as the original data, then they are the same as or very similar to the numerical vectors created therefrom. To satisfy this restriction, the unsupervised learning task of this invention adds a loss term as shown below to the existing loss function, so as to minimize the distance between two augmented data points generated from the same original data.

[0233] β·I(VecA and VecB is made from same ECG signal)·||VecA-VecB||

[0234] Where β is a hyperparameter that can be randomly adjusted by user input, I is the indicator function, and ||VecA-VecB|| is the distance between two vectors created from the two electrocardiograms. As an example, the distance can be measured using Euclidean distance, but is not limited to this. Similar to the β value, the above distance factors can be changed according to the problem situation.

[0235] As mentioned above, adding a loss term can train the encoder so that each numerical vector has a similar shape in the vector space obtained from the numerical vectors, so that each numerical vector can be effectively batch-processed in the vector space defined by the numerical vectors, thereby improving the embedding quality of numerical vectors.

[0236] When auxiliary learning tasks based on the aforementioned supervised learning / self-supervised learning / unsupervised learning are performed in parallel, the downstream task network used for learning and the ECG encoder network are trained jointly. This can be done independently beforehand during the training of the downstream network aimed at clinical diagnosis / prediction, or it can be done simultaneously with the training of the aforementioned clinical diagnosis / prediction network. In the pretraining mode, after the pretraining ends, the weights of the ECG encoder are fixed, and only the clinical diagnosis / prediction network is trained. Subsequently, as needed, the fixed weights of the ECG encoder are released, and a fine-tuning process is applied to train both (the encoder and the downstream task network for clinical diagnosis / prediction) simultaneously. When training the self-supervised learning network and the clinical diagnosis / prediction network simultaneously, the weight updates are implemented across the weights of all networks, including the ECG encoder.

[0237] As described above, the preceding / parallel learning of supervised / self-supervised / unsupervised learning enables the numerical vectors (embedded vectors) output by the ECG encoder to simultaneously incorporate relevant morphological information and their own independent morphological information into the clinically defined pattern displayed on the ECG, thereby improving its versatility (supervised / self-supervised learning). This is further enhanced by effectively batch processing, allowing the vector space of the batched numerical vectors to be efficiently applied to other types of downstream tasks for which the ECG encoder was not pre-designed. In other words, this facilitates few-shot or one-shot learning.

[0238] On the other hand, the following examples illustrate the application of the above-mentioned electrocardiogram encoder or the numerical vector extracted therefrom.

[0239] Application examples of numerical vectors

[0240] As an example of the application of this invention, numerical vectors can be used for clinical diagnosis and treatment, emergency care, and patient diagnosis and triage in disaster scenes. An input vector is created by concatenating additional information besides the numerical vector obtained from the ECG encoder, so that it can be used to perform the desired clinical diagnosis and clinical event / treatment prediction through a new downstream task network.

[0241] The aforementioned additional structured data information may include at least one of the following: biological signals (vital signs) such as age, gender, blood pressure, pulse rate, body temperature, respiratory rate, and oxygen saturation; existing structured information such as various numerical test results; unstructured data (images, sounds, biological signals, etc.) that have been converted into structured numerical information through machine learning methods; and natural language information such as symptoms, disease names, and medical records that have been transformed into numerical vectors through natural language processing.

[0242] As a preferred example, the downstream task network used can be a batch normalization layer, a dropout layer, and a non-linear activation function as described above, such as a multilayer perceptron neural network consisting of two or more fully-connected layers including ReLU. The specific structure can be changed depending on the purpose of use.

[0243] As mentioned above, when training a new downstream task network, the weights of the ECG encoder are first fixed, and then updated by training the new downstream network. This can be applied to fine-tuning by adding training to the weights of both the ECG encoder and the downstream task network.

[0244] As another example of the application of this invention, the numerical vector can be applied to electrocardiograms, critically ill patient monitors, and various medical devices. The numerical vector extracted from an ECG encoder can be used in various medical devices (smartwatches, sports equipment, etc.). In such medical devices, in addition to the numerical vector, additional information can be received as input in the same manner as described above.

[0245] For example, encoders used in medical devices are characterized by receiving single-channel to four-channel input data as electrocardiogram (ECG) signals. When connected to a patient, various medical devices need to continuously process the fluctuating data. Therefore, after dividing the patient's information into segments at predetermined time intervals (e.g., 5 seconds), the ECG encoder and downstream task network analyze each segmented data interval. Diagnostic or predictive services can be provided based on the analysis results at each time point or a weighted average of these analysis results at each time point. The diagnostic or predictive targets of such medical devices, as diagnostic or predictive information obtained through the medical device, may include, for example, the deterioration of the user's condition under normal conditions and / or exercise / pharmacy load, or various emergency situations, including arrhythmias, angina pectoris, pulmonary edema, pulmonary embolism, acute coronary syndrome, electrolyte abnormalities, etc.

[0246] Figure 5 The diagram illustrates the application of numerical vectors obtained from multiple electrocardiogram data acquired through repeated measurements, in another embodiment of the present invention.

[0247] Reference Figure 5 Electrocardiograms (ECGs) are typically performed multiple times for a single patient. When acute coronary syndrome is suspected, they are performed every few minutes or hours; for stable patients, they are performed every few weeks or years. The purpose of this repeated measurement is to allow physicians to diagnose the risk level of a specific disease / condition by clinically assessing the morphological changes of the ECG over time. If the same function were achieved through artificial intelligence, the unstructured morphological features of each repeated ECG would need to be quantified in a consistent manner, a process performed by the ECG encoder in this embodiment of the invention.

[0248] That is, the two numerical vectors obtained by the individual ECG data through their respective ECG encoders (the parameter weights of the two ECG encoders can be the same; convolution kernel parameters are shared) can be used as a single input numerical vector, following the method of analyzing two ECGs that meet specific clinical criteria (e.g., time intervals). Furthermore, the model is learned by creating a network with the same structure as the downstream task network described above and setting the diagnosis to be predicted by the output value. In this case, as an example of application, it can be applied to acute coronary syndrome, drug side effects (heart failure) after long-term administration of anticancer drugs, improvement (improvement of shock) or deterioration (heart failure / pulmonary edema) of patient status before and after intravenous infusion therapy, etc., but is not limited to these.

[0249] Figure 6 The diagram illustrates the application of N consecutively acquired numerical vectors in another embodiment of the present invention.

[0250] Reference Figure 6 This allows N consecutively executed electrocardiograms (ECGs) that meet specific clinical criteria to pass through an ECG encoder 12. This is equivalent to embedding ECG data as unstructured data, thereby obtaining N consecutively acquired numerical vectors. These consecutively embedded vectors are then used as input to train a learning model using a standard RNN (LSTM or GRU) or Transformer network to predict whether a patient's specific disease will improve / worsen over time or whether a specific clinical event will occur. The consecutive numerical vectors used as input values ​​can be enhanced by concatenating additional information, which may include clinical information (age, sex, blood pressure, pulse rate, respiratory rate, body temperature, symptoms, standard examination results) converted into numerical vectors. Furthermore, the RNN or Transformer network used here is only one instance of a neural network architecture that can be used to process numerical vectors constructed sequentially through repeated measurements; any machine learning algorithm with similar functionality can also be applied.

[0251] As an example of application, multiple 12-lead electrocardiograms obtained through repeated measurements can be used. Artificial intelligence algorithms for calculating the risk levels of acute coronary syndrome, pulmonary edema, pulmonary embolism, respiratory failure, cardiac arrest, and hyper / hypokalemia can be installed in 12-lead electrocardiogram machines or electronic medical record programs.

[0252] As another application example, the system can install a specified length (e.g., time interval: 5 seconds) of patient electrocardiogram information measured in real time through one or more channels to receive input sequentially. This allows for the real-time calculation of the risk levels associated with acute coronary syndrome, pulmonary edema, pulmonary embolism, respiratory failure, cardiac arrest, and hyper / hypokalemia. If the risk level is above a specified level, the system can notify the user through the algorithm-equipped critical care monitoring equipment and server device, but it is not limited to this.

[0253] The disease analysis device described above can be implemented by a computing device, including at least one of a processor, a memory, a user input device, and a display device.

[0254] Memory refers to a storage medium that, when executed by a processor, can perform specific tasks. It can store computer-readable software, application programs, program modules, routines, instructions, and / or data. The processor can read and execute the computer-readable software, application programs, program modules, routines, instructions, and / or data stored in memory. A user input device is a means for a user to input instructions or data required to perform a specific task. User input devices may include physical or virtual keyboards or auxiliary keyboards, buttons, mice, joysticks, trackballs, touch-sensitive input devices, or microphones. Display devices may include monitors, printers, speakers, or vibration devices.

[0255] Computing devices can include various devices such as smartphones, tablets, laptops, servers, and clients. A computing device can be a self-media device, and may include multiple computing devices operating in a distributed environment comprised of multiple computing devices connected via communication network protocols.

[0256] Furthermore, the aforementioned disease analysis method can be executed by a computing device, which includes a processor that executes the disease analysis method of converting electrocardiogram data into numerical vectors. The computing device is equipped with a memory that stores computer-readable software, application programs, program modules, routines, instructions, and / or data structures for executing the aforementioned disease analysis method.

[0257] The embodiments of the present invention described above can be implemented by various devices. For example, the embodiments of the present invention can be implemented by hardware, firmware, software, or a combination thereof.

[0258] When implemented in hardware, the disease analysis method of this invention can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, or microprocessors.

[0259] For example, the disease analysis method for converting electrocardiogram data into numerical vectors according to embodiments of the present invention can be implemented using an artificial intelligence semiconductor device that utilizes the neurons and synapses of a deep neural network, which are implemented by semiconductor devices. In this case, the semiconductor device is a currently used semiconductor device, such as SRAM or DRAM, NAND, etc., and the next-generation semiconductor device can be RRA or STT MRAM, PRAM, etc., or a combination thereof.

[0260] When the disease analysis method for converting electrocardiogram data into numerical vectors in the embodiments of the present invention is implemented by an artificial intelligence semiconductor device, the neural network model can transfer the results (weighted values) learned by the software to a synaptic simulation device according to the array batch processing or perform learning through the artificial intelligence semiconductor device.

[0261] When implemented by firmware or software, the disease analysis method for converting electrocardiogram data into numerical vectors according to embodiments of the present invention can be implemented by means of devices, steps, or functions for performing the above-described functions or work. The software code is stored in a storage unit and driven by a processor. The storage unit is located inside or outside the processor and can send and receive data with the processor through various known devices.

[0262] Furthermore, the terms "part," "device," "module," "system," "processor," "controller," "component," "interface," or "unit" as described above can refer to computer-related entities such as hardware, combinations of hardware and software, software, or running software. For example, the aforementioned structural elements can be processor-driven processes, processors, controllers, control processors, objects, execution threads, programs, and / or computers, but are not limited to these. For example, an application program running on a controller or processor, or a controller or processor itself, can be a structural element. More than one structural element may exist within a process and / or execution thread, and structural elements may reside in one device (e.g., a system, computing device, etc.) or be distributed across two or more devices.

[0263] The above description is merely an example illustrating the technical concept of the present invention. Those skilled in the art can make various modifications and variations without departing from the essential characteristics of the technical concept of the present invention. Furthermore, the embodiments of the present invention are for illustrative purposes only and do not limit the technical concept of the present invention; therefore, the scope of the technical concept of the present invention is not limited to these embodiments. It should be understood that the scope of protection of the present invention should be interpreted based on the scope of the claims, and all technical concepts within the equivalent scope fall within the scope of the claims of the present invention.

Claims

1. An apparatus for converting bioelectrical signal data into numerical vectors, characterized in that, include: The acquisition unit is used to acquire bioelectrical signal data; as well as The encoder receives the aforementioned bioelectrical signal data and uses a deep learning algorithm to calculate the first numerical vector. The first numerical vector mentioned above is structured data related to features extracted from bioelectrical signal data. These features include, in context, at least one of anatomical features and temporal features that can be extracted from bioelectrical signal data. In multi-channel bioelectrical signal data, the input of each channel reflects a specific anatomical location, and each channel corresponds to a lead. The encoder extracts the anatomical features by considering the characteristics of other channels along the ventricle when analyzing the characteristics of a specific channel. The encoder includes at least one of a compression excitation network and a non-local network. The apparatus further includes multiple downstream processing units that use a first numerical vector as an input numerical vector to simultaneously process multiple downstream tasks, and In this process, as error signals from the outputs of various downstream task networks converge at the end of the encoder due to backpropagation, the universality of the first numerical vector is improved by training the encoder.

2. The apparatus according to claim 1, characterized in that, The aforementioned first numerical vector is used as the input numerical vector of the downstream processing unit or combined with other structured data information as the input numerical vector of the downstream processing unit.

3. The apparatus according to claim 1, characterized in that, The encoder described above can provide an input numerical vector by combining multiple first numerical vectors output from each encoder from two or more encoders.

4. The apparatus according to claim 1, characterized in that, This allows N consecutive bioelectrical signal data to be processed by an encoder to provide N consecutive first numerical vectors.

5. The apparatus according to claim 1, characterized in that, After dividing bioelectric signal data according to a specified time interval, the above-mentioned device provides relevant analysis, diagnosis or prediction of a specific disease based on the first result value of each time point obtained by passing the information of each divided data interval through the encoder or the encoder and downstream processing unit, or based on the second result value obtained by calculating the weighted average of the first result values ​​for the time points.

6. The apparatus according to claim 1, characterized in that, When training the network for the downstream task, the device modifies the network weights of the downstream task through training after fixing the network weights of the encoder, and modifies the overall weights of the encoder network and the downstream task network through additional training.

7. The apparatus according to claim 1, characterized in that, Each of the aforementioned downstream processing units is executed by a multilayer perceptron having two or more fully connected layers.

8. The apparatus according to claim 7, characterized in that, The aforementioned multilayer perceptron and the aforementioned encoder's encoding network are trained together through multi-task learning, or the encoder is trained first and then trained separately.

9. The apparatus according to claim 7, characterized in that, The aforementioned multilayer perceptron can receive additional structured data input information that is different from the first numerical vector. The aforementioned additional structured data input information includes at least one of the following: age, gender, biosignals, numerical detection results, natural language-based structured data information transformed into structured data through natural language processing, and structured data information converted from biosignal data of a different type than the bioelectrical signal data acquired from the acquisition unit. The aforementioned biosignals include one or more selected from a group consisting of blood pressure, pulse rate, body temperature, respiratory rate, and oxygen saturation. The aforementioned additional structured data input information may be combined with or separately from the aforementioned first numerical vector.

10. The apparatus according to claim 7, characterized in that, It also includes a display unit that, when the multilayer sensor predicts whether a specific disease will occur, simultaneously displays the marginal probability of the specific disease occurring considering the acquired bioelectrical signal data and the marginal probability of the specific disease occurring without considering the acquired bioelectrical signal data when the multilayer sensor outputs, and displays the increase ratio of the marginal probability of the specific disease occurring considering the acquired bioelectrical signal data to the marginal probability of the specific disease occurring without considering the acquired bioelectrical signal data.

11. The apparatus according to claim 1, characterized in that, The deep learning algorithm of the aforementioned encoder is based on Convolutional Neural Networks (CNNs). The device includes an encoder subunit. The encoder subunit mentioned above includes: One or more convolutional layers; One or more fully connected layers, wherein the fully connected layers include nonlinear activation functions; and The central layer aggregates the feature sets extracted from the bioelectrical signal data of each channel to extract representative values ​​for each channel, and then readjusts the feature sets of each channel to reflect the contribution of the feature sets of each channel based on the aforementioned representative values. The aforementioned feature set includes features related to the anatomical structure of each channel. Compared to the feature sets mentioned above, the revised feature sets for each channel are more focused on features related to the anatomical structure of each channel.

12. The apparatus according to claim 11, characterized in that, The aforementioned one or more convolutional layers include depth-separable convolutional layers that individually convolve bioelectrical signal data from one or more channels.

13. The apparatus according to claim 11, characterized in that, The aforementioned centralized layer performs pooling processing on the aforementioned feature set in order to aggregate the aforementioned feature set.

14. The apparatus according to claim 11, characterized in that, The aforementioned centralized layer allows the representative value of each channel to be used to calculate the contribution of each channel through the fully connected layer, and The feature sets of each channel are then readjusted by multiplying the contribution of each channel by the feature set.

15. The apparatus according to claim 14, characterized in that, The aforementioned centralized layer will scale the representative values ​​of each channel to a specific range through the results of the fully connected layer, thereby calculating the contribution of each channel.

16. The apparatus according to claim 11, characterized in that, The encoder described above includes multiple convolutional blocks. The aforementioned sub-units are included in the remaining convolutional blocks excluding the first convolutional layer.

17. The apparatus according to claim 16, characterized in that, The aforementioned convolutional block includes a first encoder subunit and a second encoder subunit. Compared to the second encoder subunit, the first encoder subunit is closer to the input of the convolutional block, rather than its output. In the work of summarizing the feature set to extract representative values ​​and the readjustment based on the contribution of each channel in the accumulator layer, the accumulator layer focuses more on summarizing the feature set to extract representative values, compared to the second encoder subunit. Therefore, the representative values ​​of the first encoder subunit reflect more features related to the anatomical structure than those of the second encoder subunit. Compared to the first encoder subunit, the second encoder subunit is closer to the output end of the convolutional block rather than the input end. In the work of summarizing the feature set to extract representative values ​​and the work of readjusting based on the contribution of each channel in the above-mentioned intensive layer, compared to the first encoder subunit, the above-mentioned intensive layer focuses more on the work of readjusting based on the contribution of each channel.

18. The apparatus according to claim 16, characterized in that, The final convolutional block of the aforementioned encoder also includes a non-local network. The aforementioned non-local network achieves spatial attention by comparing the similarity between spatial points of the aforementioned bioelectrical signal data.

19. The apparatus according to claim 1, characterized in that, The above bioelectrical signal data are single-channel or multi-channel signals. The bioelectrical signal input to the encoder is a two-dimensional array of C×M, where C×M is the number of input leads × the number of measurements in each channel, and the leads are channels.

20. The apparatus according to claim 1, characterized in that, The above bioelectrical signal data are bioelectrical signal images. The above bioelectric signal images show the bioelectric signals of all channels in a two-dimensional plane. The aforementioned bioelectrical signal images were converted into images including all lead channels or images of individual lead channels cropped according to each lead channel. The bioelectric signal image input to the encoder is converted into a black and white image with one or more lead channels or a three-dimensional array of C×W×H with three channels: red (R), green (G), and blue (B), where C×W×H is the number of channels × the number of horizontal pixels × the number of vertical pixels.

21. The apparatus according to claim 1, characterized in that, It also includes an analysis unit that uses the first numerical vector to perform disease or health-related analysis, prediction, or provide diagnostic assistance information, wherein the analysis unit provides analysis results including at least one of disease analysis, disease prediction, and disease diagnostic assistance information.

22. The apparatus according to claim 21, characterized in that, The above analysis results include disease prediction. When the aforementioned analysis unit predicts diseases, the aforementioned bioelectrical signal data is single-channel bioelectrical signal data. When the aforementioned bioelectrical signal data is electrocardiogram (ECG) data, the aforementioned diseases include shock, respiratory failure, cardiac arrest, acute coronary syndrome, myocardial infarction, and hyperkalemia.

23. The apparatus according to claim 21, characterized in that, The above analysis results include diagnostic aids for determining whether the disease is improving or worsening using the first numerical vector. When the aforementioned analysis unit provides auxiliary information for disease diagnosis, the aforementioned bioelectrical signal data consists of multiple bioelectrical signal data measured at prescribed intervals. The aforementioned multiple bioelectrical signal data are passed through the pooling layer of the encoder to provide diagnostic auxiliary information.

24. The apparatus according to claim 21, characterized in that, The above analysis results include information that assists in disease diagnosis. The aforementioned bioelectrical signal data consists of multiple bioelectrical signal data measured at specified or irregular time intervals. The aforementioned analysis unit is trained in the following manner to determine whether a patient's specific disease improves or worsens over time: Each of the first numerical vectors of the above multiple bioelectric signal data is arranged into a sequential vector. The sequential vectors are merged along the length of the vectors to pass through a multilayer perceptron (MLP) network, or the sequential vectors are merged along the vertical direction of the vector length to pass through a transformer network, or the sequential vectors are not merged to pass through a recurrent neural network (RNN) in sequence, thereby extracting the second numerical vector.

25. The apparatus according to claim 24, characterized in that, Before inputting the sequence vector for training, the time-related information is encoded, and the encoded time-related information values ​​are then merged with the first numerical vector mentioned above. The aforementioned analysis unit receives the merged vectors as sequential vectors for training.

26. The apparatus according to claim 1, characterized in that, The encoder network described above is trained based on clinically defined morphological characteristics of bioelectrical signal data and through supervised learning. The clinically defined morphological characteristics include at least one of pulse rate, PR interval, QRS interval, QT interval, corrected QT interval, P wave axis, QRS wave axis, T wave axis, atrial fibrillation, atrial flutter, atrioventricular block, fascicular block, ST segment elevation, and myocardial infarction.

27. The apparatus according to claim 1, characterized in that, The encoder network described above will convert bioelectrical signal data in a specific way to use as training data and will be trained through self-supervised learning.

28. The apparatus according to claim 1, characterized in that, The aforementioned encoder network uses enhanced bioelectrical signal data as training data and is trained through unsupervised learning. The enhanced bioelectrical signal data, which share the original bioelectrical signal data, are input into the encoder and adjusted so that the calculated first numerical vectors are the same or have a high degree of similarity.

29. The apparatus according to claim 28, characterized in that, The above adjustments, which make the calculated first numerical vectors identical or highly similar, mean minimizing the distance between the calculated first numerical vectors.

30. The apparatus according to claim 1, characterized in that, The aforementioned device is combined with a smartwatch, a medical device or sports equipment equipped with a bioelectric signal measuring device, or a device with a smartphone application or electronic health record system.

31. The apparatus according to any one of claims 1 to 30, characterized in that, The above bioelectrical signals are electrocardiograms (ECG).

32. A method for converting bioelectrical signal data into numerical vectors, executed by a processor, characterized in that, The method includes: Acquire bioelectrical signal data; The aforementioned bioelectrical signal data is input into the encoder; and The first numerical vector is computed using deep learning algorithms. The first numerical vector is structured data related to features extracted from bioelectrical signal data. These features include, in a context, at least one of anatomical features and temporal features that can be extracted from bioelectrical signal data. In multi-channel bioelectrical signal data, the input of each channel reflects a specific anatomical location, and each channel corresponds to a lead. The anatomical features mentioned above are extracted by considering the characteristics of other channels along the choroid when analyzing the properties of a specific channel. The encoder includes at least one of a compression excitation network and a non-local network. The method further includes multiple downstream processing units simultaneously processing multiple downstream tasks by using the first numerical vector as an input numerical vector, and... In this process, as error signals from the outputs of various downstream task networks converge at the end of the encoder due to backpropagation, the universality of the first numerical vector is improved by training the encoder.

Images