Method for generating training image used for training image-based artificial intelligence model for analyzing image obtained from multi-channel one-dimensional signal and device performing the same

Abstract

To provide a training image generation device capable of generating large-scale training images with diverse image formats and signal patterns.SOLUTION: A training image generation method includes: generating a training signal on the basis of source signal information; selecting at least one output format from a plurality of preset output formats to determine an output format of the training image; determining an output section for each channel of the training signal on the basis of a length of a time section of a waveform of the determined output format; selecting a per-axis scale to determine a grid scale of the training image; drawing a grid pattern on a two-dimensional plane in accordance with the determined grid scale; setting a reference position of a waveform content of the training signal on the basis of at least one of the determined output section for each channel or the determined grid scale; and drawing the waveform content of the training image and a signal marker on the two-dimensional plane with the grid pattern drawn thereon. There is also provided a device for performing the method.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The embodiments relate to a training image generation technique, for example, a training image generation technique capable of generating a large number of training images with a variety of image formats and signal patterns, which can be used to train an image-based artificial intelligence model that analyzes images obtained from multi-channel one-dimensional signals, and an apparatus for implementing the technique. [Background technology]

[0002] Biosignals such as electrocardiograms and electroencephalograms are generally measured in the form of one-dimensional signals. At this time, the measured signals are embodied as numerical values ​​in the time domain measured in the corresponding channel and are generally expressed in a CxT type numerical array structure. The measured numerical values ​​are stored in the aforementioned numerical array structure and used.

[0003] Since the signal data values ​​are difficult for a human reader to understand as they are, they are displayed as a two-dimensional image output on paper or on the screen of a device, depicting waveforms for each channel in a format that is easy for humans to understand.

[0004] However, there are many different formats for 2D image output, and even for 2D image output used for the same purpose, the formats can vary greatly depending on the product.

[0005] Suppose we are developing an AI model to analyze signal information displayed as a 2D image output. If this AI model is trained using training images with one output format, it will not be able to analyze target images with other output formats, and will have poor analytical performance.

[0006] Therefore, in order for an AI model to learn with high analytical performance, it is necessary to prepare training images with as many different output formats as possible. However, it is practically extremely difficult to prepare a large number of training images for sufficient learning using only existing images of existing products in use. In particular, there is a limitation in that it is completely unable to prepare for future changes, such as the use of new output format images when new products are released or existing products are updated. Summary of the Invention [Problem to be solved by the invention]

[0007] In one aspect, an exemplary embodiment of the present application aims to provide a training image generation method and apparatus that can generate large-scale training images with a variety of image formats and signal patterns in order to solve problems that arise when training 2D images in only one format without considering the diversity of formats, for example, a training image generation technique used to train an image-based artificial intelligence model that analyzes images obtained from multi-channel 1D signals, and a method and apparatus for implementing the same. [Means for solving the problem]

[0008] In an embodiment of the present application, there is provided a training image generation method executed by a computing device including a processor and a memory, for example, used to train an image-based artificial intelligence model that analyzes images obtained from a multi-channel one-dimensional signal, the training image generation method including the steps of: generating a training signal based on source signal information including a multi-channel one-dimensional signal; selecting at least one output type from a plurality of preset output types to determine the output type of the training image; determining a channel-specific output interval of the training signal based on the length of the time interval of the waveform of the determined output type; selecting an axis-specific scale to determine a grid scale of the training image; displaying a grid pattern on a two-dimensional plane according to the determined grid scale; setting a reference position of the waveform content of the training signal based on at least one of the determined channel-specific output interval and the determined grid scale; and displaying the waveform content and signal indicator of the training signal on the two-dimensional plane on which the grid pattern is displayed.

[0009] In addition, an embodiment of the present application provides a training image generation device including: an acquisition unit that acquires a source signal; and an image generation unit that includes a processor and a memory; wherein the image generation unit receives the source signal information received by the acquisition unit and executes the aforementioned training image generation method. [Effects of the Invention]

[0010] In one aspect, according to an embodiment of the present application, it is possible to provide a training data set that can be used to develop artificial intelligence that generates various types of two-dimensional signal images from a source signal, such as a multi-channel one-dimensional signal, and analyzes the two-dimensional signal images with superior performance.

[0011] By utilizing the above-mentioned various types of 2D signal images, it is possible to solve problems that arise when training is performed using only one type without taking into account the diversity of the types, such as the out-of-distribution problem, where even a slight change in the type can result in incorrect analysis.

[0012] The invention according to the embodiments of the present application can be expanded and applied not only to signals in the medical and biomedical fields, but also to signal analysis in other industrial fields in which two-dimensional images are analyzed based on one-dimensional signals.

[0013] The effects of the present application are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims. [Brief explanation of the drawings]

[0014] In order to more clearly explain exemplary embodiments of the present application, the drawings necessary for the description of the embodiments are briefly introduced below. It should be understood that the following drawings are only intended to explain the embodiments of the present specification and are not intended to be limiting. In addition, for the sake of clarity, some elements in the following drawings may be shown with various modifications, such as exaggeration or omission. [Figure 1] 1 is a flow diagram of a training image generation method used to train an image-based artificial intelligence model to analyze images obtained from multi-channel one-dimensional signals, according to one aspect of the present application. [Figure 2] FIG. 2 is a schematic diagram of source signal information according to an embodiment of the present application; [Figure 3] FIG. 2 is a schematic diagram of a training signal according to an embodiment of the present application; [Figure 4] FIG. 2 is a schematic diagram of an output format according to one embodiment of the present application. [Figure 5] FIG. 1 is a schematic diagram of a grating scale according to an embodiment of the present application. [Figure 6] FIG. 10 is a diagram illustrating a grid pattern according to one embodiment of the present application. [Figure 7] FIG. 1 is a schematic diagram illustrating the results of displaying the waveform content and signal indicators of a training signal on a two-dimensional plane with a grid pattern, according to one embodiment of the present application. [Figure 8] FIG. 1 is a schematic diagram of additional information according to one embodiment of the present application; [Figure 9] FIG. 10 is a block diagram of an apparatus for performing the training image generation method according to another aspect of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0015] Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. When assigning reference numerals to components in each drawing, the same numerals may be assigned to identical components whenever possible, even if they are shown in different drawings. Furthermore, when describing the present embodiments, if it is determined that a detailed description of related known configurations or functions may obscure the gist of the present technical idea, the detailed description may be omitted.

[0016] When terms such as "comprise," "have," and "consist of" are used in this specification, other parts may be added unless "only" is used. When an element is expressed in the singular, it may also include a plural unless otherwise expressly stated.

[0017] Furthermore, when describing components of this application, terms such as first, second, A, B, (a), (b), etc. are sometimes used. Unless otherwise specified, these terms are used merely to distinguish the component from other components, and do not limit the essence, order, procedure, number, etc. of the component.

[0018] As used herein, "learning" is a term that refers to performing machine learning through procedural computing.

[0019] As used herein, network refers to a neural network of machine learning algorithms or models.

[0020] As used herein, terms such as "unit," "module," "device," or "system" are intended to refer not only to hardware but also to a combination of software driven by the hardware. For example, hardware may be a data processing device including a central processing unit (CPU), a graphic processing unit (GPU), or other processor. Furthermore, software may refer to a running process, object, executable, thread of execution, program, etc.

[0021] In this specification, the term "multi-channel" is defined to mean one or more channels, and does not exclude the case where there is only one channel.

[0022] In certain embodiments, a training image generation method used to train an image-based artificial intelligence model to analyze two-dimensional images obtained from multi-channel one-dimensional signals may be performed by a computing device including at least one processor and a memory.

[0023] The computing device is configured to receive and process signal data from an external device (eg, a signal measurement device).

[0024] FIG. 1 is a flow diagram of a method for generating training images used to train an image-based artificial intelligence model that analyzes images obtained from multi-channel one-dimensional signals, according to one aspect of the present application.

[0025] Referring to FIG. 1, the training image generating method (hereinafter referred to as training image generating method) includes a step of generating / modulating a training signal based on source signal information (S100).

[0026] FIG. 2 is a schematic diagram of source signal information according to an embodiment of the present application, and FIG. 3 is a schematic diagram of a training signal according to an embodiment of the present application.

[0027] Referring to FIG. 2, the source signal information may include one-dimensional signals for each channel measured through one or more channels attached to the subject's body. For example, the input source signals may be a channel x time (C x T) array. The source signals may have a numeric array. As shown in FIG. 2, the source signal information may include 12 one-dimensional electrocardiogram signals measured through 12 lead channels.

[0028] The source signal information may include an analog source signal, a measurement value (eg, a digital value) of the source signal.

[0029] In addition, the source signal information may further include additional source information other than the signal, such as subject information of the source signal (identification information, age, gender, readout information, etc.), source scale indicator, source channel indicator, and source grid scale.

[0030] Such a source signal is utilized as a source for generating training signals. Multiple training signals may be generated from a single source signal.

[0031] In one embodiment, the step (S100) may include the steps of: selecting at least one transformation function from a plurality of predetermined transformation functions; and transforming the source signal into a training signal using the selected transformation function.

[0032] A transformation function is a function that transforms an input signal into another signal. Such signal transformation may be used as a pre-processing step for the training image generation process. The transformation function may be, for example, a signal pre-processing function for the purpose of removing various types of noise or randomly modifying the input signal itself.

[0033] Each of the plurality of transformation functions comprises one or more transformation elements that perform a transformation function. In some embodiments, the transformation elements may represent signal attributes of the input signal that are to be transformed. The transformation functions (f n 1 ) are predefined and stored. The subscript n indicates that there are n functions, and the superscript is a number to distinguish it from other functions, which will be described later.

[0034] In a particular embodiment, the transformation function (f n 1 ) deformation element vector (P n 1 ) may include m factors, each of which is a transformation element P nm 1 (where n is a transformation function identifier and m is an element identifier). nm 1 ) may correspond to unique input information processing properties. n 1 ), each deformation element (P nm 1 ) is the setting value of the signal processing attribute and the signal is transformed. For example, the transformation function (f n 1 ) is the denoising function, the deformation factor (P nm 1 ) corresponds to a signal processing attribute that defines (sets) certain characteristics of the noise removal process.

[0035] The signal processing attributes may include, for example, methods for processing, modifying, or removing (replacing signal magnitude with a fixed value such as 0) signal magnitude, waveform, frequency range, frequency distribution, signal rise time, time range, and / or other signal attributes, and this may be performed separately for each channel, or may be applied commonly to a channel group or to all channels.

[0036] In a specific embodiment, the transformation elements of the transformation function may be embodied as hyperparameters. When one transformation function includes multiple transformation elements, the multiple transformation elements may be expressed as a hyperparameter vector, where the vector values ​​are the transformation element values. Then, each transformation function may be associated with its own hyperparameter vector.

[0037] A training signal may be generated using such a transformation function (S100). n 1 ) is a signal obtained by receiving a source signal (e.g., an electrocardiogram signal, X) and transforming it into a signal X'=f n 1 (X, P nm 1 ) may be generated (S100). Also, when the transformation element value is changed, the signal attribute is realized with the changed value in a corresponding manner, and finally, a signal with the changed signal attribute is generated (S100).

[0038] In one embodiment, the step of selecting at least one transformation function may select one of the plurality of transformation functions according to a first-first probability distribution preset for the set of the plurality of transformation functions, or may select two or more transformation functions from the plurality of transformation functions.

[0039] The first-first probability distribution defines the probability of selecting a particular transformation function(s) to generate training images from among a plurality of transformation functions.

[0040] In one embodiment, the first probability distribution may be defined by selecting one of n transformation functions. For example, the first probability distribution may have a multinomial distribution. In this case, the source signal is subjected to a single transformation process.

[0041] In another embodiment, the first probability distribution may be defined by selecting two or more transformation functions from n. For example, the first probability distribution may have a binomial distribution. In this case, different transformations may be selected in multiple steps and used to generate the training signal. Then, the source signal is subjected to multiple transformations using the selected transformation functions.

[0042] In an alternative embodiment, the first probability distribution may be an arbitrary specified initial probability distribution that is optimized through training, as will be described in more detail below.

[0043] The selected transformation function may be used as is, or the transformation element values ​​of the selected transformation function may be changed to generate a training signal (S100). The selected transformation function may be adjusted by inputting a new value for at least one transformation element as an input argument.

[0044] In one embodiment, the step of transforming the source signal into a training signal using the selected transformation function may include the steps of: changing a value of at least one transformation factor among transformation factors constituting the selected transformation function to a new value; and generating a signal reflecting a signal attribute changed according to the changed transformation factor value as the training signal. The transformation factor value may be changed based on a probability distribution.

[0045] In one embodiment, a first-second probability distribution may be set for each transformation element. If there are m transformation elements, m first-second probability distributions are defined and assigned. For each transformation element of the transformation function selected according to the first-first probability distribution, the existing transformation element value may be changed to a new value selected according to the first-second probability distribution preset for that element. The changed new value is reflected in the source signal to generate a new training signal.

[0046] The first-second probability distribution defines a probability distribution in which each transformation element of the transformation function has an individual value assigned from the entire range of possible values. In a specific embodiment, the first-second probability distribution may include a Gaussian (normal) distribution, a gamma distribution, an exponential distribution, a uniform distribution, or a chi-squared distribution as a continuous probability distribution, a binomial distribution, a negative binomial distribution, an initial sub-distribution, a Poisson distribution, a multinomial distribution, and / or a multivariate distribution for multiple transformation elements. The same or different first-second probability distributions may be set for each transformation element. In addition to the parametric approach for extracting transformation elements with a specific type of probability distribution defined as described above, transformation elements may be randomly extracted without assuming a specific distribution.

[0047] In one embodiment, when the transformation element is embodied as numerical data, the first and second probability distributions may be continuous probability distributions such as Gaussian (normal) distribution, gamma distribution, exponential distribution, uniform distribution, and chi-square distribution, and the new value of the transformation element is a value selected from the continuous probability distribution.

[0048] In one embodiment, when the transformation element is embodied as binary variable data, the first and second probability distributions may be discrete probability distributions such as binomial distribution, negative binomial distribution, initial lower distribution, and Poisson distribution. The binary variable data may be expressed as a first binary value indicating "yes" or a second binary value indicating "no." The new value of the transformation element is a value selected according to the discrete probability distribution.

[0049] In one embodiment, if the transformation element is embodied as a categorical variable, the first and second probability distributions may be multinomial probability distributions, and the new values ​​of the transformation element are values ​​selected according to the multinomial probability distribution.

[0050] In some embodiments, the set of transformation elements forming the transformation function may include at least some transformation elements that are correlated with each other. If the transformation element changed to a new value is correlated with other transformation elements, the first and second probability distributions set for the other correlated transformation elements may be multivariate probability distributions. Then, the values ​​of the other correlated transformation elements are changed to new values ​​selected according to the predefined multivariate probability distributions.

[0051] In addition to the parametric approach method in which deformation elements are extracted with a specific type of probability distribution defined as described above, deformation elements may be extracted randomly without assuming a specific distribution.

[0052] In an alternative embodiment, the first-second probability distribution may be an arbitrary specified initial probability distribution that is optimized through training, as will be described in more detail below.

[0053] As shown in FIG. 3, a portion of the generated training signal is displayed on a two-dimensional planar frame of training images.

[0054] Referring back to FIG. 1, the training image generating method includes determining the output format of the training image by selecting at least one output format from a plurality of preset output formats (S200).

[0055] FIG. 4 is a schematic diagram of an output format according to one embodiment of the present application.

[0056] 4, the output object format defines a structure for arranging output object components of a training image on a two-dimensional plane frame. The plurality of output object formats may be associated with output object components of different aspects. In a specific embodiment, the plurality of output object formats may include an output object format defining a waveform type, an output object format defining an arrangement position of the waveform, an output object format defining a position of a waveform display area, and / or an output object format defining a position of an additional information display area.

[0057] The waveform type may include a channel number.

[0058] The waveform arrangement position indicates the order and position of the waveforms for each channel, and may include the arrangement order.

[0059] The position of the waveform display area may include the coordinate range of the area (eg, grid area) in which the waveform is displayed.

[0060] The position of the additional information display area includes the coordinate range of the area where additional information other than the waveform is displayed. The additional information may include subject information of the waveform (e.g., age, sex), waveform analysis, waveform scale information, etc.

[0061] The waveform display area and the additional information display area may be separate from each other or may at least partially overlap each other.

[0062] Each of the multiple output formats consists of one or more format elements that form the layout structure of the output format. Therefore, just like the transformation function based on the transformation elements, the output format may also be expressed as a function consisting of format elements. n (n is a natural number greater than or equal to 1) output format functions (f n 2 ) are predefined and stored. Note that the subscript n is a letter indicating that there are n functions, and the superscript is a number to distinguish it from other functions described above and below.

[0063] In a particular embodiment, the output form function (f n 2 ) form vector (P n 2 ) may include m factors, each of which is a formal element P nm 2 (where n is the output form function identifier and m is the element identifier). nm 2 ) corresponds to the placement target defined in the output format, i.e., the output component.

[0064] In one embodiment, the format elements may include a channel number, a channel order, a position of the waveform display area, a position of the additional display area, a vertical spacing between each waveform, a horizontal spacing between each waveform, and / or a length of a time interval of the waveform, etc. For example, a format element of the output format that defines the position of the waveform display area may include the position of the waveform display area on a two-dimensional plane, etc.

[0065] When a form element has a specific value, an output element corresponding to the form element is embodied on the output form with the specific value and is displayed on the training image.

[0066] Similar to the transformation function, the formal elements of the output formal function may be embodied as hyperparameters. When one output formal function includes multiple transformation elements, the multiple formal elements may be expressed as a hyperparameter vector, where the vector values ​​are the formal element values. Then, each output formal function may be associated with its own hyperparameter vector.

[0067] In one embodiment, the step of selecting at least one output form function (S200) may select any one output form function according to a predetermined 2-1 probability distribution for the set of the predetermined plurality of output forms, or may select two or more output form functions from the plurality of output form functions.

[0068] The second probability distribution defines the probability of selecting a particular output form function(s) to generate training images from among a plurality of output form functions. Selecting an output form function according to the second probability distribution is similar to selecting a transform according to the first probability distribution.

[0069] In one embodiment, the second probability distribution may be defined by selecting one output function out of n. For example, the second probability distribution may have a multinomial distribution.

[0070] In another embodiment, the second probability distribution may be defined by selecting two or more output object form functions from n. For example, the second probability distribution may have a binomial distribution. In this case, different output object form functions may be selected and used to generate training images.

[0071] In an alternative embodiment, the second probability distribution may be optimized through training from any specified initial probability distribution, as will be described in more detail below.

[0072] The selected output object format may be used as is, or the values ​​of the format elements of the selected output object format may be changed to determine the output object format for the training images (S200). The selected output object format function may receive a new value for at least one format element as an input argument to adjust the function of the output object format function.

[0073] In one embodiment, the step of determining the output type of the training image by selecting at least one output type from a plurality of preset output types (S200) may include the steps of: changing the value of at least one type element of the selected output type to a new value; and determining the output type having the changed type element value as the output type of the training image. The value of the type element may be changed based on a probability.

[0074] In one embodiment, a 2-2 probability distribution may be set for each form element. If there are m form elements, m 2-2 probability distributions are defined and assigned. For each form element of the output form function selected according to the 2-1 probability distribution, the existing form element value may be changed to a new value selected according to the 2-2 probability distribution preset for that element. The output form reflecting the changed new value is used to generate training images.

[0075] The second probability distribution defines a probability distribution in which each formal element of the output formal function has an individual value assigned from the entire range of possible values. In a specific embodiment, the second probability distribution may include a Gaussian (normal) distribution, a gamma distribution, an exponential distribution, a uniform distribution, or a chi-squared distribution as a continuous probability distribution, a binomial distribution, a negative binomial distribution, an initial sub-distribution, a Poisson distribution, a multinomial distribution, and / or a multivariate distribution with multiple variations. The same or different second probability distributions may be set for each formal element. In addition to the parametric approach for extracting formal elements by defining a specific type of probability distribution, formal elements may be randomly extracted without assuming a specific distribution.

[0076] In one embodiment, if the format element is embodied as numeric data, the second probability distribution may be a continuous probability distribution such as a Gaussian (normal) distribution, a gamma distribution, an exponential distribution, a uniform distribution, or a chi-squared distribution, and the new value of the format element is a value selected from the continuous probability distribution.

[0077] In one embodiment, when a formal element is embodied as binary variable data, the second probability distribution may be a discrete probability distribution such as a binomial distribution, a negative binomial distribution, an initial lower distribution, or a Poisson distribution. The binary variable data may be expressed as a second binary value indicating "yes" or a second binary value indicating "no." The new value of the formal element is a value selected according to the discrete probability distribution.

[0078] In one embodiment, if the formal element is embodied as a categorical variable, the second probability distribution may be a multinomial probability distribution, and the new value of the formal element is a value selected according to the multinomial probability distribution.

[0079] In some embodiments, the set of format elements forming the output format function may include at least some format elements that are correlated with each other. If the format element changed to a new value is correlated with other format elements, the second-second probability distribution set for the correlated format elements may be a multivariate probability distribution. Then, the values ​​of the correlated format elements are changed to new values ​​selected according to the predefined multivariate probability distribution.

[0080] In addition to the parametric approach method for extracting form elements with a specific type of probability distribution defined as described above, form elements may be randomly extracted without assuming a specific distribution.

[0081] In an alternative embodiment, the 2-2 probability distribution may be optimized through training from any specified initial probability distribution, as will be described in more detail below.

[0082] In this way, the output format having form element values ​​modified according to the second-second probability distribution is provided as an output format for generating the training images (S200).

[0083] Referring again to FIG. 1, the training image generating method includes a step of determining an output interval for each channel of the training signal based on the length of the time interval of the waveform of the determined output type (S300), and a step of selecting a scale for each axis to determine a grid scale of the training image (S400).

[0084] The step of determining the channel-by-channel output interval of the training signal (S300) may include the steps of selecting at least one of a start point and an end point of the channel-by-channel output interval to be output to the training image within the entire length of the received source signal; and calculating the channel-by-channel interval based on the length of a waveform time interval and the selected point within the format element of the selected output format.

[0085] Generally, only a portion of the total signals measured is output to the image. The total length of the output section is determined as the time period of the waveform among the format attributes of the output type determined in step S200 (S300).

[0086] The start point or end point is selected from a range that includes the entire time interval of the determined waveform among the entire interval of the generated training signal.

[0087] The selection range of the start point may be a range from the rising point of the training signal to a point extended in the negative time direction by the time interval of the waveform determined in step S200 from the falling point of the training signal. In one example, if the entire interval of the training signal is [0, Ω] and the time interval of the waveform is w, the distribution of the start point may be [0, Ω-w]. In this case, the end point is a point obtained by extending the time interval of the waveform determined in the positive time direction from the selected start point.

[0088] The selection range of the end point may be the opposite of the selection range of the start point, and the selection range of the end point may range from the falling edge of the training signal to a point extended in the positive time direction by the time interval of the determined waveform from the rising edge of the training signal. In this example, the distribution of the end points may be [w, Ω]. In this case, the start point is a point extended in the negative time direction by the time interval of the determined waveform from the selected end point.

[0089] In one embodiment, the start point or the end point may be a point selected from a selection range for each point according to a third probability distribution that is preset, the third probability distribution defining a probability distribution in which individual values ​​are assigned from the selection range for each point.

[0090] The third probability distribution may be a Gaussian (normal) distribution, a uniform distribution, or a chi-squared distribution as a continuous probability distribution. When a start point is selected by a uniform probability distribution or the like, an end point may be automatically selected and an output interval may be determined from the training signal.

[0091] In some embodiments, the multiple channels may include synchronized channels, in which at least some of the channels have the same output duration, while in other embodiments, the multiple channels may be unsynchronized channels, in which the output durations of the multiple channels are different from each other.

[0092] By performing the operation of step S300, the waveform content displayed on the training image over the entire section of the training signal is determined.

[0093] The step of determining the grid scale of the training image (S400) may be determined by selecting a horizontal axis unit scale and / or a vertical axis unit scale of a coordinate system displaying the training signal.

[0094] FIG. 5 is a schematic diagram of a grating scale according to one embodiment of the present application.

[0095] Referring to FIG. 5, in a grid scale, the horizontal axis (or x-axis) represents time and the vertical axis (or y-axis) represents signal measurements.

[0096] In some embodiments, the step of selecting a horizontal axis unit scale and / or a vertical axis unit scale may include the step of selecting any one horizontal axis unit scale from among a plurality of horizontal axis unit scales in accordance with a predetermined 4-1 probability distribution over the entirety of the plurality of predetermined horizontal axis unit scales; and / or the step of selecting any one vertical axis unit scale from among a plurality of vertical axis unit scales in accordance with a predetermined 4-2 probability distribution over the entirety of the plurality of predetermined vertical axis unit scales.

[0097] Similar to the probability distribution 1-1, the probability distributions 4-1 and 4-2 may have a multinomial distribution, and the training signal is then displayed on the training image in selected horizontal and / or vertical axis units.

[0098] Referring again to FIG. 1, the training image generating method includes a step (S500) of representing a grid pattern on a two-dimensional plane according to the grid scale determined in the step (S400).

[0099] FIG. 6 shows a grid pattern representation according to one embodiment of the present application.

[0100] Referring to FIG. 6, the grid pattern according to the grid scale is a pattern in which grids are arranged in units of grid scales for each axis determined in step S400 (S500).

[0101] In one embodiment, the step of representing the grid pattern on a two-dimensional plane (S500) may include the step of selecting one grid pattern format from a plurality of preset grid pattern formats.

[0102] The grid pattern format defines a grid pattern using a unique display line hierarchy and / or a unique display line design of the pattern. The hierarchy indicates, for example, a large section, a medium section, a small section, etc.

[0103] Similarly to the output object form function, each of the multiple grid pattern forms consists of one or more pattern elements that form the grid pattern form. Therefore, similar to the output object form function, the grid pattern form may also be expressed as a function consisting of pattern elements. n (n is a natural number greater than or equal to 1) grid pattern form functions (f n 3 ) are predefined and stored. Note that the subscript n indicates that there are n functions, and the superscript is a number to distinguish it from other functions described above and below.

[0104] In a particular embodiment, the grid pattern form function (f n 3 ) pattern vector (P n 3 ) may contain m factors, each of which is a pattern element P nm 3 (where n is a lattice pattern format function identifier, and m is an element identifier). nm 2 ) corresponds to the pattern components defined by the grid pattern format function.

[0105] In one embodiment, the pattern elements may include the line pattern (various display formats for representing virtual lines such as solid lines, dotted lines, double lines, etc.), thickness, color, etc. The color may be represented by RGB values, CMYK channel values, etc.

[0106] If a pattern element has a specific value, the pattern element corresponding to the pattern element is embodied in the grid pattern format with the specific value and is represented on the training image.

[0107] Similar to the output function, the pattern elements of the grid pattern function may be embodied as hyperparameters. When one grid pattern function includes multiple pattern elements, the multiple pattern elements may be expressed as a hyperparameter vector, where the vector value is the pattern element value. Then, each grid pattern function may be associated with its own hyperparameter vector.

[0108] In an embodiment, the step of selecting one of the plurality of grid pattern types may select one of the grid pattern types according to a preset 5-1st probability distribution.

[0109] The probability distribution (5-1) defines the probability of selecting a particular grid pattern type from the set of grid pattern type functions to generate training images. Selecting an output product type function according to the probability distribution (5-1) is similar to selecting a transform according to the probability distribution (1-1).

[0110] In one embodiment, the 5-1 probability distribution may be defined by selecting one grid pattern function out of n. For example, the 5-1 probability distribution may have a multinomial distribution.

[0111] In an alternative embodiment, the 5-1 probability distribution may be optimized through training from any specified initial probability distribution, as will be described in more detail below.

[0112] The selected grid pattern type may be used as is, or the values ​​of the pattern elements of the selected grid pattern type may be changed to determine the grid pattern type for the training image (S500). The selected grid pattern type may adjust the function of the grid pattern type function by inputting a new value for at least one pattern element as an input argument.

[0113] In one embodiment, the step of representing the grid pattern on a two-dimensional plane (S500) may include the steps of adjusting pattern element values ​​of the selected grid pattern type; and determining the grid pattern type having the adjusted pattern element values ​​as the grid pattern type of the training image. The values ​​of the pattern elements may be changed based on probability.

[0114] In one embodiment, a 5-2 probability distribution may be set for each individual pattern element. If there are m pattern elements, m 5-2 probability distributions are defined and assigned. For each pattern element of the lattice pattern form function selected according to the 5-1 probability distribution, the existing pattern element value may be changed to a new value selected according to the 5-2 probability distribution preset for that pattern element. The lattice pattern form reflecting the changed new value is used to generate training images.

[0115] The 5-2 probability distribution defines a probability distribution in which an individual value is assigned from the entire range of values ​​that each pattern element of the grid pattern type function can have. In a specific embodiment, the 5-2 probability distribution may include a Gaussian probability distribution, a uniform probability distribution, a continuous uniform probability distribution, a normal distribution, a chi-square distribution, a binomial probability distribution, a multinomial probability distribution, and / or a multivariate probability distribution. The same or different 5-2 probability distributions may be set for each pattern element.

[0116] In one embodiment, when the pattern element is embodied as numerical data, the probability distribution of 5-2 may be a continuous probability distribution such as a Gaussian (normal) distribution, a gamma distribution, an exponential distribution, a uniform distribution, or a chi-squared distribution, and the new value of the pattern element is a value selected from the continuous probability distribution.

[0117] In one embodiment, when the pattern element is embodied as binary variable data, the probability distribution of 5-2 may be a discrete probability distribution such as a binomial distribution, a negative binomial distribution, an initial lower distribution, or a Poisson distribution. The binary variable data may be expressed as a second binary value indicating "yes" or a second binary value indicating "no." The new value of the pattern element is a value selected according to the discrete probability distribution.

[0118] In some embodiments, the set of pattern elements forming a grid-pattern-type function may include at least some pattern elements that are correlated with each other. If the pattern element changed to a new value is correlated with other pattern elements, the fifth-second probability distribution set for the correlated pattern elements may be a multivariate probability distribution. Then, the values ​​of the correlated pattern elements are changed to new values ​​selected according to a predefined multivariate probability distribution.

[0119] The grid pattern is then displayed on a two-dimensional plane in the adjusted grid pattern format selected.

[0120] In an alternative embodiment, the 5-2 probability distribution may be optimized through training from any specified initial probability distribution, as will be described in more detail below.

[0121] Referring again to FIG. 1, the training image generating method includes a step S600 of setting a reference position of the waveform content of the training signal determined in step S300 based on at least one of the output type determined in step S200, the channel output interval determined in step S300, and the grid scale determined in step S400; and a step S700 of plotting the waveform content and signal indicators of the training signal on the two-dimensional plane on which the grid pattern is plotted in step S500.

[0122] In step S200, the positions of the training signals for each channel and the positions of the output object arrangement structure are calculated on a two-dimensional plane forming a frame of the training image. In step S300, the content of the waveform to be displayed in the training image is determined.

[0123] The coordinates of the measurement values ​​for the waveform content in the training signal are calculated (S600) according to the grid scale determined in step S400. The coordinates of the measurement values ​​are positions on a two-dimensional plane and define where the waveform content of which channel is located. The coordinates of the measurement values ​​are calculated as coordinate values ​​based on the grid pattern.

[0124] The calculated training signal measurements may be utilized as reference coordinates for the waveform content of the training signal. FIG. 7 is a schematic diagram of the result of displaying the waveform content and signal indicators of a training signal on a two-dimensional plane with a grid pattern, according to one embodiment of the present application.

[0125] Referring to FIG. 7, the step of displaying the waveform content and signal identifier of the training signal on a two-dimensional plane on which a grid pattern is displayed (S700) may display the waveform content and signal identifier of each channel of the training signal on the two-dimensional plane on which the pattern is displayed based on the reference position.

[0126] In one embodiment, the step of representing the waveform content of each channel of the training signal on the two-dimensional plane on which the grid pattern is represented (S700) based on the reference position may include the steps of: defining a representation function for graphically representing the waveform content of the training signal for each channel based on the reference position; and representing the waveform content and signal identifier of the training signal using the defined representation function.

[0127] The notation function (f n 4 ) is the reference coordinate of the training signal and one or more representation vectors (P nm 4) is based on the notation vector (P n 4 ) may contain m factors, each of which is a representation element P nm 4 (where n is the notation function identifier and m is the element identifier). Incidentally, the subscript n is a letter indicating that there are n functions, and the superscript is a number to distinguish it from the other functions mentioned above.

[0128] The reference coordinates of the training signal may include coordinates of measurements of the waveform content of the training signal calculated to fit a grid pattern, such as coordinates on the horizontal axis and coordinates on the vertical axis of the signal.

[0129] The notation element (P nm 4 ) defines the waveform design and signal identifiers. The notation elements consist of a first group related to the waveform design and a second group related to the signal identifier design.

[0130] The first group of notation elements may include a waveform pattern, a thickness, a color, etc. The color may be represented by an RGB value, a CMYK channel value, etc.

[0131] The signal indicators include a channel indicator and a scale indicator. The second group of notation elements may include the position, pattern, font, color, thickness of the character (or line), etc. of the indicator. The position of the indicator is indicated as a relative position from the notated training signal.

[0132] Such definition may be made in response to user input or may be preset.

[0133] The waveform content of the training signal may be represented on a two-dimensional plane using the representation function, and a channel indicator and a scale indicator may be represented within a certain distance from the waveform of the training signal (S700).

[0134] If the representation element values ​​are adjusted, the waveform content of the training signal, the channel indicator, and the scale indicator may be represented on the two-dimensional plane, reflecting the adjusted representation element values ​​(S700).

[0135] In one embodiment, the step of representing the waveform content and signal identifiers of the training signal using the defined representation function may include adjusting values ​​of representation elements of the representation function; and representing the waveform content of each channel of the training signal on the two-dimensional plane on which the grid pattern is represented, reflecting the adjusted representation element values. The values ​​of the representation elements may be changed based on probability.

[0136] In one embodiment, a sixth probability distribution may be set for each notation element. If there are m notation elements, m fifth probability distributions are defined and assigned. For each notation element of the notation function selected according to the sixth probability distribution, the existing notation element value may be changed to a new value selected according to the sixth probability distribution preset for that element. The training image is displayed on a two-dimensional plane in a notation format reflecting the changed new value.

[0137] The sixth probability distribution defines a probability distribution in which each notation element of the notation function is assigned an individual value from the entire range of possible values. In a specific embodiment, the sixth probability distribution may include a Gaussian (normal) distribution, a uniform distribution, or a chi-squared distribution as a continuous probability distribution, a binomial distribution, a negative binomial distribution, an initial subdistribution, or a Poisson distribution as a discrete probability distribution, and a multinomial distribution and / or a multivariate probability distribution with multiple variations. The same or different sixth probability distributions may be set for each notation element. In addition to the parametric approach method for extracting notation elements with a specific type of probability distribution defined as described above, notation elements may also be randomly extracted without assuming a specific distribution.

[0138] In one embodiment, when the notation element is embodied as numeric data, the sixth probability distribution may be a continuous probability distribution such as a Gaussian (normal) distribution, a uniform distribution, a continuous uniform probability distribution, a normal distribution, or a chi-squared distribution, and the new value of the notation element is a value selected from the continuous probability distribution.

[0139] In one embodiment, when the notation element is embodied as binary variable data, the sixth probability distribution may be a binomial distribution, a negative binomial distribution, an initial lower distribution, or a Poisson distribution, which are discrete probability distributions. The binary variable data may be expressed as a second binary value indicating "yes" or a second binary value indicating "no." A new value of the notation element is a value selected according to the discrete probability distribution.

[0140] In one embodiment, if the notation element is embodied as a categorical variable, the sixth probability distribution may be a multinomial probability distribution, and the new value of the notation element is a value selected according to the multinomial probability distribution.

[0141] In some embodiments, the set of notation elements forming the notation function may include at least some notation elements that are correlated with each other. If the notation element changed to a new value is correlated with other notation elements, the sixth probability distribution set for the other correlated notation elements may be a multivariate probability distribution. Then, the values ​​of the other correlated notation elements are changed to new values ​​selected according to the predefined multivariate probability distribution. Then, the waveform content and signal identifiers of the training signal are displayed on a two-dimensional plane in the display format having the changed values ​​(S700).

[0142] In an alternative embodiment, the sixth probability distribution may be optimized through training from any specified initial probability distribution, as described in more detail below.

[0143] The training image generating method also includes the step of further representing additional information and / or scale information of the training signal (S800).

[0144] FIG. 8 is a schematic diagram of additional information according to one embodiment of the present application.

[0145] 8, the additional information of the training signal is the original attributes of the training signal, similar to the source additional information, and may be embodied as text indicating age, sex, measurement location, measurement time, or waveform analysis.

[0146] In one embodiment, the step of displaying the additional information of the training signal may include selecting and displaying additional information text arbitrarily selected from preset additional information texts, and the display position of the additional information text is based on the output format determined in step S300.

[0147] In some other embodiments, the side information of the training signal may be source side information, in which case the side information of the training signal and the side information of the source signal are identical.

[0148] The scale information is information describing the grid scale determined in step S400, and may be embodied as a symbol, a picture, or text representing the grid scale. The display position of the scale information also depends on the output format determined in step S200.

[0149] In one embodiment, the step of representing the scale information may include the steps of selecting one scale notation method from a plurality of preset scale notation methods; and representing the scale information in the selected scale notation method.

[0150] In one embodiment, the step of selecting one of the plurality of preset scale notation schemes may include selecting one of the plurality of preset scale notation schemes according to a seventh probability distribution.

[0151] The seventh probability distribution may be defined in a 1 in n manner, for example, the seventh probability distribution may have a multinomial distribution.

[0152] The training image generating method may further include the step of further modifying the generated training image (S900).

[0153] The step (S900) may include selecting at least one image augmentation function from a plurality of preset image augmentation functions; and further transforming the training image using the selected image augmentation function.

[0154] The image dilation function may consist of multiple dilation factors, which may include a type of deformation, a frequency, an intensity, and / or an epoch number.

[0155] The step of selecting the expansion function is similar to the step of selecting the transformation function. In one embodiment, the step of selecting the expansion function may select one image expansion function from among the plurality of transformation functions or select two or more image expansion functions from among the plurality of image expansion functions according to an eighth probability distribution preset for the set of the plurality of image expansion functions.

[0156] The eighth probability distribution defines the probability of selecting a particular image expansion function(s) to generate training images from the universe of image expansion functions.

[0157] In one embodiment, the eighth probability distribution may be defined by selecting an image expansion function 1 out of n. For example, the eighth probability distribution may have a multinomial distribution.

[0158] In another embodiment, the eighth probability distribution may be defined by selecting two or more image expansion functions out of n. For example, the eighth probability distribution may include a binomial distribution.

[0159] The training image generation method may further include converting data of the generated training image into a tensor of W×H×C' type (S1000), where W is the width of the image, H is the height of the image, and C' is the number of color channels (C'=1 in the case of monochrome). When a multi-channel one-dimensional signal of C×T real number array type is input as a source signal, the training image generation method may generate training images and output the results converted into a three-dimensional real number array of W*h*C type.

[0160] In the alternative embodiment, the probability distribution may be optimized through training from any specified initial probability distribution.

[0161] As previously mentioned, the hyperparameter vector may include binomial, multinomial, and numerical data.

[0162] The binomial data is selected according to a binomial distribution. As described above, the selection according to the binomial distribution may be performed more than once. The label data for the binomial data is implemented as a multilabel choice in which "yes" is selected for one or more items.

[0163] The multinomial data is selected according to a multinomial distribution. As described above, the selection according to the multinomial distribution may be performed independently. Then, the label data for the multinomial data is implemented as a single label choice.

[0164] Numerical data is selected according to a probability distribution with a bounded or unlimited range. Such probability distributions may include continuous probability distributions such as Gaussian (normal) distribution, gamma distribution, exponential distribution, uniform distribution, chi-square distribution, etc. As mentioned above, selection according to a uniform distribution is selecting real numbers with a bounded or unlimited range.

[0165] The elements embodied in the data format may correspond to hyperparameters of a machine learning model and may be targets for optimization.

[0166] Such hyperparameters may be optimized in the process of training an image-based artificial intelligence model that analyzes multi-channel one-dimensional signal images using the generated training images.

[0167] The hyperparameters are optimized with the following goals: a) improving accuracy on the text dataset on which the AI ​​was trained; b) improving robustness against domain shifts and adversarial attacks; and c) improving the embedding quality of the latent vectors (i.e., minimizing the distance between identical concepts in the latent space).

[0168] The hyperparameters may be optimized through optimization algorithms such as, but not limited to, grid search, random search, Gaussian process, and tree-structured Parzen Estimator (TPE).

[0169] According to another aspect of the present application, the training image generation method may be performed by a component configured to receive and process data.

[0170] FIG. 9 is a block diagram of an apparatus for executing the training image generation method according to another aspect of the present application.

[0171] Referring to FIG. 9, the apparatus includes an acquisition unit 10; and an image generation unit 100.

[0172] The acquisition unit 10 acquires a source signal from a device capable of measuring a source signal such as a multi-channel one-dimensional signal. For example, the acquisition unit 10 can acquire an electrocardiogram signal directly or indirectly from an electrocardiogram measurement device attached to a part of a body of a subject to measure the multi-channel one-dimensional signal of the subject (user), such as an electrocardiogram signal.

[0173] The acquisition unit 10 may be connected to receive information from an electrocardiogram device that measures an electrocardiogram signal of the subject via a sensor attached to a part of the subject's body, and the acquisition unit 10 may then directly acquire the electrocardiogram signal from the electrocardiogram device.

[0174] The sensor may be attached to a part of the subject's body to measure the subject's (user's) electrocardiogram signal. The electrocardiogram signal acquired from the sensor and the electrocardiogram signal measuring device may be converted into a digital signal via an analog-digital converter (ADC). The electrocardiogram measuring device (not shown) may measure a biosignal when it determines that the user's body is in contact with the touch panel for a predetermined period of time or longer. According to another embodiment, the acquiring unit 10 may acquire not only an electrocardiogram signal (raw signal) but also an electrocardiogram image that is output to paper or an image and visualized based on the previously acquired electrocardiogram signal.

[0175] The acquisition unit 10 may be connected to receive information from an electrocardiogram device that measures an electrocardiogram signal of the subject via a sensor attached to a part of the subject's body, and the acquisition unit 10 may then directly acquire the electrocardiogram signal from the electrocardiogram device.

[0176] Alternatively, the acquisition unit 10 may be connected to an external device for wired or wireless electrical communication. In this case, the acquisition unit 10 may acquire electrocardiogram signal data from electrocardiogram signal data previously acquired or stored in the external device. The external device may be connected to an electrocardiogram measurement device itself or may acquire electrocardiogram signal data via another external device connected to the electrocardiogram measurement device. Therefore, acquisition of electrocardiogram signal data by the acquisition unit 10 from an external device may be treated as indirect acquisition of an electrocardiogram signal.

[0177] The source signal information measured through one or more channels is acquired by the acquisition unit 10. The source signal information may include an analog source signal, digital source signal information, or a source image representing the source signal.

[0178] The image generation unit 100 is a computing device including a processor and a memory, and may perform steps (S100 to S800) of the training image generation method of FIG. 1 upon receiving source signal information received by the acquisition unit 10.

[0179] In one embodiment, the image generating unit 100 may be implemented as a server, and the acquiring unit 10 may be a device (e.g., a user terminal or signal input equipment) connected to the server for inputting data.

[0180] In this case, the server may be a number of computer systems or computer software implemented as a network server, and may provide various information in the form of a website. Here, the term "network server" refers to a computer system or computer software (network server program) that is connected to a subordinate device capable of communicating with other network servers via a computer network, such as a private intranet or the Internet, receives a request to execute a task, executes the task in response, and provides the execution results. However, in addition to such a network server program, the term should be interpreted as a broader concept that includes a series of application programs running on the network server and, in some cases, various databases built internally. For example, if various databases are included, the server may be configured to use external database information, such as a cloud. In this case, the server may connect to an external database server (e.g., a cloud server) and communicate data through operation.

[0181] The training image generation method and the operation of the apparatus for performing the same according to the embodiments described above may be embodied at least in part in a computer program and recorded on a computer-readable recording medium, for example, in a program product comprising a computer-readable medium containing program code, which may be executed by a processor to perform any or all of the steps, operations, or processes described therein.

[0182] The computer may be a desktop computer, laptop computer, notebook, smartphone, or any similar computing device, or may be integrated with any of these. A computer is a device that has one or more alternative special-purpose processors, memory, storage space, and networking components (either wireless or wired). The computer may run an operating system such as, for example, a Microsoft Windows-compatible operating system, Apple OSX or iOS, a Linux distribution, or Google's Android OS.

[0183] The computer-readable recording medium includes all kinds of recording devices in which computer-readable data is stored. Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. The computer-readable recording medium may also be distributed among computer systems connected via a network, so that computer-readable code is stored and executed in a distributed manner. Functional programs, codes, and code segments for implementing the present embodiment should be easily understood by those skilled in the art to which the present embodiment pertains.

[0184] The present invention has been described above with reference to the embodiments shown in the drawings. However, these are merely illustrative, and those skilled in the art will understand that various modifications and variations of the embodiments are possible based on these. However, such modifications should be considered to be within the technical scope of protection of the present invention. Therefore, the true technical scope of protection of the present invention should be determined by the technical ideas of the appended claims. [Industrial Applicability]

[0185] The present invention relates to a training image generation technique used to train an image-based artificial intelligence model and an apparatus for implementing the same, and can be applied to signal analysis in a variety of industrial fields that analyze two-dimensional images based on signals in the medical and bio fields and other one-dimensional signals.

Claims

1. An apparatus for generating various types of two-dimensional images from electrocardiogram signals, an acquisition unit for acquiring electrocardiogram signals measured via 12 lead channels; an image generation unit including a processor and a memory; Including, The electrocardiogram image generating device is characterized in that the image generating unit is configured to select at least one of a plurality of preset electrocardiogram output formats in accordance with a probability distribution to determine the output format of the two-dimensional image, arrange and represent the 12-lead electrocardiogram signal on a two-dimensional plane in accordance with the selected output format, and generate a plurality of two-dimensional images in different formats from the same electrocardiogram signal.

2. The electrocardiogram image generating device described in claim 1, characterized in that the image generating unit determines a grid scale by probabilistically selecting an axial scale, displays a grid pattern based on the determined grid scale on a two-dimensional plane, and displays the waveform of the 12-lead electrocardiogram signal on the two-dimensional plane on which the grid pattern is displayed.

3. The output format is composed of format elements that define the waveform arrangement order of 12 leads, the interval between each lead waveform, the position of the waveform display area, and the position of the additional information display area; The electrocardiogram image generating device according to claim 1 , wherein the image generating unit generates various output formats by changing the value of each format element according to a probability distribution.

4. The electrocardiogram image generating device described in claim 1, characterized in that the image generating unit determines the display line design of the grid pattern, the line thickness and hue of the waveform, and the notation method of the channel indicators and scale indicators according to their respective probability distributions, thereby generating two-dimensional images of visually diverse forms from the same electrocardiogram signal.

5. The image generation unit converts the generated two-dimensional image into a tensor of W × H × C form, 2. The electrocardiogram image generating device of claim 1, wherein the two-dimensional image is used in an image-based artificial intelligence model for analyzing electrocardiogram images.

6. A dataset for training an artificial intelligence model, comprising a plurality of two-dimensional electrocardiogram images of different formats generated by an electrocardiogram image generating device described in any one of claims 1 to 5.

Images