Method for providing information on classification of doppler sonogram, and device for providing information on classification of doppler sonogram by using same

Abstract

The present invention provides a method for providing information on classification of a Doppler sonogram, implemented by a processor, and a device using the method, the method comprising the steps of: receiving a Doppler echocardiogram of an object; extracting sectional views by using the Doppler echocardiogram as an input; and classifying the sectional views for the Doppler echocardiogram on the basis of the received Doppler echocardiogram, by using a sectional view classifying model that has been trained to classify sectional views.

Description

Method for providing information on Doppler ultrasound image classification and device for providing information on Doppler ultrasound image classification using the same

[0001] The present invention relates to a method for providing information on Doppler ultrasound image classification and a device for providing information on Doppler ultrasound image classification using the same.

[0002] Echocardiography is performed by projecting ultrasound waves on the three-dimensional structure of the heart in multiple planes to obtain images of the heart and measure hemodynamic variables.

[0003] At this time, the medical staff positions the ultrasound probe in a location where it is easy to obtain Doppler ultrasound images to obtain multifaceted images through the anatomical structures around the heart, such as between the ribs, and records the images by finding the appropriate slice through rotation and tilting.

[0004] Meanwhile, among cardiac ultrasound modes, Doppler mode allows for the measurement of blood flow velocity within blood vessels. In particular, blood flow measurement methods based on Doppler ultrasound have the unique advantage of being able to noninvasively measure blood flow velocity in real time, and are widely used in modern medical diagnostics.

[0005] At this time, Doppler ultrasound images can provide blood flow information for various cross-sectional views. Typically, Doppler images acquired during cardiac ultrasound only provide velocity-time information about blood vessels.

[0006] That is, during the image analysis stage, medical staff must classify which cross-section the acquired Doppler ultrasound image corresponds to.

[0007] Here, the deviation in the classification of cross-sectional Doppler ultrasound images obtained depending on the skill level of the medical staff may be large, and therefore, there is a continuous demand for the development of a new information provision system capable of recognizing and classifying cross-sectional images.

[0008] The background technology of the invention has been prepared to facilitate a better understanding of the present invention. It should not be construed as an admission that the matters described in the background technology of the invention constitute prior art.

[0009] Meanwhile, in order to solve the aforementioned problem, the inventors of the present invention attempted to develop an information providing system based on an artificial neural network that is trained to recognize cross-sectional views of Doppler cardiac ultrasound images and to distinguish between each cross-sectional view.

[0010] In particular, the inventors of the present invention were able to recognize and classify cross-sectional views of Doppler ultrasound images by applying an artificial neural network.

[0011] More specifically, the inventors of the present invention focused on an artificial neural network trained to extract cross-sectional views using Doppler echocardiography images as input and classify each cross-sectional view corresponding to the extracted cross-sectional views.

[0012] Meanwhile, the inventors of the present invention focused more on a classification model that derives a distance value using a Doppler cross-section distance graph based on the correlation of cross-sections for extracted Doppler ultrasound images for Doppler cardiac ultrasound images, and outputs a probability vector corresponding to each cross-section using the distance value.

[0013] More specifically, the inventors of the present invention, based on a classification model, represent a cross-section for a Doppler ultrasound image as a Doppler cross-section distance graph based on the correlation between each cross-section, input the distance value derived by the Doppler cross-section distance graph into the ξ function, and train the network using the probability vector matching the input distance value, thereby outputting (at the time of inference) a probability vector more suitable for the Decision Tree shown in Fig. 5 that matches the distance value, thereby constructing a system so that each cross-section can be classified with high accuracy.

[0014] As a result, the inventors of the present invention were able to recognize that it is possible to classify cross-sectional views of Doppler ultrasound images with high accuracy based on the classification model and decision tree [Fig. 5] for the acquired Doppler cardiac ultrasound images.

[0015] The inventors of the present invention were able to realize that by providing a new information provision system, unnecessary diagnosis time can be reduced and cross-sectional diagrams can be classified with high accuracy using an artificial neural network-based system.

[0016] That is, the inventors of the present invention expected that by providing a new information provision system, classification of Doppler ultrasound image cross-sections can be performed with high accuracy regardless of the skill level of medical staff, and thus, highly reliable analysis results for Doppler cardiac ultrasound images can be provided.

[0017] Accordingly, the problem to be solved by the present invention is to provide a method for providing information on a Doppler ultrasound image and a device using the same, which is configured to classify a cross-sectional view extracted from a received Doppler cardiac ultrasound image with high accuracy using an artificial neural network-based classification model.

[0018] The tasks of the present invention are not limited to the tasks mentioned above, and other tasks not mentioned will be clearly understood by those skilled in the art from the description below.

[0019] In order to solve the above-described problem, a method for providing information on a Doppler ultrasound image according to one embodiment of the present invention is provided.

[0020] The above information providing method is an information providing method for Doppler cardiac ultrasound image classification implemented by a processor, comprising the steps of receiving a Doppler cardiac ultrasound image of an object, extracting a cross-sectional view using the Doppler cardiac ultrasound image as input, and classifying the cross-sectional view for the Doppler cardiac ultrasound image using a classification model and a decision tree.

[0021] According to a feature of the present invention, the step of classifying a cross-sectional view of a Doppler cardiac ultrasound image using a classification model further includes the step of calculating a distance value from a Doppler cross-sectional view distance graph, the step of inputting the distance value calculated using the classification model into a ξ function to output a matching probability vector, and the step of learning a network based on the output probability vector and classifying the cross-sectional view of a Doppler cardiac ultrasound image using the network probability output and a Decision Tree.

[0022] According to another feature of the present invention, the step of classifying a cross-sectional view for a Doppler cardiac ultrasound image using a classification model may further include the step of obtaining a probability corresponding to each of a plurality of cross-sectional views based on an output probability vector, and the step of determining a cross-sectional view having the highest value among the corresponding probabilities as a cross-sectional view for the Doppler cardiac ultrasound image.

[0023] According to another feature of the present invention, the cross-section is a plurality of cross-sections, and the labeling generation for the cross-section classification model calculates a distance value from a Doppler cross-section distance graph, inputs the calculated distance value into a ξ function, and uses the matching probability vector as ground-truth during training, so that the probability corresponding to each cross-section reflecting the relative distance between the Doppler ultrasound image cross-sections can be finally output.

[0024] According to another feature of the present invention, the plurality of cross-sections may be one of MV inflow PW, AV (LVOT) PW, PV (RVOT) PW, AV (AS) - CW, PV (PS) - CW, MV (MR) - CW, TV (TR) - CW, AV (AR) - CW, PV (PR) - CW, MV (MS) - CW, pulmonary vein, Septal Annulus TDI or Lateral Annulus TDI.

[0025] According to another feature of the present invention, the Doppler view distance graph is a binary tree, and each cross-section may be arranged in the binary tree according to the degree of correlation between each cross-section.

[0026] According to another feature of the present invention, the classification model further includes a step of calculating a distance value from a Doppler cross-section distance graph when training to classify cross-sections for Doppler cardiac ultrasound images, wherein the step of calculating the distance value may be generating a training GT using a binary tree created by considering the correlation between a plurality of cross-sections for the cross-sections for the Doppler cardiac ultrasound images. The final cross-section classification is obtained using a Doppler Decision Tree using a probability value considering the distance between the Doppler cross-sections.

[0027] According to another feature of the present invention, the step of generating a label of a cross-sectional view for a Doppler cardiac ultrasound image to train a classification model may further include a step of inputting the calculated distance value into a ξ function and outputting a matching probability vector. In this case, the ξ function is

[0028]

[0029] ξ(d) = (1 / (d+1))^lambda

[0030]

[0031] , d is the distance between the Doppler cross sections obtained through binary (d>=0), and lambda(λ) is a hyperparameter value that determines how close to 0 the probability is mapped as the distance increases, and can be any real number greater than 1.

[0032]

[0033] According to another feature of the present invention, the step of classifying a cross-sectional view for a Doppler cardiac ultrasound image using a classification model may further include a step of classifying a cross-sectional view for a Doppler cardiac ultrasound image based on a probability vector learned through the above process, and at this time, the step of classifying a cross-sectional view for a Doppler cardiac ultrasound image based on the calculated probability vector may further include a step of calculating a probability corresponding to each cross-sectional view calculated based on the calculated probability vector and determining the cross-sectional view for the Doppler cardiac ultrasound image through a Decision Tree.

[0034]

[0035] According to another feature of the present invention, the step of classifying the cross-section may include a step of determining at least one Doppler mode cross-section among MV inflow PW, AV (LVOT) PW, PV (RVOT) PW, AV (AS) - CW, PV (PS) - CW, MV (MR) - CW, TV (TR) - CW, AV (AR) - CW, PV (PR) - CW, MV (MS) - CW, pulmonary vein or Septal Annulus TDI, and Lateral Annulus TDI, by using a probability corresponding to each cross-section calculated using a cross-section classification model.

[0036] According to another feature of the present invention, the step of classifying the cross-section may include a step of determining the cross-section having the highest probability value among MV inflow PW, AV (LVOT) PW, PV (RVOT) PW, AV (AS) - CW, PV (PS) - CW, MV (MR) - CW, TV (TR) - CW, AV (AR) - CW, PV (PR) - CW, MV (MS) - CW, pulmonary vein or Septal Annulus TDI, and Lateral Annulus TDI as the cross-section for the Doppler cardiac ultrasound image, using the probability corresponding to each cross-section calculated using the cross-section classification model and the Decision Tree.

[0037]

[0038] In order to solve the above-described problem, a device for classification information in a Doppler ultrasound image according to another embodiment of the present invention is provided.

[0039] The above information providing device includes an ultrasound probe that provides a Doppler echocardiographic image of an object and a processor functionally connected to a communication unit. At this time, the processor is configured to classify a cross-sectional view of a Doppler echocardiographic image based on the received Doppler echocardiographic image using a cross-sectional view classification model learned to classify a cross-sectional view by taking each Doppler echocardiographic image as input.

[0040] According to a feature of the present invention, the classification model may include an output unit that takes a distance value as input and outputs a probability vector.

[0041] According to another feature of the present invention, the cross-section comprises multiple cross-sections, and the cross-section classification label can calculate distance values ​​corresponding to each cross-section using a Doppler cross-section distance graph. Furthermore, using the calculated distance values ​​and the gyro function, a probability vector corresponding to each cross-section reflecting the distances between the multiple cross-sections for the Doppler cardiac ultrasound image can be output.

[0042] According to another feature of the present invention, the processor may be configured to obtain a probability corresponding to each of a plurality of output cross-sectional views, and determine, through a decision tree, a cross-sectional view having the highest value among the corresponding probabilities as a cross-sectional view for a Doppler cardiac ultrasound image.

[0043] Specific details of other embodiments are included in the detailed description and drawings.

[0044] The present invention provides an information providing system for Doppler ultrasound image classification based on an artificial neural network configured to classify ultrasound cross-sectional views using Doppler cardiac ultrasound images, thereby providing highly reliable cardiac ultrasound diagnosis results.

[0045] More specifically, the present invention recognizes a cross-sectional view of a Doppler ultrasound image and classifies it using a classification model, thereby being able to classify with high accuracy which cross-sectional view a Doppler image corresponds to.

[0046] That is, the present invention provides an information providing system for Doppler ultrasound image classification based on an artificial neural network, thereby enabling the classification of cross-sectional views of Doppler cardiac ultrasound images with high accuracy regardless of the level of skill of medical staff, thereby contributing to more accurate decision-making and treatment planning at the image analysis stage.

[0047] The effects according to the present invention are not limited to those exemplified above, and more diverse effects are included in this specification.

[0048] FIG. 1 illustrates a system for providing information on Doppler ultrasound image classification using a device for providing information on Doppler ultrasound image classification according to one embodiment of the present invention.

[0049] FIG. 2a is a block diagram showing the configuration of a medical device according to one embodiment of the present invention.

[0050] FIG. 2b is a block diagram showing the configuration of an information providing server according to one embodiment of the present invention.

[0051] FIG. 3 illustrates a procedure of a method for providing information on Doppler ultrasound image classification according to one embodiment of the present invention.

[0052] FIG. 4 illustrates an example of a process for deriving a probability corresponding to each cross-section derived based on a distance-based label generation function zai according to one embodiment of the present invention.

[0053] FIG. 5 is an exemplary diagram illustrating a final Doppler ultrasound image classification decision tree based on the probability value of a network learned using a Doppler cross-section distance graph for Doppler ultrasound image classification according to one embodiment of the present invention.

[0054] FIG. 6 is an exemplary diagram illustrating a Doppler cross-sectional distance graph created using each cross-sectional view of a Doppler ultrasound image for Doppler ultrasound image classification according to one embodiment of the present invention.

[0055] FIG. 7 is an exemplary diagram illustrating a gyro function graph for outputting a probability vector reflecting the distance of a cross-sectional view according to one embodiment of the present invention.

[0056] The advantages of the invention and the methods for achieving them will become clearer with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms. These embodiments are provided solely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention. The present invention is defined solely by the scope of the claims.

[0057] The shapes, sizes, ratios, angles, numbers, etc. disclosed in the drawings for explaining embodiments of the present invention are exemplary, and therefore the present invention is not limited to the matters illustrated. In addition, in describing the present invention, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. When the terms “includes,” “has,” and “consists of” are used in this specification, other parts may be added unless “only” is used. When a component is expressed in the singular, it includes a case where the plural is included unless there is a specifically explicit description.

[0058] When interpreting components, it is interpreted as including the error range even if there is no separate explicit description.

[0059] The individual features of the various embodiments of the present invention can be partially or wholly combined or combined with each other, and as can be fully understood by those skilled in the art, various technical connections and operations are possible, and each embodiment can be implemented independently of each other or can be implemented together in a related relationship.

[0060] For clarity in the interpretation of this specification, the terms used in this specification are defined below.

[0061] The term "subject" as used herein may refer to any subject seeking information regarding Doppler ultrasound image classification. Meanwhile, the subject disclosed herein may be any mammal other than a human, but is not limited thereto.

[0062] As used herein, the term "ultrasound image" may be an ultrasound image of the heart that can be obtained by a non-invasive method.

[0063] Meanwhile, the ultrasound image may be a still image or a video composed of multiple cuts. For example, the video may be classified into cross-sectional views for each frame of the Doppler cardiac ultrasound image according to a method for providing information on Doppler ultrasound images according to an embodiment of the present invention. As a result, the present invention can provide a streaming service by performing cross-sectional view classification simultaneously with the reception of the Doppler cardiac ultrasound image from an imaging diagnostic device, and can also provide Doppler ultrasound image information in real time.

[0064] Meanwhile, ultrasound images may be two-dimensional videos, but are not limited thereto and may also be three-dimensional images.

[0065] As used herein, the term "Doppler cardiac ultrasound image" or "Doppler ultrasound image" may refer to an image capable of non-invasively analyzing the velocity of blood flow within a blood vessel using various ultrasound modes. In this case, Doppler ultrasound images can provide blood flow information for various cross-sectional views.

[0066] Within this specification, "cross-sectional view" refers to a cross-sectional view for a Doppler ultrasound image, which may include a cross-sectional view for a pulsed wave (PW) Doppler mode, a cross-sectional view for a continuous wave (CW) Doppler mode, and a cross-sectional view for a tissue Doppler imaging (TDI) mode.

[0067] More specifically, “cross-sectional views for Doppler ultrasound imaging” may include cross-sectional views of MV inflow PW, AL (LVOT) PW, PV (RVOT) PW and pulmonary vein for intermittent wave Doppler mode, cross-sectional views of AV (AS) CW, PV (PS) CW, MV (MR) - CW, TV (TR) - CW, AV (AR) - CW, PV (PR) - CW and MV (MS) - CW) for continuous wave Doppler mode, and cross-sectional views of Septal Annulus PW TDI and Lateral Annulus PW TDI for tissue Doppler imaging mode.

[0068] According to a feature of the present invention, the ultrasound image may be an image in DICOM format that includes metadata for the ultrasound image.

[0069] Here, “metadata” can correspond to image DICOM tag information.

[0070] For example, a Doppler ultrasound image can be acquired by cropping only the relevant area based on the coordinate values ​​of the DICOM header.

[0071] At this time, the metadata may further include information such as whether the Doppler ultrasound image is a still image or a video, a color image or a black and white image.

[0072] Meanwhile, Doppler cardiac ultrasound images for multiple cross-sections can have ultrasound cross-sections recognized and classified within the images regardless of the Doppler ultrasound mode by a cross-section classification model.

[0073] As used herein, the term "cross-sectional view classification model" may be a model configured to take a Doppler echocardiographic image as input and output a prediction corresponding to each of MV inflow PW, AV (LVOT) PW, PV (RVOT) PW, AV (AS) - CW, PV (PS) - CW, MV (MR) - CW, TV (TR) - CW, AV (AR) - CW, PV (PR) - CW, MV (MS) - CW, pulmonary vein, Septal Annulus TDI, and Lateral Annulus TDI for a cross-sectional view of the Doppler ultrasound image, respectively.

[0074] According to a feature of the present invention, the cross-sectional classification model may be a model trained to classify cross-sectional views in various Doppler modes based on a learning Doppler echocardiographic image. In this case, the learning Doppler echocardiographic image may be an image in which cross-sectional views in PW, CW, and TDI Doppler modes are each labeled.

[0075] According to another feature of the present invention, the cross-sectional classification model may be a model having 13 output nodes trained to classify 13 cross-sectional views of MV inflow PW, AV (LVOT) PW, PV (RVOT) PW, AV (AS) - CW, PV (PS) - CW, MV (MR) - CW, TV (TR) - CW, AV (AR) - CW, PV (PR) - CW, MV (MS) - CW, pulmonary vein, Septal Annulus TDI, and Lateral Annulus TDI for Doppler ultrasound images. However, the present invention is not limited thereto.

[0076] According to another feature of the present invention, the cross-sectional view classification model may include a step of calculating a distance value from a Doppler view distance graph, inputting the distance value calculated using the classification model into a ξ function to output a matching probability vector, and calculating a probability corresponding to each cross-sectional view based on the probability vector. At this time, the Doppler view distance graph may be a binary tree, and each cross-sectional view may be arranged in the binary tree according to the degree of interrelationship between each cross-sectional view.

[0077] According to another feature of the present invention, the cross-sectional classification model may further include a step of inputting the calculated distance value into the ξ function and outputting a matching probability vector. At this time, the ξ function is

[0078]

[0079] ξ(d) = (1 / (d+1))^lambda

[0080]

[0081] , d is the distance between the Doppler cross sections obtained through binary (d>=0), and lambda(λ) is a hyperparameter value that determines how close to 0 the probability is mapped as the distance increases, and can be any real number greater than 1.

[0082]

[0083] That is, in various embodiments of the present invention, the cross-sectional classification model can classify cross-sectional views with high accuracy regardless of the mode of the input Doppler ultrasound image.

[0084] Meanwhile, the cross-sectional classification model may be based on DBNet, but is not limited thereto. For example, the classification models may be based on at least one algorithm selected from among U-net, VGG net, DenseNet, and a Fully Convolutional Network (FCN) with an encoder-decoder structure, a deep neural network (DNN) such as SegNet, DeconvNet, DeepLAB V3+, SqueezeNet, Alexnet, ResNet18, MobileNet-v2, GoogLeNet, Resnet-v2, Resnet50, RetinaNet, Resnet101, and Inception-v3.

[0085] Furthermore, the cross-sectional classification model may be an ensemble model based on at least two algorithm models among the aforementioned algorithms.

[0086]

[0087] Hereinafter, with reference to FIGS. 1, 2a and 2b, a system for providing information on Doppler ultrasound image classification using a device for providing information on Doppler ultrasound image classification according to one embodiment of the present invention and a device for providing information on Doppler ultrasound images will be described.

[0088] FIG. 1 illustrates a system for providing information on Doppler ultrasound image classification using a device for providing information on Doppler ultrasound image classification according to one embodiment of the present invention.

[0089] FIG. 2A illustrates an exemplary configuration of a medical device that provides information on Doppler ultrasound image classification according to one embodiment of the present invention.

[0090] FIG. 2b illustrates an example configuration of a device for providing information on Doppler ultrasound image classification according to one embodiment of the present invention.

[0091] First, referring to FIG. 1, the information providing system (1000) may be a system configured to provide information related to a Doppler ultrasound image based on a Doppler cardiac ultrasound image of an object. At this time, the information providing system (1000) may be configured with a medical device (100) that receives information related to Doppler ultrasound image classification, a Doppler cardiac ultrasound image diagnosis device (200) that provides a Doppler cardiac ultrasound image, and an information providing server (300) that generates information related to Doppler ultrasound image classification based on the received Doppler cardiac ultrasound image.

[0092] First, next, the medical device (100) is an electronic device that provides a user interface for displaying information related to Doppler ultrasound image classification, and may include at least one of a smartphone, a tablet PC (Personal Computer), a laptop, and / or a PC.

[0093] The medical device (100) can receive prediction results related to classification of Doppler ultrasound images for an object from the information providing server (300) and display the received results through a display unit to be described later.

[0094] The information providing server (300) may include a general-purpose computer, laptop, and / or data server that performs various operations to determine information related to Doppler ultrasound image classification based on a Doppler cardiac ultrasound image provided from a Doppler cardiac ultrasound imaging diagnostic device (200), such as an ultrasound diagnostic device. In this case, the information providing server (300) may be a device for accessing a web server that provides a web page or a mobile web server that provides a mobile website, but is not limited thereto.

[0095] More specifically, the information providing server (300) can receive a Doppler cardiac ultrasound image from a Doppler cardiac ultrasound imaging diagnostic device (200), classify the ultrasound mode and cross-section of the received Doppler cardiac ultrasound image, and provide information related to the Doppler ultrasound image. At this time, the information providing server (300) can classify the ultrasound cross-section from the Doppler cardiac ultrasound image using a classification model.

[0096] The information providing server (300) can provide the cross-sectional classification results for the Doppler ultrasound image to the medical device (100).

[0097] Information provided from the information provision server (300) in this manner may be provided as a web page via a web browser installed on a medical device (100), or may be provided in the form of an application or program. In various embodiments, such data may be provided in a form included in a platform in a client-server environment.

[0098] Next, with reference to FIGS. 2a and 2b, the components of the information providing server (300) of the present invention will be specifically described.

[0099] First, referring to FIG. 2A, the medical device (100) may include a memory interface (110), one or more processors (120), and a peripheral interface (130). Various components within the medical device (100) may be connected by one or more communication buses or signal lines.

[0100] The memory interface (110) is connected to the memory (150) and can transmit various data to the processor (120). Here, the memory (150) can include at least one type of storage medium among flash memory type, hard disk type, multimedia card micro type, card type memory (e.g., SD or XD memory, etc.), RAM, SRAM, ROM, EEPROM, PROM, network storage, cloud, and blockchain data.

[0101] In various embodiments, the memory (150) can store at least one of an operating system (151), a communication module (152), a graphical user interface (GUI) module (153), a sensor processing module (154), a telephone module (155), and an application module (156). Specifically, the operating system (151) can include instructions for processing basic system services and instructions for performing hardware operations. The communication module (152) can communicate with at least one of other devices, computers, and servers. The graphical user interface (GUI) module (153) can process a graphical user interface. The sensor processing module (154) can process sensor-related functions (e.g., processing voice input received using one or more microphones (192)). The telephone module (155) can process telephone-related functions. The application module (156) can perform various functions of the user application, such as electronic messaging, web browsing, media processing, navigation, imaging, and other processing functions. In addition, the medical device (100) can store one or more software applications (156-1, 156-2) (e.g., information providing applications) associated with a type of service in the memory (150).

[0102] In various embodiments, the memory (150) can store a digital assistant client module (157) (hereinafter, DA client module), and accordingly, can store commands for performing client-side functions of the digital assistant and various user data (158).

[0103] Meanwhile, the DA client module (157) can obtain the user's voice input, text input, touch input, and / or gesture input through various user interfaces (e.g., I / O subsystem (140)) provided in the medical device (100).

[0104] Additionally, the DA client module (157) can output data in audiovisual and tactile forms. For example, the DA client module (157) can output data consisting of a combination of at least two or more of voice, sound, notification, text message, menu, graphic, video, animation, and vibration. In addition, the DA client module (157) can communicate with a digital assistant server (not shown) using a communication subsystem (180).

[0105] In various embodiments, the DA client module (157) may collect additional information about the surroundings of the medical device (100) from various sensors, subsystems, and peripheral devices to construct a context associated with the user input. For example, the DA client module (157) may provide context information along with the user input to a digital assistant server to infer the user's intent. Here, the context information that may accompany the user input may include sensor information, such as lighting, ambient noise, ambient temperature, images of the surrounding environment, video, etc. As another example, the context information may include the physical state of the medical device (100) (e.g., device orientation, device location, device temperature, power level, speed, acceleration, motion pattern, cellular signal strength, etc.). As another example, the context information may include information related to the software state of the medical device (100) (e.g., processes running on the medical device (100), installed programs, past and current network activity, background services, error logs, resource usage, etc.).

[0106] In various embodiments, the memory (150) may include additional or deleted instructions, and further may include additional configurations other than those illustrated in FIG. 2A of the medical device (100), or may exclude some configurations.

[0107] The processor (120) can control the overall operation of the medical device (100) and execute various commands to implement an interface that provides information related to Doppler ultrasound images by running an application or program stored in the memory (150).

[0108] The processor (120) may correspond to a computing device such as a CPU (Central Processing Unit) or an AP (Application Processor). In addition, the processor (120) may be implemented in the form of an integrated chip (Integrated Chip (IC)) such as a SoC (System on Chip) in which various computing devices such as an NPU (Neural Processing Unit) are integrated.

[0109] The peripheral interface (130) can be connected to various sensors, subsystems, and peripheral devices to provide data so that the medical device (100) can perform various functions. Here, it can be understood that the medical device (100) performs a certain function as being performed by the processor (120).

[0110] The peripheral interface (130) can receive data from a motion sensor (160), a light sensor (light sensor) (161), and a proximity sensor (162), through which the medical device (100) can perform orientation, light, and proximity detection functions, etc. For another example, the peripheral interface (130) can receive data from other sensors (163) (positioning system - GPS receiver, temperature sensor, biometric sensor), through which the medical device (100) can perform functions related to the other sensors (163).

[0111] In various embodiments, the medical device (100) may include a camera subsystem (170) connected to a peripheral interface (130) and an optical sensor (171) connected thereto, which may enable the medical device (100) to perform various photographing functions, such as taking pictures and recording video clips.

[0112] In various embodiments, the medical device (100) may include a communication subsystem (180) connected to a peripheral interface (130). The communication subsystem (180) may be comprised of one or more wired / wireless networks and may include various communication ports, radio frequency transceivers, and optical transceivers.

[0113] In various embodiments, the medical device (100) includes an audio subsystem (190) coupled to a peripheral interface (130), the audio subsystem (190) including one or more speakers (191) and one or more microphones (192), such that the medical device (100) can perform voice-activated functions, such as voice recognition, voice replication, digital recording, and telephony functions.

[0114] In various embodiments, the medical device (100) may include an I / O subsystem (140) connected to a peripheral interface (130). For example, the I / O subsystem (140) may control a touch screen (143) included in the medical device (100) via a touch screen controller (141). As an example, the touch screen controller (141) may detect a user's contact and movement or cessation of contact and movement using any one of a plurality of touch sensing technologies, such as capacitive, resistive, infrared, surface acoustic wave technology, proximity sensor array, etc. As another example, the I / O subsystem (140) may control other input / control devices (144) included in the medical device (100) via other input controller(s) (142). As an example, the other input controller(s) (142) may control one or more buttons, rocker switches, thumb-wheels, infrared ports, USB ports, and pointer devices such as a stylus.

[0115] Next, referring to FIG. 2b, the information providing server (300) may include a communication interface (310), a memory (320), an I / O interface (330), and a processor (340), each component of which may communicate with each other via one or more communication buses or signal lines.

[0116] The communication interface (310) can be connected to a medical device (100) and a Doppler cardiac ultrasound imaging device (200) via a wired / wireless communication network to exchange data. For example, the communication interface (310) can receive a Doppler cardiac ultrasound image from the Doppler cardiac ultrasound imaging device (200) and transmit information about the determined cross-section to the medical device (100).

[0117] Meanwhile, the communication interface (310) that enables transmission and reception of such data includes a wired communication port (311) and a wireless circuit (312), wherein the wired communication port (311) may include one or more wired interfaces, for example, Ethernet, Universal Serial Bus (USB), FireWire, etc. In addition, the wireless circuit (312) may transmit and receive data with an external device via an RF signal or an optical signal. In addition, the wireless communication may use at least one of a plurality of communication standards, protocols, and technologies, for example, GSM, EDGE, CDMA, TDMA, Bluetooth, Wi-Fi, VoIP, Wi-MAX, or any other suitable communication protocol.

[0118] The memory (320) can store various data used in the information providing server (300). For example, the memory (320) can store a Doppler cardiac ultrasound image or a cross-sectional classification model learned to classify ultrasound cross-sections within a Doppler cardiac ultrasound image.

[0119] In various embodiments, the memory (320) may include a volatile or non-volatile storage medium capable of storing various data, commands, and information. For example, the memory (320) may include at least one type of storage medium among flash memory type, hard disk type, multimedia card micro type, card type memory (e.g., SD or XD memory, etc.), RAM, SRAM, ROM, EEPROM, PROM, network storage storage, cloud, and blockchain data.

[0120] In various embodiments, the memory (320) may store a configuration of at least one of an operating system (321), a communication module (322), a user interface module (323), and one or more applications (324).

[0121] An operating system (321) (e.g., an embedded operating system such as LINUX, UNIX, MAC OS, WINDOWS, VxWorks, etc.) may include various software components and drivers to control and manage general system operations (e.g., memory management, storage device control, power management, etc.) and may support communication between various hardware, firmware, and software components.

[0122] The communication module (323) can support communication with other devices via the communication interface (310). The communication module (320) can include various software components for processing data received by the wired communication port (311) or wireless circuit (312) of the communication interface (310).

[0123] The user interface module (323) can receive a user's request or input from a keyboard, touch screen, microphone, etc. through an I / O interface (330) and provide a user interface on the display.

[0124] The application (324) may include a program or module configured to be executed by one or more processors (330). Here, the application for providing information associated with Doppler ultrasound images may be implemented on a server farm.

[0125] The I / O interface (330) can connect at least one of an input / output device (not shown) of the information providing server (300), such as a display, a keyboard, a touch screen, and a microphone, to the user interface module (323). The I / O interface (330) can receive user input (e.g., voice input, keyboard input, touch input, etc.) together with the user interface module (323) and process a command according to the received input.

[0126] The processor (340) is connected to a communication interface (310), a memory (320), and an I / O interface (330) to control the overall operation of the information providing server (300), and can perform various commands for providing information through an application or program stored in the memory (320).

[0127] The processor (340) may correspond to a computing device such as a CPU (Central Processing Unit) or an AP (Application Processor). In addition, the processor (340) may be implemented in the form of an integrated chip (Integrated Chip (IC)) such as a SoC (System on Chip) in which various computing devices are integrated. Alternatively, the processor (340) may include a module for calculating an artificial neural network model, such as an NPU (Neural Processing Unit).

[0128] In various embodiments, the processor (340) can be configured to classify and provide cross-sectional views within a Doppler echocardiography image using classification models.

[0129]

[0130] Hereinafter, with reference to FIGS. 3 to 7, a method for providing information on a Doppler ultrasound image according to one embodiment of the present invention will be specifically described.

[0131] FIG. 3 illustrates a procedure of a method for providing information on Doppler ultrasound image classification according to one embodiment of the present invention.

[0132] FIG. 4 illustrates an example process of how to replace an existing one-hot vector using a gyro function to generate a label for Doppler ultrasound image classification learning according to one embodiment of the present invention.

[0133] FIG. 5 is an exemplary diagram illustrating a Doppler cross-sectional distance graph created using each cross-sectional view of a Doppler ultrasound image for Doppler ultrasound image classification according to one embodiment of the present invention.

[0134] FIG. 6 illustrates an example of a gyro function graph for outputting a probability vector of a cross-sectional view according to one embodiment of the present invention.

[0135] FIG. 7 is an exemplary diagram illustrating a gyro function graph for outputting a probability vector reflecting the distance of a cross-sectional view according to one embodiment of the present invention.

[0136]

[0137] First, referring to FIG. 3, the information provision procedure according to one embodiment of the present invention is as follows.

[0138] First, a Doppler echocardiographic image of an object is received (S310). Next, a cross-sectional view is extracted using the Doppler echocardiographic image as input (S320). The cross-sectional view of the Doppler echocardiographic image is classified using a classification model (S330).

[0139] More specifically, in the step (S310) where a Doppler cardiac ultrasound image is received, a Doppler cardiac ultrasound image of the target area, i.e., the heart area, can be received.

[0140] According to a feature of the present invention, in the step (S310) of receiving a Doppler cardiac ultrasound image, a cardiac ultrasound image in DICOM format can be received. At this time, the cardiac ultrasound image in DICOM format can include metadata such as coordinates for the Doppler cardiac ultrasound image.

[0141] Next, a step (S320) of extracting a cross-sectional view by inputting a Doppler ultrasound image is performed.

[0142] According to a feature of the present invention, the cross-sectional view may be a Doppler mode cross-sectional view of any one of MV inflow PW, AV (LVOT) PW, PV (RVOT) PW, AV (AS) - CW, PV (PS) - CW, MV (MR) - CW, TV (TR) - CW, AV (AR) - CW, PV (PR) - CW, MV (MS) - CW, pulmonary vein, Septal Annulus TDI and Lateral Annulus TDI.

[0143] Next, a step (S330) of classifying a cross-sectional view of a Doppler cardiac ultrasound image using a classification model is performed.

[0144] According to a feature of the present invention, in the step (S330) of classifying a cross-sectional view for a Doppler cardiac ultrasound image using a classification model, a probability corresponding to each of a plurality of cross-sectional views for the input Doppler image can be calculated and output by the cross-sectional view classification model.

[0145] At this time, referring to FIG. 4, the classification model calculates a distance value from a Doppler view distance graph, inputs the calculated distance value into a gyro function to output a matching probability vector, and based on the output probability vector, obtains a probability corresponding to each of a plurality of cross-sections, and determines the cross-section having the highest value of probability among the cross-sections as a cross-section for a Doppler cardiac ultrasound image.

[0146]

[0147] For example, referring to FIG. 5 together, in the step (S330) of classifying a cross-section for a Doppler cardiac ultrasound image using a classification model, each of the Doppler cardiac ultrasound images (412a) in the cardiac ultrasound image (412) is input into the cross-section classification model (420).

[0148] At this time, the cross-section may include a cross-section for pulsed wave (PW) Doppler mode, a cross-section for continuous wave (CW) Doppler mode, and a cross-section for tissue Doppler imaging (TDI) mode.

[0149] At this time, the step of classifying the cross-section may include a step of classifying the cross-section according to the Doppler mode of the cross-section for the intermittent Doppler mode, the cross-section for the continuous wave Doppler mode, or the cross-section for the tissue Doppler image mode, based on the received image, using a cross-section classification model.

[0150] According to another feature of the present invention, a Doppler echocardiographic image includes a baseline that serves as a reference for the direction of the Doppler flow, and the step of classifying a cross-section according to a Doppler mode of the information providing method may further include a step of classifying the cross-section according to the Doppler mode in detail based on the position of the baseline.

[0151] According to another feature of the present invention, the cross-section is a cross-section for intermittent wave Doppler mode (PW), and the step of subdividing may further include a step of determining the cross-section as MV inflow PW when the baseline is located at the top of the Doppler echocardiographic image, or determining the cross-section as AV (LVOT) PW or PV (RVOT) PW when the baseline is located at the bottom of the Doppler echocardiographic image.

[0152] According to another feature of the present invention, the cross-section is a cross-section for a continuous wave Doppler mode (CW), and the step of subdividing may further include a step of determining the cross-section as at least one of AV (AS) - CW, PV (PS) - CW, MV (MR) - CW, and TV (TR) - CW when the baseline is located at the top of the Doppler echocardiographic image, or determining the cross-section as at least one of AV (AR) - CW, PV (PR) - CW, and MV (MS) - CW when the baseline is located at the bottom of the Doppler echocardiographic image.

[0153] According to another feature of the present invention, the cross-section is a cross-section for tissue Doppler imaging mode (TDI), and the step of classifying the details may further include a step of determining at least one of Septal annulus TDI or Lateral annulus TDI.

[0154] According to another feature of the present invention, the information providing method may further include, prior to the receiving step, a step of receiving an ultrasound image in a DICOM (Digital Imaging and Communications in Medicine) format that displays metadata including tagging information for the Doppler cardiac ultrasound image, and a step of acquiring a Doppler cardiac ultrasound image based on the metadata.

[0155] According to another feature of the present invention, the step of classifying the cross-sectional view may include the step of determining, using a cross-sectional view classification model, at least one Doppler mode cross-sectional view among MV inflow PW, AV (LVOT) PW, PV (RVOT) PW, AV (AS) - CW, PV (PS) - CW, MV (MR) - CW, TV (TR) - CW, AV (AR) - CW, PV (PR) - CW, MV (MS) - CW, pulmonary vein, Septal Annulus TDI, and Lateral Annulus TDI for a cardiac ultrasound image.

[0156]

[0157] According to another feature of the present invention, in the step of classifying a cross-section, the cross-section classification model can calculate a distance value from a Doppler cross-section distance graph.

[0158] For example, referring to FIG. 6, the Doppler view distance graph may be a binary tree, and each view may be arranged in the binary tree according to the degree of interrelation between each view. The binary tree may be generated through the degree of similarity between each view. For example, MV(MS) CW and MV(MR) CW, which contain hemodynamic information targeting the same valve as MV inflow PW, may be binary trees designed to exhibit relatively smaller distances compared to PV(PR) CW, PV(RVOT) PW, PV(PS) CW, which target other valves.

[0159] In one embodiment of the present invention, d(yi, yj) can represent the distance between two nodes as follows.

[0160]

[0161]

[0162] At this time, the cross-section to be classified is i, and all remaining cross-sections can be represented as ㄱi, and when i is MV inflow PW, the distance is

[0163]

[0164] can be expressed as

[0165] That is, when i is MV inflow PW, the distance value to MV inflow PW is 0, and when i is MV inflow PW, the distance to PV (PR) CW includes Up, MV, Atrioventicular Valves, Image, Ventriculoarterial Valves, PV, Up, and PV (PR) CW nodes because the common node is Image, so the distance value is 8, and when i is MV inflow PW, the distance to TV (TR) CW includes Up, MV, Atrioventricular Valves, TV, Down, and TV (TR) CW because the common node is Atrioventricular Valves, so the distance value can be 6.

[0166]

[0167] According to another feature of the present invention, in the step of classifying the cross-section, the cross-section classification model can input a distance value calculated from the Doppler cross-section distance graph into a gyro function and output a matching probability vector.

[0168] For example, referring to FIG. 7, the distance value derived above can be input into the ξ function to output a matching probability vector.

[0169] According to another feature of the present invention, the step of generating a label of a cross-sectional view for a Doppler cardiac ultrasound image to train a classification model may further include a step of inputting the calculated distance value into a ξ function and outputting a matching probability vector. In this case, the ξ function is

[0170] ξ(d) = (1 / (d+1))^lambda

[0171] , d is the distance value between the Doppler cross-sections obtained through binary (d>=0), and lambda (λ) is a hyperparameter value that determines how close the probability is to 0 as the distance increases, and can be a number exceeding 1. Here, the final label is generated through the softmax function to make it a value between 0 and 1 after passing through the gyro function, and the optimal lambda value is selected when the validation loss is the lowest value or the validation accuracy is the highest.

[0172]

[0173] According to another feature of the present invention, the step of generating a label of a cross-section can calculate a probability corresponding to each cross-section by using the output probability vector function as an input, and at this time, the probability corresponding to each cross-section can be calculated as a value between 0 and 1 using a softmax function.

[0174] According to another feature of the present invention, the step of classifying the cross-section may include a step of determining at least one Doppler mode cross-section among MV inflow PW, AV (LVOT) PW, PV (RVOT) PW, AV (AS) - CW, PV (PS) - CW, MV (MR) - CW, TV (TR) - CW, AV (AR) - CW, PV (PR) - CW, MV (MS) - CW, pulmonary vein or Septal Annulus TDI, and Lateral Annulus TDI, by using a probability corresponding to each cross-section calculated using a cross-section classification model.

[0175]

[0176] The learning data of the cross-sectional classification model of the present invention is not limited to what has been described above, and the structure of the cross-sectional classification model is not limited to what has been described above.

[0177]

[0178] Although the embodiments of the present invention have been described in more detail with reference to the attached drawings, the present invention is not necessarily limited to these embodiments, and various modifications may be implemented without departing from the technical spirit of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain it, and the scope of the technical spirit of the present invention is not limited by these embodiments. Therefore, it should be understood that the embodiments described above are exemplary in all aspects and not restrictive. The protection scope of the present invention should be interpreted by the claims below, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of the rights of the present invention.

[0179] [Explanation of symbols]

[0180] 100: Medical Devices

[0181] 110: Memory interface 120: Processor

[0182] 130: Peripheral Interface 140: I / O Subsystem

[0183] 141: Touch screen controller 142: Other input controllers

[0184] 143: Touch screen

[0185] 144: Other input control devices

[0186] 150: Memory 151: Operating System

[0187] 152: Communication Module 153: GUI Module

[0188] 154: Sensor processing module 155: Phone module

[0189] 156: Applications

[0190] 156-1, 156-2: Applications

[0191] 157: Digital Assistant Client Module

[0192] 158: User data

[0193] 160: Motion sensor 161: Light sensor

[0194] 162: Proximity sensor 163: Other sensors

[0195] 170: Camera subsystem 171: Optical sensor

[0196] 180: Communication Subsystem

[0197] 190: Audio subsystem

[0198] 191: Speaker 192: Microphone

[0199] 300: Information server

[0200] 310: Communication Interface

[0201] 311: Wired communication port 312: Wireless circuit

[0202] 320: Memory

[0203] 321: Operating system 322: Communication module

[0204] 323: User Interface Module 324: Application

[0205] 330: I / O interface 340: Processor

Claims

1. A method for providing information on classification of Doppler cardiac ultrasound images implemented by a processor, A step of receiving a Doppler echocardiographic image of an object; A step of extracting a cross-sectional view using a Doppler echocardiographic image as input, and A step of classifying a cross-sectional diagram for a Doppler cardiac ultrasound image based on the extracted cross-sectional diagram using a cross-sectional diagram classification model learned to classify the cross-sectional diagram. Method for providing information on classification of Doppler ultrasound images.

2. In paragraph 1, The above extracted cross-sections include cross-sections for intermittent Doppler mode, continuous wave Doppler mode and tissue Doppler imaging mode, The steps of classifying the above cross-section are: A step of classifying a cross-section according to the Doppler mode of the intermittent Doppler mode, the continuous wave Doppler mode, or the tissue Doppler imaging mode based on the extracted cross-section, Method for providing information on classification of Doppler ultrasound images.

3. In paragraph 2, The cross-sectional view for the above intermittent wave Doppler mode, continuous wave Doppler mode and tissue Doppler imaging mode is any one of MV inflow PW, AV (LVOT) PW, PV (RVOT) PW, AV (AS) - CW, PV (PS) - CW, MV (MR) - CW, TV (TR) - CW, AV (AR) - CW, PV (PR) - CW, MV (MS) - CW, pulmonary vein, Septal Annulus TDI and Lateral Annulus TDI. Method for providing information on classification of Doppler ultrasound images.

4. In paragraph 1, The steps of classifying the above cross-section are: Further comprising a step of calculating a distance value corresponding to each cross-section in the Doppler cross-section distance graph, Method for providing information on classification of Doppler ultrasound images.

5. In paragraph 4, The above Doppler cross-section distance graph is a binary tree, and the cross-sections are arranged in the binary tree according to the degree of correlation between each cross-section. Method for providing information on classification of Doppler ultrasound images.

6. In paragraph 4, The step of classifying the above cross-section is: It further includes a step of inputting the calculated distance value into a zai function and outputting a matching probability vector, and the zai function is: ξ(d) = (1 / (d+1))^λ It is, Method for providing information on classification of Doppler ultrasound images.

7. In paragraph 6, The steps of classifying the above cross-section are: Further comprising a step of obtaining a probability corresponding to each cross-section using the probability vector output above. Method for providing information on classification of Doppler ultrasound images.

8. In paragraph 7, The steps of classifying the above cross-section are: Further comprising a step of converting the obtained probability into a value between 0 and 1 using a softmax function. Method for providing information on classification of Doppler ultrasound images.

9. In paragraph 8, The steps of classifying the above cross-section are: According to the probability corresponding to each cross-section obtained above For the above cardiac ultrasound image, a step of determining one Doppler mode cross-section having the highest probability value among MV inflow PW, AV (LVOT) PW, PV (RVOT) PW, AV (AS) - CW, PV (PS) - CW, MV (MR) - CW, TV (TR) - CW, AV (AR) - CW, PV (PR) - CW, MV (MS) - CW, pulmonary vein, Septal Annulus TDI and Lateral Annulus TDI as a cross-section for the Doppler cardiac ultrasound image is further included. Method for providing information on classification of Doppler ultrasound images.

10. A communication unit configured to receive a Doppler echocardiographic image of an object, and including a processor functionally connected to the above communication unit, The above processor, A cross-sectional diagram is extracted from the Doppler echocardiographic image as input and a cross-sectional diagram classification model learned to classify the cross-sectional diagram is used to classify the cross-sectional diagram for the Doppler echocardiographic image based on the received Doppler echocardiographic image. A device for providing information on classification of Doppler echocardiographic images.

11. In paragraph 10, The above cross-section includes a cross-section for intermittent wave Doppler mode, a cross-section for continuous wave Doppler mode and a cross-section for tissue Doppler imaging mode, The above classification model is, Based on the received Doppler echocardiographic image, a cross-sectional view for the intermittent Doppler mode, a cross-sectional view for the continuous wave Doppler mode, or a cross-sectional view according to the Doppler mode of the tissue Doppler imaging mode is extracted and classified. A device for providing information on classification of Doppler echocardiographic images.

12. In paragraph 11, The cross-sectional view for the above extracted intermittent wave Doppler mode, continuous wave Doppler mode and tissue Doppler image mode is any one of MV inflow PW, AV (LVOT) PW, PV (RVOT) PW, AV (AS) - CW, PV (PS) - CW, MV (MR) - CW, TV (TR) - CW, AV (AR) - CW, PV (PR) - CW, MV (MS) - CW, pulmonary vein, Septal Annulus TDI and Lateral Annulus TDI. A device for providing information on classification of Doppler echocardiographic images.

13. In paragraph 10, In the Doppler cross-section distance graph, the distance value corresponding to each cross-section is calculated, and the calculated distance value is converted to the ξ function. ξ(d) = (1 / (d+1))^λ Including an output section that outputs a matching probability vector by inputting A device for providing information on classification of Doppler echocardiographic images.

14. In paragraph 13, The above Doppler cross-section distance graph is a binary tree, and the cross-sections are arranged in the binary tree according to the degree of correlation between each cross-section. A device for providing information on classification of Doppler echocardiographic images.

15. In paragraph 14, The above processor, Obtain the probability corresponding to each of the cross-sections output through the output section, It is configured to determine the cross-section obtained through the decision tree as the cross-section for the Doppler cardiac ultrasound image. A device for providing information on Doppler ultrasound image classification.

16. In paragraph 15, The above processor, Further configured to convert the above-obtained probability into a value between 0 and 1 using a softmax function, A device for providing information on Doppler ultrasound image classification.

17. In paragraph 16, The above processor, According to the probability corresponding to each cross-section obtained above For the above cardiac ultrasound image, it is further configured to determine one Doppler mode cross-section having the highest probability value among MV inflow PW, AV (LVOT) PW, PV (RVOT) PW, AV (AS) - CW, PV (PS) - CW, MV (MR) - CW, TV (TR) - CW, AV (AR) - CW, PV (PR) - CW, MV (MS) - CW, pulmonary vein, Septal Annulus TDI and Lateral Annulus TDI as a cross-section for the Doppler cardiac ultrasound image. A device for providing information on Doppler ultrasound image classification.

Images