Method for quantifying breast composition

Abstract

To provide a method for quantifying a breast composition performed by a computing device according to one embodiment.SOLUTION: A method for quantifying a breast composition may further include: acquiring an ultrasound image; identifying a breast region in the ultrasound image; identifying a fibro-glandular tissue region in the ultrasound image; and quantifying at least one of a breast tissue component and a glandular tissue component based on the identified breast region and the identified fibro-glandular tissue region.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to a method for quantifying breast components, and more specifically, to a technique for quantifying breast tissue components or glandular tissue components by utilizing breast ultrasound images.

Background Art

[0002] The breast is composed of tissues in a ratio of fibro-glandular tissue (FGT) and fat. Since the composition of breast tissue is related to the sensitivity of breast cancer detection and the risk of occurrence, software for quantitative analysis thereof is required. When a reader subjectively performs quantification, there is a problem that the degree of quantification agreement among readers is low, and particularly when many readers perform quantification, it is less reliable.

[0003] In addition, for analyzing breast tissue, medical imaging techniques such as mammography and ultrasound can be used. Mammography has a limitation in that it is difficult to distinguish high-density breasts from risky breast tissues. On the other hand, ultrasound is widely used in diagnostic examinations due to advantages such as being able to distinguish breast tissue components, low cost, easy access, and safety from radiation exposure. However, there is a limitation in that the boundaries between ultrasound tissues are similar and difficult to distinguish, and the resolution is reduced due to video quality, shadows, and noise that depend on the user's proficiency. Therefore, there is a need for quantitative analysis software that utilizes ultrasound.

[0004] Patent Document 1 discloses an ultrasonic image processing method and apparatus, and a breast cancer diagnostic apparatus.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] The present disclosure aims to provide a method for quantifying components of the breast that can automatically and quantitatively measure at least one of Breast Tissue Component or Glandular Tissue Component from breast ultrasound using deep learning technology and classify grades.

[0007] On the other hand, the technical problems to be achieved by the present disclosure are not limited to the technical problems mentioned above, and various technical problems may be included within the scope obvious to those skilled in the art from the content described below.

Means for Solving the Problems

[0008] According to an embodiment of the present disclosure for realizing the above-mentioned problems, a method for quantifying components of the breast, executed by a computing device, is disclosed. The method may include: acquiring an ultrasound video; identifying a breast region in the ultrasound video; identifying a fibro-glandular tissue region in the ultrasound video; and quantifying at least one of Breast Tissue Component or Glandular Tissue Component based on the identified breast region and the identified fibro-glandular tissue region.

[0009] In one embodiment, the step of identifying a breast region in the ultrasound video may include: generating a breast inference region mask in the ultrasound video using a deep learning model; generating a breast region binary map based on the generated breast inference region mask; and generating a breast region segmentation mask based on the breast region binary map.

[0010] In one embodiment, the breast region segmentation mask can be obtained based on the largest region among a plurality of regions included in the breast region binary map.

[0011] In one embodiment, the breast inference region mask includes a breast region probability map, and based on the generated breast inference region mask, the step of generating a breast region binary map may include classifying the breast region probability map based on a predetermined threshold value to generate the breast region binary map.

[0012] In one embodiment, the step of identifying the fibro-glandular tissue region in the ultrasonic image

[0013] may include generating a fibro-glandular inference region mask in the ultrasonic image by utilizing a deep learning model; generating a fibro-glandular region binary map based on the generated fibro-glandular inference region mask; and generating a fibro-glandular region segmentation mask based on the fibro-glandular region binary map.

[0014] In one embodiment, the fibro-glandular inference region mask includes a fibro-glandular region probability map, and based on the generated fibro-glandular inference region mask, the step of generating a fibro-glandular region binary map may include classifying the fibro-glandular region probability map based on a predetermined threshold value to generate the fibro-glandular region binary map.

[0015] In one embodiment, based on the fibro-glandular region binary map, the step of generating a fibro-glandular region segmentation mask may include performing an operation on the breast region segmentation mask and the fibro-glandular region binary map to generate an intra-breast fibro-glandular region mask; and generating the fibro-glandular region segmentation mask based on the intra-breast fibro-glandular region mask.

[0016] In one embodiment, the fibro-glandular region segmentation mask can be obtained based on the largest region among a plurality of regions included in the intramammary fibro-glandular region mask.

[0017] In one embodiment, the step of quantifying at least one of the breast tissue component or the glandular tissue component based on the identified breast region and the identified fibro-glandular tissue region may include: identifying a glandular tissue region based on the identified fibro-glandular tissue region; and analyzing the identified fibro-glandular tissue region and the identified glandular tissue region to quantify the glandular tissue component.

[0018] In one embodiment, the step of identifying a glandular tissue region based on the identified fibro-glandular tissue region may include: generating a glandular tissue region binary map based on the fibro-glandular region segmentation mask; and generating a glandular tissue region segmentation mask based on the glandular tissue region binary map.

[0019] In one embodiment, the step of generating a glandular tissue region binary map based on the fibro-glandular region segmentation mask may include: extracting a glandular tissue region of interest mask with the fibro-glandular region segmentation mask; detecting a polygonal region of interest including the glandular tissue region of interest; performing histogram-based normalization on the ultrasonic image inside the polygonal region of interest; generating a region of interest normalized ultrasonic image by utilizing the ultrasonic image inside the polygonal region of interest on which the histogram-based normalization has been performed and the ultrasonic image outside the polygonal region of interest; performing an operation on the region of interest normalized ultrasonic image and the glandular tissue region of interest mask to generate a glandular tissue region of interest image; and classifying the glandular tissue region of interest image based on a predetermined threshold to generate the glandular tissue region binary map.

[0020] In one embodiment, the step of quantifying at least one of the breast tissue component or the glandular tissue component based on the identified breast region and the identified fibro-glandular tissue region may include: calculating the ratio of the breast tissue component by utilizing pixel values of a fibro-glandular region segmentation mask and pixel values of a breast region segmentation mask; and mapping the ratio of the breast tissue component to breast tissue component (BTC) grades.

[0021] In one embodiment, the step of quantifying at least one of the breast tissue component or the glandular tissue component based on the identified breast region and the identified fibro-glandular tissue region may include: calculating the ratio of the glandular tissue component by utilizing pixel values of a glandular tissue region segmentation mask and pixel values of a fibro-glandular region segmentation mask; and mapping the ratio of the glandular tissue component to glandular tissue component (GTC) grades.

[0022] According to one embodiment of the present disclosure for achieving the above-described problems, a computer program stored in a computer-readable recording medium is disclosed. When the computer program is executed by one or more processors, the one or more processors are caused to perform the following operations for quantifying components of a breast, the operations including: acquiring an ultrasonic image; identifying a breast region in the ultrasonic image; identifying a fibro-glandular tissue region in the ultrasonic image; and quantifying at least one of a breast tissue component or a glandular tissue component based on the identified breast region and the identified fibro-glandular tissue region.

[0023] In one embodiment, the operation of identifying the breast region in the ultrasonic image includes: an operation of generating a breast inference region mask in the ultrasonic image by utilizing a deep learning model; an operation of generating a breast region binary map based on the generated breast inference region mask; and an operation of generating a breast region segmentation mask based on the breast region binary map.

[0024] In one embodiment, the breast inference region mask includes a breast region probability map, and the operation of generating a breast region binary map based on the generated breast inference region mask may include an operation of classifying the breast region probability map based on a predetermined threshold value to generate the breast region binary map.

[0025] In one embodiment, the operation of identifying the fibro-glandular tissue region in the ultrasonic image includes: an operation of generating a fibro-glandular inference region mask in the ultrasonic image by utilizing a deep learning model; an operation of generating a fibro-glandular region binary map based on the generated fibro-glandular inference region mask; and an operation of generating a fibro-glandular region segmentation mask based on the fibro-glandular region binary map.

[0026] In one embodiment, the fibro-glandular inference region mask includes a fibro-glandular region probability map, and the operation of generating a fibro-glandular region binary map based on the generated fibro-glandular inference region mask may include an operation of classifying the fibro-glandular region probability map based on a predetermined threshold value to generate the fibro-glandular region binary map.

[0027] In one embodiment, the operation of quantifying at least one of the breast tissue component or the glandular tissue component based on the identified breast region and the identified fibro-glandular tissue region includes: an operation of identifying a glandular tissue region based on the identified fibro-glandular tissue region; and an operation of analyzing the identified fibro-glandular tissue region and the identified glandular tissue region to quantify the glandular tissue component.

[0028] In one embodiment, the operation of quantifying at least one of the breast tissue component or the glandular tissue component based on the identified breast region and the identified fibro-glandular tissue region may include an operation of calculating a ratio of the breast tissue component by utilizing pixel values of a fibro-glandular region segmentation mask and pixel values of a breast region segmentation mask; and an operation of mapping the ratio of the breast tissue component to a breast tissue component (BTC) grade.

[0029] In one embodiment, the operation of quantifying at least one of the breast tissue component or the glandular tissue component based on the identified breast region and the identified fibro-glandular tissue region may include an operation of calculating a ratio of the glandular tissue component by utilizing pixel values of a glandular tissue region segmentation mask and pixel values of a fibro-glandular region segmentation mask; and an operation of mapping the ratio of the glandular tissue component to a glandular tissue component (GTC) grade.

[0030] A computing device according to an embodiment of the present disclosure for realizing the above-described problem is disclosed. The device includes at least one processor; and a memory, and the at least one processor is configured to acquire an ultrasonic image, identify a breast region in the ultrasonic image, identify a fibro-glandular tissue region in the ultrasonic image, and quantify at least one of a breast tissue component or a glandular tissue component based on the identified breast region and the identified fibro-glandular tissue region.

[0031] In one embodiment, the at least one processor may be configured to generate a breast inference region mask in the ultrasonic image by utilizing a deep learning model, generate a breast region binary map based on the generated breast inference region mask, and generate a breast region segmentation mask based on the breast region binary map.

[0032] In one embodiment, the breast inference region mask includes a breast region probability map, and the at least one processor may be configured to classify the breast region probability map based on a predetermined threshold to generate the breast region binary map.

[0033] In one embodiment, the at least one processor utilizes a deep learning model to generate a fibroglandular inference region mask in the ultrasonic image; generates a fibroglandular region binary map based on the generated fibroglandular inference region mask; and may be configured to generate a fibroglandular region segmentation mask based on the fibroglandular region binary map.

[0034] In one embodiment, the fibroglandular inference region mask includes a fibroglandular region probability map, and the at least one processor may be configured to classify the fibroglandular region probability map based on a predetermined threshold to generate the fibroglandular region binary map.

[0035] In one embodiment, the at least one processor identifies a glandular tissue region based on the identified fibroglandular tissue region; and may be configured to analyze the identified fibroglandular tissue region and the identified glandular tissue region to quantify the glandular tissue component.

[0036] In one embodiment, the at least one processor utilizes the pixel values of the fibroglandular region segmentation mask and the pixel values of the breast region segmentation mask to calculate the ratio of the breast tissue component; and may be configured to map the ratio of the breast tissue component according to the breast tissue component (BTC) grade.

[0037] In one embodiment, the at least one processor utilizes the pixel values of the glandular tissue region segmentation mask and the pixel values of the fibroglandular region segmentation mask to calculate the ratio of the glandular tissue component; and may be configured to map the ratio of the glandular tissue component according to the glandular tissue component (GTC) grade.

Advantages of the Invention

[0038] The present disclosure utilizes deep learning technology to automatically and quantitatively measure at least one of a breast tissue component or a glandular tissue component, classify the grades, thereby assisting the reader in breast tissue analysis in breast ultrasound diagnosis and enhancing the inter-reader agreement and intra-reader agreement.

[0039] On the other hand, the effects of the present disclosure are not limited to the effects mentioned above, and various effects may be included within the scope obvious to a person skilled in the art from the content described below.

Brief Description of the Drawings

[0040]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Modes for Carrying Out the Invention

[0041] Various embodiments will be described below with reference to the drawings, and like reference numerals throughout the drawings are used to represent like components. Various explanations are presented herein to facilitate understanding of the present disclosure. However, these embodiments can be implemented without these specific explanations.

[0042] As used herein, terms such as "component", "module", "system", etc. refer to computer-related entities, hardware, firmware, software, combinations of software and hardware, or the execution of software. For example, a component can be, but is not limited to, a processing procedure (procedure) executed on a processor, a processor, an object, an execution thread, a program, and / or a computer. For example, both an application executed on a computing device and the computing device can be components. One or more components can reside within a processor and / or an execution thread, one component can be localized within one computer, or can be distributed among two or more computers. Also, such components can be executed from various computer-readable media having various data structures stored therein. A component can communicate through local and / or remote processing, for example, by signals having one or more data packets (e.g., data and / or signals from one component interacting with other components in a local system, a distributed system, and transmitted through a network such as the Internet to other systems).

[0043] The term "or" is used with the intention of meaning an inclusive "or" rather than an exclusive "or". That is, when not specifically specified and not clear from the context, "X uses A or B" shall be taken to mean one of the natural inclusive substitutions. That is, "X uses A or B" can be considered to apply to any of the following cases: X uses A; X uses B; or X uses both A and B. Also, the term "and / or" in this specification shall be understood to refer to all possible combinations of one or more of the listed related items and to include them.

[0044] Also, the term "comprising (including)" as a predicate and / or "comprising (including)" as a modifier shall be understood to mean that the said feature and / or component exists. However, the term "comprising (including)" as a predicate and / or "comprising (including)" as a modifier shall be understood not to exclude the existence or addition of one or more other further features, components and / or groups thereof. Also, when the number is not specifically specified or when it is not clear from the context that the singular form is indicated, the singular shall generally be interpreted to mean "one or more" in this specification and the claims.

[0045] And with respect to the term "at least one of A or B", it is desired to be interpreted to mean "the case where only A is included", "the case where only B is included", "the case of a combination of A and B".

[0046] One of ordinary skill in the art should further recognize that the various exemplary logical blocks, configurations, modules, circuits, means, logics, and algorithmic steps described as being related to the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the various exemplary components, blocks, configurations, means, logics, modules, circuits, and steps have been generally described in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints of the overall system. A skilled artisan can implement the described functionality in various ways for each particular application, provided that such implementation decisions do not be construed as departing from the scope of the present disclosure.

[0047] The description of the embodiments shown herein is provided so that those of ordinary skill in the art of the present disclosure can make use of or implement the present invention. Various modifications to these embodiments will be apparent to those of ordinary skill in the art of the present disclosure, and the general principles defined herein can be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure is not limited to the embodiments shown herein, but should be construed in the broadest scope consistent with the principles and novel features shown herein.

[0048] In the present disclosure, the network function, the artificial neural circuit network, and the neural network can be used interchangeably.

[0049] FIG. 1 is a block configuration diagram of a computing device for quantifying breast components according to an embodiment of the present disclosure.

[0050] The configuration of the computing device (100) illustrated in FIG. 1 is merely an exemplary illustration shown in a simplified manner. In one embodiment of the present disclosure, the computer device (100) may include other configurations for implementing the computing environment of the computer device (100), and it is also possible to configure the computer device (100) with only a part of the disclosed configurations.

[0051] The computer device (100) can include a processor (110), a memory (130), and a network unit (150).

[0052] The processor (100) can be composed of one or more cores and can include processors for data analysis and deep learning, such as a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), and a tensor processing unit (TPU). The processor (110) can read a computer program stored in the memory (130) and execute data processing for machine learning in an embodiment of the present disclosure. Based on an embodiment of the present disclosure, the processor (110) can perform operations for neural network learning. In deep learning (DL), the processor (110) can execute calculations for neural network learning, such as processing input data for learning, extracting features from the input data, calculating errors, and updating the weights of the neural network using backpropagation. At least one of the CPU, GPGPU, and TPU of the processor (110) can process network function learning. For example, both the CPU and GPGPU can perform network function learning and data classification using the network function. Note that in an embodiment of the present disclosure, the processors of multiple computing devices can be used together to perform network function learning and data classification using the network function. Also, the computer program executed in the computing device in an embodiment of the present disclosure can be a program executable by a CPU, GPGPU, or TPU.

[0053] In one embodiment of the present disclosure, the processor (110) can propose relevant content from user information related to the voice signal and the speech recognition (STT) result text. For example, the processor (110) (1) performs STT on the input audio (voice signal) (for example, in the case of a video file, separates the audio and video and performs it on the audio), (2) determines candidate target words by performing topic segmentation and keyword extraction on the STT result text, or determines candidate target words related to the audio based on user account information (for example, occupation, user dictionary, etc.), (3) compares the determined plurality of candidate target words with the speech recognition (STT) result text, and when a word with an edit distance less than the threshold is detected, determines the candidate target word as the target word (= core word), and (4) can propose content related to the speech recognition (STT) result by proposing content related to the target word.

[0054] In one embodiment of the present disclosure, the memory (130) can store any form of information generated or determined by the processor (110) and any form of information received by the network unit (550).

[0055] In one embodiment of the present disclosure, the memory (130) can include at least one type of storage medium such as a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read-Only Memory (ROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Programmable Read-Only Memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. The computing device (100) can also operate in cooperation with a web storage that executes the storage function of the memory (130) on the internet. The description regarding the foregoing memory is merely illustrative and the present disclosure is not limited thereto.

[0056] The network unit (150) in one embodiment of the present disclosure can use various wired communication systems such as a Public Switched Telephone Network (PSTN), x Digital Subscriber Line (xDSL), Rate Adaptive DSL (RADSL), Multi Rate DSL (MDSL), Very High Speed DSL (VDSL), Universal Asymmetric DSL (UADSL), High Bit Rate DSL (HDSL), and a Local Area Network (LAN).

[0057] In addition, the network unit (150) in this specification can utilize various wireless communication systems such as CDMA (Code Division Multi Access), TDMA (Time Division Multi Access), FDMA (Frequency Division Multi Access), OFDMA (Orthogonal Frequency Division Multi Access), SC-FDMA (Single Carrier-FDMA), and other systems.

[0058] The network unit (150) in the present disclosure can be configured regardless of the communication mode such as wired and wireless, and can be various communication networks such as a short-distance communication network (PAN: Personal Area Network), a short-range communication network (WAN: Wide Area Network), etc. Further, the network can be a known World Wide Web (WWW), and can also utilize wireless transmission technologies used for short-distance communication such as infrared (IrDA: Infrared Data Association) or Bluetooth (registered trademark).

[0059] The technology described in this specification can be used not only in the above-mentioned network but also in other networks.

[0060] FIG. 2 is a conceptual diagram showing a neural network according to an embodiment of the present disclosure.

[0061] Throughout this specification, an arithmetic model, a neural circuit network, a network function, and a neural network can be used in the same meaning. A neural circuit network is generally composed of a set of interconnected computing units called nodes. Such nodes can also be referred to as neurons. A neural circuit network is configured to include at least one or more nodes. The nodes (or neurons) constituting the neural circuit network can be interconnected by one or more links.

[0062] In a neural circuit network, one or more nodes connected via links can form a relationship of relative input nodes and output nodes. The concepts of input nodes and output nodes are relative. Any node that becomes an output node for a certain node can become an input node in the relationship with other nodes, and vice versa. As described above, the relationship between input nodes and output nodes can be established centered around links. One input node can be connected via links to one or more output nodes, and vice versa.

[0063] In the relationship between an input node and an output node connected via one link, it is possible for the value of the data of the output node to be determined based on the data input to the input node. Here, the node interconnecting the input node and the output node can have a weight value. The weight value can be variable and can be changed by the user or an algorithm for the neural circuit network to perform a desired function. For example, when one or more input nodes are interconnected to one output node by respective links, the output node can determine the value of the output node based on the values input to the input nodes connected to the output node and the weight values set for the respective links corresponding to the input nodes.

[0064] As described above, in a neural circuit network, one or more nodes are interconnected via one or more links to form a relationship of input nodes and output nodes within the neural circuit network. In a neural circuit network, the characteristics of the neural circuit network can be determined by the number of nodes and links, the correlation between nodes and links, and the values of the weight values assigned to each link. For example, if there are two neural circuit networks with the same number of nodes and links and different values of the weight values of the links, the two neural circuit networks can be recognized as different.

[0065] A neural circuit network can be composed of a set of one or more nodes. A subset of the nodes that make up the neural circuit network can form a layer. Among the multiple nodes that make up the neural circuit network, some can form one layer based on the distance from the first input node. For example, the set of nodes with a distance of n from the first input node can form the nth layer. The distance from the first input node can be defined based on the minimum number of links that must be traversed to reach the node from the first input node. However, such a definition of a layer is arbitrarily cited for explanation purposes, and the composition of layers in a neural circuit network can be defined in a way different from the above explanation. For example, the layer of nodes can also be defined based on the distance from the final output node.

[0066] The first input node can mean one or more nodes among the nodes in the neural circuit network where data is directly input without passing through a link in relation to other nodes. Or, in the network of the neural circuit network, in the relationship between nodes based on links, it can mean a node that does not have other input nodes connected via a link. Similarly, the final output node can mean one or more nodes among the nodes in the neural circuit network that do not have an output node in relation to other nodes. Also, a hidden node can mean a node that is not the first input node or the final output node and constitutes the nodes of the neural circuit network.

[0067] According to an embodiment of the present disclosure, the neural circuit network may be a neural circuit network in which the number of nodes in the input layer is the same as the number of nodes in the output layer, and as it progresses from the input layer to the hidden layer, the number of nodes first decreases and then increases again. According to an embodiment of the present disclosure, the neural circuit network may be a neural circuit network in which the number of nodes in the input layer is less than the number of nodes in the output layer, and as it progresses from the input layer to the hidden layer, the number of nodes decreases. Further, according to another embodiment of the present disclosure, the neural circuit network may be a neural circuit network in which the number of nodes in the input layer is more than the number of nodes in the output layer, and as it progresses from the input layer to the hidden layer, the number of nodes increases. The neural circuit network in another embodiment of the present disclosure may be a neural circuit network that combines the above-described neural circuit networks.

[0068] A deep neural network (DNN) can be meant to refer to a neural network that includes multiple hidden layers in addition to an input layer and an output layer. By using a deep neural network, the latent structures of data can be grasped. That is, it is possible to grasp the latent structures of photos, articles, videos, voices, music (for example, whether a certain object is shown in a photo, what the content and sentiment of an article are, what the content and sentiment of a voice are, etc.). Deep neural networks can include convolutional neural networks (CNNs), recurrent neural networks (RNNs), autoencoders, GANs (Generative Adversarial Networks), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), Q-networks, U-networks, Sham networks, Generative Adversarial Networks (GANs), etc. The aforementioned deep neural networks are merely illustrative and the present disclosure is not limited thereto.

[0069] In one embodiment of the present disclosure, the network function may also include an autoencoder. The autoencoder can be a type of artificial neural network for outputting output data similar to the input data. The autoencoder can include at least one hidden layer, and an odd number of hidden layers can be arranged between the input and output layers. The number of nodes in each layer can decrease from the number of nodes in the input layer towards the middle layer called the bottleneck layer (encoding), and can expand in a form symmetric to the reduction from the bottleneck layer towards the output layer (symmetric to the input layer). The autoencoder can perform non-linear dimensionality reduction. The number of input and output layers can correspond to the dimensions after preprocessing of the input data. In the autoencoder structure, the number of nodes in the hidden layers included in the encoder can have a structure that decreases as it gets farther from the input data. If the number of nodes in the bottleneck layer (the layer with the fewest number of nodes located between the encoder and the decoder) is too small, there is a possibility that not enough information will be transmitted, so it may be maintained above a certain number (for example, more than half of the input layer, etc.).

[0070] The neural network can be learned in at least one of supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. The learning of the neural network can be a process of providing the neural network with knowledge for the neural network to perform a specific operation.

[0071] A neural network can be trained in a direction to minimize the error of the output. In the training of a neural network, training data is repeatedly input into the neural network, the error between the output of the neural network regarding the training data and the target is calculated, and the error of the neural network is backpropagated from the output layer of the neural network to the input layer in the direction to reduce the error, and the weight values of each node of the neural network are updated. In the case of supervised learning, training data with correct answers labeled for each individual training data is used (that is, labeled training data), and in the case of unsupervised learning, there may be cases where correct answers are not labeled for individual training data. That is, for example, the training data in supervised learning regarding data classification can be data with categories labeled for each of the training data. By inputting the labeled training data into the neural network and comparing the output (category) of the neural network with the label of the training data, it is possible to calculate the error. As another example, in the case of unsupervised learning regarding data classification, it is possible to calculate the error by comparing the input training data with the output of the neural network. The calculated error is backpropagated in the reverse direction (that is, from the output layer to the input layer direction) in the neural network, and it is possible to update the connection weight values of each node of each layer of the neural network through backpropagation. The change amount of the connection weight value of each updated node can be determined by the learning rate. The calculation of the neural network for the input data and the backpropagation of the error can constitute the learning cycle (epoch). The application method of the learning rate can change depending on the number of repetitions of the learning cycle of the neural network. For example, in the initial stage of the neural network training, the learning rate can be increased to improve the efficiency by enabling the neural network to quickly secure a certain level of performance, and in the latter half of the training, the learning rate can be decreased to increase the accuracy.

[0072] In the learning of neural networks, generally, the training data can be a subset of the actual data (i.e., the data to be processed using the learned neural network). Therefore, there can be a learning cycle in which the error related to the training data decreases while the error related to the actual data increases. Overfitting is a phenomenon in which the error increases in the actual data because of excessive learning about the training data. For example, a neural network that has learned cats by seeing yellow cats may not be able to recognize a cat of a color other than yellow as a cat, which can be a kind of overfitting. Overfitting can cause an increase in the error of machine learning algorithms. To prevent such overfitting, various optimization methods can be applied. To prevent overfitting, methods such as increasing the training data, regularization, dropout (deactivating some of the nodes in the network during the learning process), and utilization of a batch normalization layer can be applied.

[0073] The present disclosure relates to a method for performing breast composition evaluation techniques, quantification of glandular tissue components, and grading by utilizing deep learning technology from breast ultrasound images. Further, the present disclosure is a technology related to software that automatically quantifies breast tissue components and glandular tissue components and performs grading using ultrasound.

[0074] FIG. 3 is a block configuration diagram of a plurality of modules for quantifying breast components according to an embodiment of the present disclosure, and FIG. 4 is a drawing schematically showing operations for quantifying breast components according to an embodiment of the present disclosure.

[0075] According to an embodiment of the present disclosure, the computing device 100 may include a breast region segmentation module 111, a fibroglandular region segmentation module 112, and a glandular tissue region segmentation module 113. On the other hand, a plurality of modules that may be included in such a computing device 100 may be controlled by the processor 110 or implemented by the operation of the processor 110. Also, in order to quantify breast components, the modules that may be included in the computing device 100 are not limited to the plurality of modules described above, and additional modules may be included. Hereinafter, a plurality of exemplary modules for quantifying breast components will be described in more detail.

[0076] According to an embodiment of the present disclosure, the computing device 100 can acquire an ultrasonic image (S10). The ultrasonic image can be an ultrasonic image acquired using an ultrasonic probe, which is a device that generates and transmits ultrasonic waves with an ultrasonic inspection device and additionally receives the reflected echoes. Also, the ultrasonic image may include a medical image acquired with the freeze mode of the probe activated. As an example, the computing device 100 can scan the breast using an ultrasonic device (e.g., an ultrasonic probe, etc.) to acquire an ultrasonic image.

[0077] According to an embodiment of the present disclosure, the breast region segmentation module 111 can identify (infer) the breast region from the ultrasonic image (S11). First, prior to identifying (inferring) the breast region, the breast region segmentation module 111 can i) perform preprocessing on the acquired ultrasonic image. The breast region segmentation module 111 can perform preprocessing to adjust the input image size for ultrasonic images having various sizes and pixel values. Also, the breast region segmentation module 111 can perform preprocessing to normalize the pixel range for the ultrasonic image. However, the preprocessing operation is not limited to this, and various preprocessing operations for applying to a deep learning model can be performed.

[0078] Next, the breast region segmentation module 111 can generate a breast inference region mask in the ultrasonic image by utilizing ii) a deep learning model. Here, the ultrasonic image can be the ultrasonic image preprocessed through the various operations described above. As an example, the deep learning model can include, but is not limited to, U-NET, MASK R-CNN, DeepLap, PSPNet, FCN (Fully Convolutional Network), SegNet, ENet, etc., and can include deep learning models that have already been developed or will be developed in the future. For reference, the deep learning model can be a model trained to infer the breast region. In other words, the breast region segmentation module 111 can generate a breast inference region mask in the preprocessed ultrasonic image by utilizing a deep learning model trained to infer the breast region. For example, the breast inference region mask can include a breast region probability map having the same size as the input image size. The range of the probability map can include values between 0 and 1.

[0079] Also, the breast region segmentation module 111 can generate a breast region binary map based on the generated breast inference region mask. More specifically, the breast region segmentation module 111 can classify the breast region probability map based on a predetermined threshold value (for example, 0.5) to generate the breast region binary map. For example, the breast region segmentation module 111 can assign 0 when the breast region probability map is below the predetermined threshold value (for example, corresponding to 0 to 0.5), and assign 1 when it exceeds the threshold value (for example, corresponding to 0.6 to 1) to generate a breast region binary map.

[0080] Next, the breast region segmentation module 111 can generate a breast region segmentation mask 11 based on iii) the breast region binary map. For example, the breast region segmentation module 111 can perform image processing operations on the breast region binary map to generate a breast region segmentation mask. As an example, the breast region segmentation mask can be obtained based on the largest region among a plurality of regions included in the breast region binary map. The breast region segmentation module 111 can generate the breast region segmentation mask 11 by leaving only the largest region in the breast region binary map.

[0081] According to an embodiment of the present disclosure, the fibro-glandular region segmentation module 112 can identify (infer) the fibro-glandular tissue region in the ultrasonic image (S12). First, prior to identifying (inferring) the fibro-glandular tissue region, the fibro-glandular region segmentation module 112 can i) perform preprocessing on the acquired ultrasonic image. The fibro-glandular region segmentation module 112 can perform preprocessing to adjust the input image size for ultrasonic images having various sizes and pixel values. In addition, the fibro-glandular region segmentation module 112 can perform preprocessing to normalize the pixel range for the ultrasonic image. However, the preprocessing operation is not limited to this, and various preprocessing operations for applying to the deep learning model can be performed.

[0082] Next, the fibro-glandular region segmentation module 112 can generate a fibro-glandular inference region mask in the ultrasonic image by utilizing ii) a deep learning model. Here, the ultrasonic image can be the ultrasonic image preprocessed through the various operations described above. As an example, the deep learning model can include, but is not limited to, U-NET, MASK R-CNN, DeepLap, PSPNet, FCN (Fully Convolutional Network), SegNet, ENet, etc., and can include deep learning models that have already been developed or will be developed in the future. For reference, the deep learning model can be a model trained to infer the fibro-glandular region. In other words, the fibro-glandular region segmentation module 112 can generate a fibro-glandular inference region mask in the preprocessed ultrasonic image by utilizing a deep learning model trained to infer the region. For example, the fibro-glandular inference region mask can include a fibro-glandular region probability map having the same size as the input image size. The range of the probability map can include values between 0 and 1.

[0083] Also, the fibro-glandular region segmentation module 112 can generate a fibro-glandular region binary map based on the generated fibro-glandular inference region mask. More specifically, the fibro-glandular region segmentation module 112 can classify the fibro-glandular region probability map based on a predetermined threshold (e.g., 0.5) to generate the fibro-glandular region binary map. For example, the fibro-glandular region segmentation module 112 can assign 0 when the value is below the predetermined threshold (e.g., corresponding to 0 to 0.5) and assign 1 when the value exceeds the threshold (e.g., corresponding to 0.6 to 1) based on the predetermined threshold of the fibro-glandular region probability map to generate the fibro-glandular region binary map.

[0084] Next, the fibro-glandular region segmentation module 112 can generate a fibro-glandular region segmentation mask based on iii) the fibro-glandular region binary map. For example, the fibro-glandular region segmentation module 112 can perform an operation on the breast region segmentation mask 11 and the fibro-glandular region binary map to generate an intra-breast fibro-glandular region mask. For example, the fibro-glandular region segmentation module 112 can perform a Hadamard Product operation on the breast region segmentation mask 11 and the fibro-glandular region binary map to generate an intra-breast fibro-glandular region mask. Also, the fibro-glandular region segmentation module 112 can generate the fibro-glandular region segmentation mask 21 based on the intra-breast fibro-glandular region mask. As an example, the fibro-glandular region segmentation mask can be obtained based on the largest region among a plurality of regions included in the intra-breast fibro-glandular region mask. The fibro-glandular region segmentation module 112 can generate the fibro-glandular region segmentation mask 21 by leaving only the largest region in the intra-breast fibro-glandular region mask.

[0085] According to an embodiment of the present disclosure, the glandular tissue region segmentation module 113 can identify a glandular tissue region based on the identified fibro-glandular tissue region. First, i) the glandular tissue region segmentation module 113 can generate a glandular tissue region binary map based on the fibro-glandular region segmentation mask 21. For example, the glandular tissue region segmentation module 113 can receive the fibro-glandular region segmentation mask 21 as an input, perform video processing, and then generate a glandular tissue region binary map. The glandular tissue region segmentation module 113 can generate a glandular tissue region binary map based on the fibro-glandular region segmentation mask 21 after performing video processing through a plurality of operations described below.

[0086] Exemplarily, the glandular tissue region segmentation module 113 can extract a region of interest (ROI) mask for the glandular tissue using the fibrous gland region segmentation mask 21 to generate a glandular tissue region binary map. For example, the glandular tissue region segmentation module 113 can apply an erosion filter to the fibrous gland region segmentation mask 21 generated by the fibrous gland region segmentation module 112 to extract the glandular tissue ROI mask. For reference, an erosion filter can process an image using a small kernel (filter) called a structuring element, which mainly has a square or circular shape and can adjust the pixels of the image while overlapping the kernel with the image. Also, the glandular tissue region segmentation module 113 can (2) detect a polygonal-shaped region of interest that includes the glandular tissue ROI. Exemplarily, the glandular tissue region segmentation module 113 can detect a box-shaped region of interest with a minimum size that completely includes the glandular tissue ROI. Also, the glandular tissue region segmentation module 113 can (3) perform histogram-based normalization on the ultrasonic image within the polygonal-shaped region of interest. For example, the glandular tissue region segmentation module 113 can perform histogram-based normalization on the ultrasonic image within a polygonal-shaped (e.g., square) region of interest. The glandular tissue region segmentation module 113 can perform histogram-based normalization on the ultrasonic image within a polygonal-shaped (e.g., square) region of interest and leave the original image unchanged outside the polygonal-shaped (e.g., square) region of interest to generate a region-of-interest-normalized ultrasonic image. Also, the glandular tissue region segmentation module 113 can (4) perform an operation on the region-of-interest-normalized ultrasonic image and the glandular tissue ROI mask to generate a glandular tissue ROI image. For example, the glandular tissue region segmentation module 113 can perform a Hadamard Product operation on the region-of-interest-normalized ultrasonic image and the glandular tissue ROI mask to generate a glandular tissue ROI image.In addition, the glandular tissue region segmentation module 113 can generate the glandular tissue region binary map by classifying based on a predetermined threshold value for the (5) glandular tissue-related video. As an example, values greater than or equal to the predetermined threshold value in the glandular tissue region of interest video can be mapped to 1, and values less than the predetermined threshold value can be mapped to 0 to generate the glandular tissue region binary map.

[0087] Next, the glandular tissue region segmentation module 113 can generate the glandular tissue region segmentation mask 31 based on ii) the glandular tissue region binary map. For example, the glandular tissue region segmentation module 113 can utilize the glandular tissue region of interest to generate the glandular tissue region segmentation mask 31 through video processing operations. The glandular tissue region segmentation module 113 can apply a median filter to the glandular tissue region binary map to remove noise and generate the glandular tissue region segmentation mask 31.

[0088] FIG. 5a is a drawing for explaining an operation of calculating the ratio and grade of breast tissue components according to an embodiment of the present disclosure, and FIG. 5b is a drawing for explaining an operation of calculating the ratio and grade of glandular tissue components according to an embodiment of the present disclosure.

[0089] According to one embodiment of the present disclosure, the computing device 100 can quantify at least one of a Breast Tissue Component or a Glandular Tissue Component based on the identified breast region and the identified fibroglandular tissue region. In breast ultrasound images, the breast tissue component (BTC) can be classified into three categories (A, B, C). First, homogenous-fat means the case where breast tissue is mostly composed of fat. Homogenous-fat breasts correspond to a relatively low grade among the breast cancer incidence risk grades. Also, homogenous-fibroglandular means the case where fat, fibrous tissue, and mammary gland tissue are uniformly distributed in the breast tissue. Homogenous-fibroglandular breasts correspond to an intermediate grade among the breast cancer incidence risk grades. Also, heterogeneous means the case where the density and shadow of the breast tissue are not uniform. Heterogeneous breasts correspond to a relatively high risk grade among the breast cancer incidence risk grades.

[0090] Another analysis index for breast tissue is the glandular tissue component (GTC), which can be classified into grades P1, P2, P3, P4, etc. The evaluation of GTC means the ratio of the gray area showing glandular tissue to the white area showing fibrous stroma in the fibroglandular tissue (FGT) surrounded by subcutaneous and retroareolar fat in breast ultrasound. At this time, the fat lobules of FGT, which are distinguished from the tissue, are not included in the GTC evaluation. The GTC (Glandular Tissue Composition) grade is qualitatively classified as minimal (less than 25% of FGT), mild (25% - 49% of FGT), moderate (50% - 74% of FGT), or marked (75% or more of FGT) for each woman based on bilateral breast ultrasound images, and can be recorded in the reading report with the abbreviations P1, P2, P3, P4. GTC can also be reclassified as "low" for minimal or mild GTC and "high" for moderate or marked GTC based on the GTC that constitutes 50% of breast FGT.

[0091] According to an embodiment of the present disclosure, the computing device 100 can quantify a Breast Tissue Component based on the identified breast region and the identified fibroglandular tissue region (S14). Exemplarily, referring to FIG. 5a, the computing device 100 can quantify the breast tissue component based on the breast region 10 identified through the breast region segmentation module 111 and the fibroglandular tissue region 20 identified through the fibroglandular region segmentation module 112. Exemplarily, the computing device 100 can utilize the pixel values of the fibroglandular region segmentation mask 21 and the pixel values of the breast region segmentation mask 11 to calculate the ratio of the breast tissue component. As an example, the computing device 100 can calculate the ratio of the breast tissue component based on [the sum of the number of pixels with a value of 1 in the fibroglandular region segmentation mask 21 / the sum of the number of pixels with a value of 1 in the breast region segmentation mask 11]. Further, the computing device 100 can map the ratio of the breast tissue component according to the breast tissue component (BTC) grade. For example, the computing device 100 can map the calculated ratio of the breast tissue component to any one of the A, B, and C grades for the predetermined breast tissue component (BTC). As an example, as shown in FIG. 5a, the computing device 100 can use the first color (e.g., blue) to represent the breast region 10 identified in the form of a line or a dotted line, and use the second color (e.g., yellow) to represent the fibroglandular tissue region 10 identified in the form of a line or a dotted line, and provide it through the user interface.

[0092] According to an embodiment of the present disclosure, the computing device 100 can analyze the identified fibrous glandular tissue region and the identified glandular tissue region to quantify the glandular tissue component (S15). Exemplarily, referring to FIG. 5b, the computing device 100 can quantify the glandular tissue component based on the fibrous glandular tissue region 20 identified through the fibrous gland region segmentation module 112 and the glandular tissue region 30 identified through the glandular tissue region segmentation module 113. Exemplarily, the computing device 100 can calculate the ratio of the glandular tissue component based on [the sum of the number of pixels with a pixel value of 1 in the glandular tissue region segmentation mask 31 / the sum of the number of pixels with a pixel value of 1 in the fibrous gland region segmentation mask 21]. Further, the computing device 100 can map the ratio of the glandular tissue component according to the glandular tissue component (GTC) grade. For example, the computing device 100 can map the calculated ratio of the glandular tissue component to any one of the P1, P2, P3, P4 grades for the predetermined glandular tissue component (GTC). As an example, as shown in FIG. 5b, the computing device 100 can use the second color (e.g., yellow) to represent the fibrous glandular tissue region 10 identified in the form of a line or a dotted line, and use the third color (e.g., red) to represent the glandular tissue region 30 identified in the form of a line or a dotted line, and provide it through the user interface.

[0093] According to an embodiment of the present disclosure, the computing device 100 can select a representative video from ultrasonic videos (S16). As an example, the computing device 100 can extract meaningful videos from breast ultrasonic videos containing noise. The computing device 100 can determine whether the breast ultrasonic video is suitable for evaluating breast components. For example, the computing device 100 can perform an evaluation on the breast ultrasonic video with respect to whether it is suitable for the calculated degrees of breast tissue components (BTC) and glandular tissue components (GTC). Further, the computing device 100 can select K representative videos for evaluating breast components based on the determination result of whether the breast components are suitable for evaluation. For example, the computing device 100 can determine the sharpness of the video, the noise level of the video, the degree of shaking of the video, the range of the video (for example, check whether a video with a sufficient range including the entire breast is included), the angle of the video, the quality of the video, the distribution of breast tissue, the density of breast tissue, and shadows, etc. for the acquired breast ultrasonic video, and select the K representative videos. However, the judgment conditions for selecting representative videos are not limited to this. Also, the computing device 100 can average the selected K representative videos and calculate the ratios and grades for the breast tissue components (BTC) and glandular tissue components (GTC) per Exam unit. For reference, when scanning a patient using a medical imaging device (e.g., ultrasound, X-ray, MRI, etc.), the images generated by the device can be grouped into one "Exam", and this can be used as the Exam unit.

[0094] Hereinafter, based on the above-described content, the operation flow of the present application will be briefly described.

[0095] FIG. 6 is a flowchart showing a method for quantifying breast components according to an embodiment of the present disclosure.

[0096] The method for quantifying the components of the breast shown in FIG. 6 can be executed by the computing device 100 described above. Therefore, even for the content omitted below, the content described for the computing device 100 can be equally applied to the description of the method for quantifying the components of the breast.

[0097] Referring to FIG. 6, the method for quantifying the components of the breast may include a step of acquiring an ultrasonic image (S110), a step of identifying a breast region in the ultrasonic image (S120), a step of identifying a fibro-glandular tissue region in the ultrasonic image (S130), and a step of quantifying at least one of a breast tissue component or a glandular tissue component based on the identified breast region and the identified fibro-glandular tissue region (S140).

[0098] Step (S110) is a step of acquiring an ultrasonic image. As an example, an ultrasonic device (e.g., an ultrasonic probe, etc.) can be used to scan the breast and acquire an ultrasonic image.

[0099] Step (S120) is a step of identifying a breast region in the ultrasonic image. The step (S120) may include a step of generating a breast inference region mask in the ultrasonic image by utilizing a deep learning model, a step of generating a breast region binary map based on the generated breast inference region mask, and a step of generating a breast region segmentation mask based on the breast region binary map. Here, the breast region segmentation mask can be obtained based on the largest region among the plurality of regions included in the breast region binary map.

[0100] Step (S130) is a step of identifying a fibro-glandular tissue region in the ultrasonic image. The step (S130) may include a step of generating a fibro-glandular inference region mask in the ultrasonic image by utilizing a deep learning model, a step of generating a fibro-glandular region binary map based on the generated fibro-glandular inference region mask, and a step of generating a fibro-glandular region segmentation mask based on the fibro-glandular region binary map.

[0101] Step (S140) is a step of quantifying at least one of a breast tissue component or a glandular tissue component based on the identified breast region and the identified fibro-glandular tissue region. Step (S140) may include a step of identifying a glandular tissue region based on the identified fibro-glandular tissue region, and a step of analyzing the identified fibro-glandular tissue region and the identified glandular tissue region to quantify the glandular tissue component. Also, step (S140) may include a step of calculating a ratio of the breast tissue component by utilizing pixel values of the fibro-glandular region segmentation mask and pixel values of the breast region segmentation mask, and a step of mapping the ratio of the breast tissue component according to a breast tissue component (BTC) grade. Also, step (S140) may include a step of calculating a ratio of the glandular tissue component by utilizing pixel values of the glandular tissue region segmentation mask and pixel values of the fibro-glandular region segmentation mask, and a step of mapping the ratio of the glandular tissue component according to a glandular tissue component (GTC) grade.

[0102] The steps mentioned in the above description may be further divided into additional steps or combined into fewer steps according to embodiments of the present disclosure. Also, some steps may be omitted as necessary, and the order between steps may be changed.

[0103] Based on an embodiment of the present disclosure, a computer-readable storage medium storing a data structure is disclosed. A data structure can be meant to organize, manage, and store data that enables efficient access to and modification of the data. A data structure can be meant to be a data organization for solving a specific problem (e.g., data search in the shortest time, data storage, data modification). A data structure can also be defined as the physical or logical relationship between data elements designed to support a specific data processing function. The logical relationship between data elements can include the concatenation relationship between data elements considered by the user. The physical relationship between data elements can include the actual relationship between data elements physically stored in a computer-readable storage medium (e.g., hard disk). A data structure can specifically include a set of data, the relationship between data, and functions or commands applicable to the data. With an effectively designed data structure, a computing device can perform calculations while minimizing the use of the resources of the computing device. Specifically, the computing device can enhance the efficiency of operations, reading, insertion, deletion, comparison, exchange, and search through an effectively designed data structure.

[0104] Data structures can be classified into linear data structures and non-linear data structures according to their forms. A linear data structure may be a structure in which only one data is connected after another data. Linear data structures can include lists, stacks, queues, and deques. A list can mean a series of data sets with an internal order. A list can include a linked list. A linked list can be a data structure in which data is connected in a way that each data has a pointer and is connected in a column. In a linked list, the pointer can include connection information with the next or previous data. A linked list can be represented as a singly linked list, a doubly linked list, or a circular linked list according to its form. A stack may be a data list structure with restricted access to data. A stack can be a linear data structure that can process (e.g., insert or delete) data only at one end of the data structure. The data stored in a stack can be a data structure (LIFO - Last In First Out) where the later the data enters, the earlier it comes out. A queue is a data arrangement structure with restricted access to data and, unlike a stack, can be a data structure (FIFO - First In First Out) where the later the data is stored, the later it comes out. A deque can be a data structure that can process data at both ends of the data structure.

[0105] A non-linear data structure may be a structure in which multiple data are connected after one data. Non-linear data structures can include graph data structures. A graph data structure can be defined by vertices and edges, and an edge can include a line connecting two different vertices. A graph data structure can include a tree data structure. A tree data structure can be a data structure formed by a path connecting two different vertices among the multiple vertices included in the tree. That is, it can be a data structure that does not form a loop in the graph data structure.

[0106] Throughout this specification, the terms computational model, neural circuit network, network function, and neural network can be used interchangeably. (Hereinafter, they will be uniformly described using the term neural network.) A data structure can include a neural network. A data structure that includes a neural network can be stored in a computer-readable storage medium. A data structure that includes a neural network can also include data input to the neural network, weight values of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, and a loss function for training the neural network. A data structure that includes a neural network can include any of the components of the disclosed configuration. That is, a data structure that includes a neural network can be configured to include all or any combination of data input to the neural network, weight values of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, a loss function for training the neural network, etc. In addition to the above-described configuration, a data structure that includes a neural network can include any other information that determines the characteristics of the neural network. Also, the data structure can include all forms of data used or generated in the computational process of the neural network, and is not limited to the foregoing matters. A computer-readable storage medium can include a computer-readable recording medium and / or a computer-readable transmission medium. A neural network can generally be composed of a set of interconnected computational units, which can be called nodes. Such nodes can be called neurons. A neural network is composed of at least one or more nodes.

[0107] A data structure can include data input to a neural network. A data structure including data input to a neural network can be stored in a computer-readable storage medium. The data input to the neural network can include learning data input during the learning process of the neural network and / or input data input to the neural network after learning is completed. The data input to the neural network can include pre-processed data and / or data to be pre-processed. The pre-processing can include a data processing process for inputting the data to the neural network. Therefore, the data structure can include data to be pre-processed and data generated by the pre-processing. The above-described data structure is merely an example, and the present disclosure is not limited thereto.

[0108] A data structure can include the weights of a neural network. (In this specification, weights and parameters can be used interchangeably.) And a data structure including the weights of a neural network can be stored in a computer-readable storage medium. A neural network can include a plurality of weights. The weights are variable and can be varied by a user or an algorithm in order for the neural network to perform a desired function. For example, when one or more input nodes are interconnected by respective links to one output node, the output node can determine an output node value based on the values input to the input nodes connected to the output node and the parameters set for the respective links corresponding to the input nodes. The above-described data structure is merely an example, and the present disclosure is not limited thereto.

[0109] By way of example and not limitation, the weighted values can include weighted values that vary during the neural network learning process and / or weighted values after the neural network learning is completed. The weighted values that are varied during the neural network learning process can include the weighted values at the start of the learning cycle and / or the weighted values that are varied during the learning cycle. The weighted values after the neural network learning is completed can include the weighted values after the learning cycle is completed. Accordingly, a data structure including the weighted values of the neural network can include a data structure including the weighted values that are varied during the neural network learning process and / or the weighted values after the neural network learning is completed. Accordingly, the above-described weighted values and / or combinations of each weighted value shall be included in a data structure including the weighted values of the neural network. The foregoing data structure is merely illustrative and the present disclosure is not limited thereto.

[0110] A data structure including the weighted values of the neural network can be stored in a computer-readable storage medium (e.g., memory, hard disk) after going through a serialization process. Serialization can be a process of converting a data structure into a form that can be stored in the same or another computing device and later reconstructed and used. A computing device can serialize a data structure and transmit and receive data via a network. A data structure including the weighted values of the serialized neural network can be reconstructed on the same computing device or another computing device through deserialization. A data structure including the weighted values of the neural network is not limited to serialization. Further, a data structure including the weighted values of the neural network can include a data structure (e.g., non-linear data structures such as B-Tree, Trie, m-way search tree, AVL tree, Red-Black Tree) for enhancing the efficiency of operations while minimizing the use of the resources of the computing device. The foregoing matters are merely illustrative and the present disclosure is not limited thereto.

[0111] The data structure can include hyper-parameters of the neural network. And the data structure including the hyper-parameters of the neural network can be stored in a computer-readable storage medium. The hyper-parameters can be variables that can be varied by the user. The hyper-parameters can include, for example, a learning rate, a cost function, the number of learning cycle iterations, weight initialization (e.g., setting the range of weights to be initialized), the number of Hidden Units (e.g., the number of Hidden layers, the number of nodes in the Hidden layer). The foregoing data structure is merely illustrative and the present disclosure is not limited thereto.

[0112] FIG. 7 is a simplified and general schematic diagram relating to an exemplary computing environment in which embodiments of the present disclosure can be implemented. Although it has been previously stated that the present disclosure can generally be implemented by a computing device, those skilled in the art will well understand that the present disclosure can be implemented in combination with computer-executable instructions and / or other program modules that can be executed on one or more computers and / or as a combination of hardware and software.

[0113] Generally, a module in this specification includes routines, programs, components, data structures, and the like that perform specific tasks or implement specific abstract data types. Also, those skilled in the art will well understand that the method of the present disclosure can be implemented by other computer system configurations, including single-processor or multi-processor computer systems, minicomputers, mainframe computers, as well as personal computers, handheld computing devices, microprocessor-based, or programmable household appliances, and the like (each of which can operate in connection with one or more related devices).

[0114] The embodiments described in this disclosure may further be implemented in a distributed computing environment where a task is performed by a remote processing device connected through a communication network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0115] Computers include a variety of computer-readable media. Any media accessible by a computer can be a computer-readable media, and such computer-readable media includes volatile and non-volatile media, transitory and non-transitory media, removable and non-removable media. By way of example and not limitation, computer-readable media can include computer-readable storage media and computer-readable transmission media. Computer-readable storage media includes volatile and non-volatile media, transitory and non-transitory media, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, DVD (digital video disk) or other optical disk storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or any other media that can be accessed by a computer and used to store information.

[0116] A computer-readable transmission medium typically implements computer-readable instructions, data structures, program modules, or other data, etc. in a modulated data signal, such as a carrier wave or other transport mechanism, and includes all information transmission media. The term "modulated data signal" means a signal that has set or changed one or more of the characteristics of the signal so as to encode information in the signal. By way of example and not limitation, computer-readable transmission media include wired media such as a wired network or a direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. Any combination of the foregoing media is also intended to be included within the scope of computer-readable transmission media.

[0117] An exemplary environment (1100) for implementing various aspects of the present disclosure including a computer (1102) is shown, and the computer (1102) includes a processing device (1104), a system memory (1106), and a system bus (1108). The system bus (1108) connects system components, including but not limited to the system memory (1106), to the processing device (1104). The processing device (1104) can be any of a variety of commercial processors. Dual processors and other multiprocessor architectures can also be utilized as the processing device (1104).

[0118] The system bus (1108) can be any of a plurality of types of bus structures that can be further interconnected to a local bus that uses any of a memory bus, a peripheral device bus, and various commercial bus architectures. The system memory (1106) includes a read-only memory (ROM) (1110) and a random access memory (RAM) (1112). The basic input / output system (BIOS) is stored in non-volatile memory (1110) such as ROM, EPROM, EEPROM, etc., and this BIOS includes basic routines that support the exchange of information between a plurality of components in the computer (1102) during startup and the like. The RAM (1112) can also include high-speed RAM such as static RAM for caching data.

[0119] In the computer (1102), there are also included a built-in hard disk drive (HDD) (1114) (for example, EIDE, SATA) - this built-in hard disk drive (1114) can also be configured for external use in an appropriate chassis (not shown), a magnetic floppy disk drive (FDD) (1116) (for example, for reading from and writing to a removable diskette (1118)), and an optical disk drive (1120) (for example, for reading from a CD-ROM disk (1122) or other high-capacity optical media such as a DVD and writing to the same). The hard disk drive (1114), the magnetic disk drive (1116), and the optical disk drive (1120) can be connected to the system bus (1108) by a hard disk drive interface (1124), a magnetic disk drive interface (1126), and an optical drive interface (1128), respectively. The interface (1124) for implementing an external drive includes, for example, at least one or both of USB (Universal Serial Bus) and IEEE1394 interface technologies.

[0120] These drives and the computer-readable media associated therewith provide non-volatile storage of data, data structures, computer-executable instructions, and the like. In the case of computer (1102), the drives and media correspond to storing any data in a suitable digital format. Although the foregoing description of computer-readable storage media refers to HDDs, removable magnetic disks, and removable optical media such as CDs or DVDs, those skilled in the art will recognize that other types of storage media readable by a computer, such as zip drives, magnetic cassettes, flash memory cards, cartridges, and the like, can also be used in exemplary operating environments, and furthermore, it will be well understood that any of such media can contain computer-executable instructions for performing the methods of the present disclosure.

[0121] A number of program modules, including an operating system (1130), one or more application programs (1132), other program modules (1134), and program data (1136), can be stored on the drives and in RAM (1112). All or a portion of the operating system, applications, modules, and / or data can also be cached in RAM (1112). It will be well understood that the present disclosure can be implemented by various commercially available operating systems or combinations of multiple operating systems.

[0122] The user can input commands and information into the computer (1102) through one or more wired and wireless input devices, such as pointing devices like a keyboard (1138) and a mouse (1140). Other input devices (not shown) may include a microphone, an IR remote control, a joystick, a game pad, a stylus pen, a touch screen, and so on. These and other input devices may be connected to the processing device (1104) through an input device interface (1142) that is often connected to the system bus (1108), but can also be connected by other interfaces such as a parallel port, an IEEE 1394 serial port, a game port, a USB port, an IR interface, and so on.

[0123] A monitor (1144) or other types of display devices are also connected to the system bus (1108) through an interface such as a video adapter (1146). In addition to the monitor (1144), the computer generally includes other peripheral output devices such as speakers, printers, and so on (not shown).

[0124] The computer (1102) can operate in a networked environment using logical connections to one or more remote computers, such as (multiple) remote computers (1148) via wired and / or wireless communication. The (multiple) remote computers (1148) can be workstations, server computers, routers, personal computers, portable computers, microprocessor-based entertainment devices, peer devices, or other common network nodes, and generally include many or all of the components described for the computer (1102), but for simplicity, only the memory storage device (1150) is shown. The illustrated logical connections include wired and wireless connections in a local area network (LAN) (1152) and / or a larger network, such as a wide area network (WAN) (1154). Such LAN and WAN networking environments are common in offices and companies, facilitating enterprise-wide computer networks such as intranets, all of which can be connected to worldwide computer networks, such as the Internet.

[0125] When used in a LAN networking environment, the computer (1102) is connected to the local network (1152) through a wired and / or wireless communication network interface, or adapter (1156). The adapter (1156) can facilitate wired or wireless communication to the LAN (1152), which also includes a wireless access point installed therein for communicating with the wireless adapter (1156). When used in a WAN networking environment, the computer (1102) can include a modem (1158), be connected to a communication server on the WAN (1154), or have other means of establishing communication through the WAN (1154), such as through the Internet. The modem (1158), which can be an internal or external, wired or wireless device, is connected to the system bus (1108) through a serial port interface (1142). In a networked environment, program modules or portions thereof described for the computer (1102) can be stored in a remote memory / storage device (1150). It is readily understood that the network connections shown are exemplary and that other means of establishing communication links between multiple computers can be used.

[0126] The computer (1102) operates to communicate with any wireless device or unit arranged and operating in wireless communication, such as a printer, scanner, desktop and / or portable computer, PDA (portable data assistant), communication satellite, any equipment or location related to a wirelessly detectable tag, and a telephone. This includes at least Wi-Fi and Bluetooth (registered trademark) wireless technologies. Thus, the communication can be in a predefined structure like a conventional network or simply an ad hoc communication between at least two devices.

[0127] Wi-Fi (Wireless Fidelity) enables connection to the Internet and the like without being wired. Wi-Fi is a wireless technology like a cellular phone that allows such devices, for example, computers to send and receive data indoors and outdoors, that is, from anywhere within the coverage area of a base station. Wi-Fi networks use wireless technologies such as IEEE 802.11 (a, b, g, etc.) to provide a secure, reliable, and high-speed wireless connection. Wi-Fi can be used to connect computers to each other and to the Internet and wired networks (using IEEE 802.3 or Ethernet). Wi-Fi networks can operate at data rates such as 11 Mbps (802.11a) or 54 Mbps (802.11b) in the unlicensed 2.4 and 5 GHz wireless bands, or can operate in products that include both bands (dual-band).

[0128] Those of ordinary skill in the art of the present disclosure can appreciate that information and signals can be represented using any of a variety of different technologies and techniques. For example, the data, instructions, commands, information, signals, bits, symbols, and chips that can be referenced in the foregoing description can be represented by voltages, currents, electromagnetic waves, magnetic fields, etc., or particles, optical fields, etc., or particles, or any combination thereof.

[0129] Those of ordinary skill in the art of the present disclosure will appreciate that the various exemplary logical blocks, modules, processors, means, circuits, algorithm steps recited in the description of the embodiments disclosed herein can be implemented by electronic hardware, various forms of programs or design codes (referred to herein as "software" for convenience), or any combination of these. To clearly illustrate such interchangeability between hardware and software, the various exemplary components, blocks, modules, circuits, and steps have been generally described above in terms of their functions. Whether such functions are implemented in hardware or software depends on the design constraints imposed on a particular application and the overall system. Those of ordinary skill in the art of the present disclosure can implement the functions described in various ways for individual specific applications, but such implementation decisions should not be construed as departing from the scope of the present disclosure.

[0130] The various embodiments shown herein can be implemented by a method, apparatus, or article of manufacture using standard programming and / or engineering techniques. The term "article of manufacture" includes any computer program, carrier, or medium accessible from any computer-readable device. For example, computer-readable storage media includes, but is not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical disks (e.g., CDs, DVDs, etc.), smart cards, and flash memory devices (e.g., EEPROMs, cards, sticks, key drives, etc.). Also, the various storage media shown herein include one or more devices for storing information and / or other machine-readable media.

[0131] It should be understood that the specific order or hierarchical structure of the multiple steps in the presented process is an example of an exemplary approach. Based on design priorities, it should be understood that within the scope of the present disclosure, the specific order or hierarchical structure of the steps in the process can be rearranged. The appended method claims provide elements of various steps in sample order, but are not meant to be limited to the specific order or hierarchical structure shown.

[0132] The description of the presented embodiments is provided so that a person of ordinary skill in any art of the present disclosure can make use of or practice the present disclosure. Various modifications to such embodiments will be readily apparent to persons of ordinary skill in the art of the present disclosure, and the general principles defined herein can be applied to other embodiments without departing from the scope of the present disclosure. Accordingly, the present disclosure is not limited by the embodiments shown herein, but should be construed in the broadest scope consistent with the principles and novel features shown herein.

Claims

1. 1. A method for quantifying breast constituents, performed by a computing device, comprising: acquiring an ultrasound image; identifying a breast region in the ultrasound image; identifying a fibro-glandular tissue region in the ultrasound image; and quantitating at least one of a Breast Tissue Component or a Glandular Tissue Component based on the identified breast area and the identified fibroglandular tissue area; A method comprising:

2. The step of identifying a breast region in the ultrasound image includes: generating a breast inference region mask from the ultrasound image using a deep learning model; generating a breast region binary map based on the generated breast inference region mask; and generating a breast segmentation mask based on the breast region binary map; The method of claim 1 , comprising:

3. The method of claim 2 , wherein the breast segmentation mask is obtained based on a largest region of a plurality of regions included in the breast region binary map.

4. the breast inference region mask comprises a breast region probability map; generating a breast region binary map based on the generated breast inference region mask, classifying the breast region probability map based on a predetermined threshold value to generate the breast region binary map; The method of claim 2 , comprising:

5. The step of identifying a fibroglandular tissue region in the ultrasound image includes: generating a fibrous gland inference region mask in the ultrasound image using a deep learning model; generating a fibrous gland region binary map based on the generated fibrous gland inferred region mask; and generating a fibrous gland region segmentation mask based on the fibrous gland region binary map; The method of claim 2 , comprising:

6. the fibrous gland inference region mask comprises a fibrous gland region probability map; The step of generating a fibrous gland region binary map based on the generated fibrous gland inference region mask includes: classifying the fibrous gland region probability map based on a predetermined threshold value to generate the fibrous gland region binary map; The method of claim 5 , comprising:

7. The step of generating a fibrous gland region segmentation mask based on the fibrous gland region binary map includes: performing an operation on the breast segmentation mask and the fibro-glandular region binary map to generate an intra-breast fibro-glandular region mask; and generating the fibroglandular region segmentation mask based on the intra-breast fibroglandular region mask; The method of claim 5 , comprising:

8. The method of claim 7 , wherein the fibroglandular region segmentation mask is obtained based on the largest region among a plurality of regions included in the intra-breast fibroglandular region mask.

9. Quantifying at least one of the breast tissue components or the glandular tissue components based on the identified breast area and the identified fibroglandular tissue area includes: identifying a glandular tissue region based on the identified fibroglandular tissue region; and analyzing the identified fibroglandular tissue regions and the identified glandular tissue regions to quantify the glandular tissue components; The method of claim 1 , comprising:

10. identifying a glandular tissue region based on the identified fibroglandular tissue region, generating a glandular tissue region binary map based on the fibro-glandular region segmentation mask; and generating a glandular tissue region segmentation mask based on the glandular tissue region binary map; 10. The method of claim 9, comprising:

11. generating a binary map of glandular tissue regions based on the fibroglandular tissue region segmentation mask, extracting a glandular tissue region of interest mask from the fibroglandular tissue segmentation mask; detecting a polygonal region of interest including the glandular tissue region of interest; performing histogram-based normalization on the ultrasound image within the polygonal shaped region of interest; generating a region-of-interest normalized ultrasound image by using the ultrasound image inside the polygonal region of interest on which the histogram-based normalization has been performed and the ultrasound image outside the polygonal region of interest; performing an operation on the region-of-interest normalized ultrasound image and the glandular tissue region-of-interest mask to generate a glandular tissue region-of-interest image; and classifying the glandular tissue image of interest based on a predetermined threshold value to generate a binary map of the glandular tissue region; The method of claim 10, comprising:

12. Quantifying at least one of the breast tissue components or the glandular tissue components based on the identified breast area and the identified fibroglandular tissue area includes: calculating a percentage of the breast tissue component using pixel values ​​of a fibroglandular region segmentation mask and pixel values ​​of a breast region segmentation mask; and mapping the percentage of breast tissue components according to breast tissue composition (BTC) grades; The method of claim 1 , comprising:

13. Quantifying at least one of the breast tissue components or the glandular tissue components based on the identified breast area and the identified fibroglandular tissue area includes: calculating a ratio of the glandular tissue component using pixel values ​​of a glandular tissue region segmentation mask and pixel values ​​of a fibro-glandular region segmentation mask; and mapping the proportion of glandular tissue components according to glandular tissue composition (GTC) grades; The method of claim 1 , comprising:

14. A computer program stored on a computer readable recording medium, the computer program, when executed by one or more processors, causing the one or more processors to perform the following operations for quantifying breast components, the operations comprising: acquiring an ultrasound image; identifying a breast region in the ultrasound image; identifying fibro-glandular tissue regions in the ultrasound image; and quantification of at least one of a Breast Tissue Component or a Glandular Tissue Component based on the identified breast area and the identified fibroglandular tissue area; A computer program stored on a computer-readable recording medium, comprising:

15. The operation of identifying a breast region in the ultrasound image includes: generating a breast inference region mask from the ultrasound image utilizing a deep learning model; generating a breast region binary map based on the generated breast inference region mask; and generating a breast segmentation mask based on said breast region binary map; 15. A computer program stored on a computer-readable recording medium according to claim 14, comprising:

16. the breast inference region mask comprises a breast region probability map; The operation of generating a breast region binary map based on the generated breast inference region mask includes: The computer program product of claim 15 , further comprising: classifying the breast region probability map based on a predetermined threshold value to generate the breast region binary map.

17. The operation of identifying a fibroglandular tissue region in the ultrasound image includes: generating a fibroid inference region mask in the ultrasound image using a deep learning model; generating a fibrous gland region binary map based on the generated fibrous gland inferred region mask; and generating a fibrous gland region segmentation mask based on the fibrous gland region binary map; 16. A computer program stored on a computer-readable recording medium according to claim 15, comprising:

18. the fibrous gland inference region mask comprises a fibrous gland region probability map; The operation of generating a fibrous gland region binary map based on the generated fibrous gland inference region mask includes:

20. The computer program product stored on a computer-readable recording medium of claim 17, further comprising an operation of classifying the fibrous gland region probability map based on a predetermined threshold value to generate the fibrous gland region binary map.

19. Quantifying at least one of the breast tissue components or the glandular tissue components based on the identified breast region and the identified fibroglandular tissue region includes: identifying a glandular tissue region based on the identified fibroglandular tissue region; and analyzing the identified fibroglandular tissue regions and the identified glandular tissue regions to quantify the glandular tissue content; 15. A computer program stored on a computer-readable recording medium according to claim 14, comprising:

20. Quantifying at least one of the breast tissue components or the glandular tissue components based on the identified breast region and the identified fibroglandular tissue region includes: calculating a percentage of the breast tissue component using pixel values ​​of a fibroglandular segmentation mask and pixel values ​​of a breast segmentation mask; and mapping said percentage of breast tissue composition according to a breast tissue composition (BTC) grade; 15. A computer program stored on a computer-readable recording medium according to claim 14, comprising:

21. Quantifying at least one of the breast tissue components or the glandular tissue components based on the identified breast region and the identified fibroglandular tissue region includes: calculating a ratio of the glandular tissue component using pixel values ​​of a glandular tissue region segmentation mask and pixel values ​​of a fibro-glandular region segmentation mask; and mapping the proportion of glandular tissue components according to a glandular tissue composition (GTC) grade; 15. A computer program stored on a computer-readable recording medium according to claim 14, comprising:

22. 1. A computing device comprising: at least one processor; and memory; Including, The at least one processor: Obtaining an ultrasound image; identifying a breast region in the ultrasound image; Identifying fibro-glandular tissue regions in the ultrasound image; and The apparatus is configured to quantify at least one of a Breast Tissue Component or a Glandular Tissue Component based on the identified breast area and the identified fibroglandular tissue area.

23. The at least one processor: Utilizing a deep learning model to generate a breast inference region mask in the ultrasound image; generating a breast region binary map based on the generated breast inference region mask; and The apparatus of claim 22 , configured to generate a breast segmentation mask based on the breast region binary map.

24. the breast inference region mask comprises a breast region probability map; The at least one processor:

24. The apparatus of claim 23, configured to classify the breast region probability map with respect to a predetermined threshold value to generate the breast region binary map.

25. The at least one processor: Utilizing a deep learning model to generate a fibrous gland inference region mask in the ultrasound image; generating a fibrous gland region binary map based on the generated fibrous gland inferred region mask; and The apparatus of claim 23 , configured to generate a fibrous gland region segmentation mask based on the fibrous gland region binary map.

26. the fibrous gland inference region mask comprises a fibrous gland region probability map; The at least one processor:

26. The apparatus of claim 25, configured to classify the fibrous gland region probability map based on a predetermined threshold value to generate the fibrous gland region binary map.

27. The at least one processor: identifying a glandular tissue region based on the identified fibroglandular tissue region; and 23. The apparatus of claim 22, configured to analyze the identified fibroglandular tissue regions and the identified glandular tissue regions to quantify the glandular tissue component.

28. The at least one processor: Utilizing pixel values ​​of the fibroglandular segmentation mask and pixel values ​​of the breast segmentation mask to calculate a percentage of the breast tissue component; and 23. The apparatus of claim 22, configured to map the percentage of breast tissue content by a breast tissue content (BTC) grade.

29. The at least one processor: Calculating the proportion of the glandular tissue component using pixel values ​​of the glandular tissue region segmentation mask and pixel values ​​of the fibroglandular tissue region segmentation mask; and 23. The apparatus of claim 22, configured to map the proportion of glandular tissue constituents by a glandular tissue constituent (GTC) grade.

Images