Ultrasound diagnostic device, storage medium, and learning model generation method

Abstract

To examine a subject while considering a balance between the time required for diagnosis and medical costs.SOLUTION: An ultrasound diagnostic apparatus 1 executes first to n-th diagnostic assistance algorithms. The first to n-th diagnostic assistance algorithms include: a first diagnostic assistance algorithm that analyzes a first ultrasound image acquired between a scan start time point t0 and a first time point t1, and outputs an analysis result, where candidates for the analysis result include a diagnostic result and information for recommending continuation of examination; and a second diagnostic assistance algorithm that, when the first diagnostic assistance algorithm outputs information for recommending continuation of the ultrasound examination to a user, analyzes a second ultrasound image acquired at a second time point t2 after the first time point t1, and outputs an analysis result, where candidates for the analysis result include a diagnostic result and information for recommending continuation of examination.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an ultrasound diagnostic device having a diagnostic support function for supporting the diagnosis of a subject, a storage medium used in the ultrasound diagnostic device, and a learning model generation method for generating a learning model that can be used to support the diagnosis. [Background technology]

[0002] In the field of medical image diagnosis, much research has been done on image classification using machine learning, and some of these techniques have been put to practical use. While there are various methods for machine learning, we will briefly explain "supervised learning" as an example. In supervised learning, medical images (training data) linked to diagnostic results (answers) are prepared, and this training data is input into machine learning software for training. This type of training can generate a training model. In clinical settings, such training models can be used by radiologists who interpret images using dedicated workstations. Specifically, the radiologist selects an image from multiple medical images of a patient displayed on the workstation and inputs the selected image into the training model, which then outputs diagnostic information inferred by machine learning. The diagnostic information output by the training model can be used as reference information to assist in diagnosis.

[0003] Ultrasound diagnostic devices are also known as devices for non-invasively examining the inside of a subject's body. An operator of an ultrasound diagnostic device presses an ultrasound probe against the body surface of a subject (patient) and manipulates the probe to display cross-sectional images of the living body in real time.

[0004] Contrast agents are now also used in ultrasound examinations, just like CT and MRI. In contrast-enhanced examinations, contrast agents are injected into the subject's veins and circulate throughout the body, including the organ of interest. Ultrasound contrast examinations provide additional diagnostic information that cannot be determined from cross-sectional images alone, based on the speed at which the contrast agent flows into the region of interest, the pattern of the shape of the contrast agent, and the pattern showing changes in the contrast agent over time, and are expected to improve the accuracy of differential diagnosis of benign and malignant tumors. Contrast-enhanced ultrasound examinations are used, for example, for qualitative diagnosis of breast lesions and diagnosis of the extent of lesions.

[0005] In Japan, Sonazoid (registered trademark) has been put into practical use as an ultrasound diagnostic contrast agent. Sonazoid has the unique characteristic of accumulating in areas where normal liver cells are present. On the other hand, contrast agents in abnormal areas tend to be washed away earlier than contrast agents in normal liver cells. Therefore, the characteristic of accumulating in areas where normal liver cells are present is clearly visible in contrast images obtained a certain time (e.g., 10 minutes) after administration of the contrast agent. In other words, information obtained by ultrasound contrast examinations contains information useful for diagnosis, such as the inflow pattern immediately after administration of the contrast agent and the accumulation pattern after a certain time (e.g., 10 minutes) has elapsed. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-112200 Summary of the Invention [Problem to be solved by the invention]

[0007] Ultrasound diagnostic equipment is operated by technicians, but doctors may also operate it. In this case, ultrasound tomographic images and diagnosis can be performed simultaneously, allowing doctors to expand the area of ​​observation at their discretion and collect detailed image data over a long period of time, enabling detailed observation and diagnosis, which offers significant benefits to patients. However, one issue with doctors performing ultrasound imaging is the medical costs. Therefore, when doctors perform ultrasound examinations of patients, they must perform medical procedures and manage their medical care while considering the balance between the time spent on diagnosis and medical costs.

[0008] When considering the balance between patient benefits and medical costs, one issue that must be addressed is the approximately 10 minutes required for a diagnosis using contrast-enhanced ultrasound. The 10 minutes refers to the observation time after contrast administration. In addition to the observation time, time is also required for preliminary confirmation of the tomographic image, securing an intravenous route, and preparing the contrast agent. Therefore, a contrast-enhanced diagnosis takes approximately three to five times longer than a simple tomographic image diagnosis. Reducing this time is an important issue.

[0009] Therefore, there is a demand for a technology that can examine a subject while taking into consideration the balance between the time required for diagnosis and medical costs. [Means for solving the problem]

[0010] A first aspect of the present invention is an ultrasound diagnostic apparatus that executes a plurality of diagnostic assistance algorithms to output information necessary to assist in the diagnosis of a subject based on an ultrasound image obtained from the subject to which a contrast agent has been administered, the ultrasound diagnostic apparatus comprising: The plurality of diagnostic assistance algorithms include: a first diagnostic assistance algorithm that analyzes a first ultrasound image acquired from the start of scanning until a first time point has elapsed, and outputs an analysis result, wherein candidates for the analysis result of the first diagnostic assistance algorithm include a diagnosis result for the subject and information for recommending to a user that the ultrasound examination continue; a second diagnostic assistance algorithm that, when the first diagnostic assistance algorithm outputs information recommending a user to continue the ultrasound examination, analyzes a second ultrasound image acquired at a second time point after the first time point and outputs an analysis result, wherein candidates for the analysis result of the second diagnostic assistance algorithm include a diagnosis result of the subject and information recommending a user to continue the ultrasound examination; The ultrasound diagnostic device includes:

[0011] A second aspect of the present invention is a computer-executable non-transitory storage medium storing a plurality of diagnostic assistance algorithms for outputting information necessary to assist in the diagnosis of a subject based on an ultrasound image obtained from the subject to which a contrast agent has been administered, the medium comprising: The instructions stored on the storage medium, when executed by one or more processors, cause the one or more processors to: a first diagnostic assistance algorithm that analyzes a first ultrasound image acquired from the start of scanning until a first time point has elapsed, and outputs an analysis result, wherein candidates for the analysis result of the first diagnostic assistance algorithm include a diagnosis result for the subject and information for recommending to a user that the ultrasound examination continue; a second diagnostic assistance algorithm that, when the first diagnostic assistance algorithm outputs information recommending a user to continue the ultrasound examination, analyzes a second ultrasound image acquired at a second time point after the first time point and outputs an analysis result, wherein candidates for the analysis result of the second diagnostic assistance algorithm include a diagnosis result of the subject and information recommending a user to continue the ultrasound examination; and It is a storage medium that executes the above.

[0012] A third aspect of the present invention is to create a first learning model by having software learn the first learning data including a first learning image; Creating a second learning model by having software learn second learning data including second learning images, the second learning images including learning images acquired at a later time than the first learning images. A method for generating a learning model, comprising: [Effects of the Invention]

[0013] In the present invention, the first diagnostic assistance algorithm analyzes the first ultrasound images acquired between the start of the scan and the first time point, and outputs an analysis result. The candidate analysis results of this first diagnostic assistance algorithm include a diagnosis result for the subject.

[0014] Therefore, if a diagnosis of the subject is possible based on the first ultrasound images acquired between the start of the scan and the first time point, the first diagnostic assistance algorithm can output a diagnostic result. Therefore, if a diagnosis is possible without ultrasound images from the first time point onwards, the first diagnostic assistance algorithm outputs a diagnostic result of the subject, allowing the user to avoid the task of acquiring ultrasound images from the second time point onwards that are not necessary for the diagnostic result of the subject.

[0015] On the other hand, the candidate analysis results of the first diagnostic assistance algorithm also include information for recommending to the user to continue the ultrasound examination. Therefore, if the first ultrasound images acquired between the start of the scan and the first time point are not enough to diagnose the subject or to accurately diagnose the subject, the first diagnostic assistance algorithm can output information for recommending to the user to continue the ultrasound examination.

[0016] Therefore, the first diagnostic assistance algorithm can determine whether i) a diagnostic result can be output based on the first ultrasound image acquired between the start of the scan and the first time point, or ii) it is difficult to output a diagnostic result based on the first ultrasound image, and therefore it is desirable to recommend to the user that the ultrasound examination continue. Therefore, continuation of the ultrasound examination is recommended only when ultrasound images from the first time point onward are necessary for diagnosis, thereby minimizing medical costs incurred by users (such as doctors) engaging in ultrasound examinations without reducing diagnostic accuracy.

[0017] Furthermore, when information recommending the user to continue the ultrasound examination is output, the second diagnostic assistance algorithm analyzes a second ultrasound image acquired at a second time point after the first time point, and outputs a diagnosis result for the subject. Similar to the first diagnostic assistance algorithm, the output candidates of this second diagnostic assistance algorithm include the diagnosis result for the subject.

[0018] Therefore, if a diagnosis of the subject is possible based on a second ultrasound image acquired at a second time point after the first time point, the second diagnostic assistance algorithm can output a diagnostic result. Therefore, if a diagnosis of the subject is possible without an ultrasound image at a time point after the second time point, the second diagnostic assistance algorithm outputs a diagnostic result of the subject, so that the user can avoid the task of acquiring ultrasound images at a time point after the third time point that are not necessary for the diagnostic result of the subject.

[0019] On the other hand, the output candidates of the second diagnostic assistance algorithm also include information for recommending to the user to continue the ultrasound examination. Therefore, when the second ultrasound image acquired at the second time point alone is not sufficient to diagnose the subject or it is difficult to accurately diagnose the subject, the second diagnostic assistance algorithm can output information for recommending to the user to continue the ultrasound examination.

[0020] Therefore, the second diagnostic assistance algorithm can determine whether i) a diagnostic result can be output based on the second ultrasound image acquired at the second time point, or ii) it is difficult to output a diagnostic result based on the second ultrasound image, and therefore it is desirable to recommend to the user that the ultrasound examination continue. Therefore, continuation of the ultrasound examination is recommended only when an ultrasound image from a time point later than the second time point is necessary for diagnosis, thereby minimizing medical costs incurred by the user (such as a doctor) engaging in ultrasound examinations without reducing diagnostic accuracy. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a block diagram of an ultrasonic diagnostic apparatus 1 according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram showing an example of the flow of a contrast agent examination according to the present embodiment. [Figure 3] FIG. 10 is an explanatory diagram of a method for generating a learning model used in the first diagnostic assistance algorithm (step ST13). [Figure 4] FIG. 10 is an explanatory diagram of a method for generating a learning model used in the second diagnostic assistance algorithm (step ST23). [Figure 5] FIG. 1 is an explanatory diagram of a method for generating a learning model used in the i-th diagnostic assistance algorithm. [Figure 6] 10 is an explanatory diagram of a method for generating a learning model used in the n-th diagnostic assistance algorithm.FIG. 11 is a diagram showing an example of the flow of step ST3.FIG. [Figure 7] FIG. 10 is a diagram showing an example of a flow when performing observation from administration of a contrast agent to time t3. DETAILED DESCRIPTION OF THE INVENTION

[0022] Hereinafter, a description will be given of an embodiment of the invention, but the present invention is not limited to the following embodiment.

[0023] FIG. 1 is a block diagram of an ultrasonic diagnostic apparatus 1 according to this embodiment of the present invention. The ultrasound diagnostic device 1 includes an ultrasound probe 2, a transmit beamformer 3, a transmitter 4, a receiver 5, a receive beamformer 6, a processor 7, a display unit 8, a memory 9, and a user interface 10.

[0024] The ultrasound probe 2 has a plurality of transducer elements 2a arranged in an array. A transmit beamformer 3 and a transmitter 4 drive the plurality of transducer elements 2a arranged in the ultrasound probe 2, and ultrasound waves are transmitted from the transducer elements 2a. The ultrasound waves transmitted from the transducer elements 2a are reflected inside the subject, and the reflected echoes are received by the transducer elements 2a. The transducer elements 2a convert the received echoes into electrical signals and output these electrical signals as echo signals to the receiver 5. The receiver 5 performs predetermined processing on the echo signals and outputs them to the receive beamformer 6. The receive beamformer 6 performs receive beamforming on the signals received from the receiver 5 and outputs echo data.

[0025] The receive beamformer 6 may be a hardware beamformer or a software beamformer. If the receive beamformer 6 is a software beamformer, the receive beamformer 6 may include one or more processors, including one or more of: i) a graphics processing unit (GPU), ii) a microprocessor, iii) a central processing unit (CPU), iv) a digital signal processor (DSP), or v) another type of processor capable of performing logical operations. The processor(s) constituting the receive beamformer 6 may be separate from or comprised within the processor 7.

[0026] The ultrasound probe 2 may include electrical circuitry for performing all or part of the transmit beamforming and / or receive beamforming. For example, all or part of the transmit beamformer 3, transmitter 4, receiver 5, and receive beamformer 6 may be provided within the ultrasound probe 2.

[0027] The processor 7 controls the transmit beamformer 3, the transmitter 4, the receiver 5, and the receive beamformer 6. The processor 7 is also in electronic communication with the ultrasound probe 2. The processor 7 controls which transducer elements 2a are active and further controls the shape of the ultrasound beam transmitted from the ultrasound probe 2. The processor 7 is also in electronic communication with the display 8 and the user interface 10. The processor 7 can process the echo data to generate an ultrasound image. The term "electronic communication" can be defined to include both wired and wireless communication. According to one embodiment, the processor 7 can include a central processing unit (CPU). According to other embodiments, the processor 7 can include other electronic components capable of performing processing functions, such as a digital signal processor, a field programmable gate array (FPGA), a graphics processing unit (GPU), or other types of processors. According to other embodiments, the processor 7 can include multiple electronic components capable of performing processing functions. For example, the processor 7 can include two or more electronic components selected from the list of electronic components including a central processing unit, a digital signal processor, a field programmable gate array, and a graphics processing unit. Processor 7 may also include a complex demodulator (not shown) for demodulating the RF data.

[0028] The processor 7 can also generate various ultrasound images (e.g., B-mode images, color Doppler images, M-mode images, color M-mode images, spectral Doppler images, elastography images, TVI images, strain images, strain velocity images, etc.) based on data obtained by processing by the receive beamformer 6. One or more modules can also generate these ultrasound images.

[0029] The image beams and / or image frames may be stored and timing information indicating when the data was acquired in memory. The modules may include, for example, a scan conversion module that performs a scan conversion operation to convert the image frames from coordinate beam space to display space coordinates. A video processor module may be provided that reads the image frames from memory and displays the image frames in real time while a procedure is being performed on the subject. The video processor module may store the image frames in image memory, and the ultrasound images may be read from the image memory and displayed on display 8.

[0030] As used herein, the term "image" may broadly refer to both a visible image and data representing a visible image, and the term "data" may include raw data, which is ultrasound data before a scan conversion operation, and image data, which is data after a scan conversion operation.

[0031] The above-mentioned processing tasks handled by the processor 7 may be executed by a plurality of processors.

[0032] Furthermore, when the receive beamformer 6 is a software beamformer, the processing performed by the beamformer may be executed by a single processor or by multiple processors.

[0033] The display unit 8 is, for example, an LED (Light Emitting Diode) display unit, an LCD (Liquid Crystal Display), or an organic EL (Electro-Luminescence) display unit. The display unit 8 displays an ultrasound image.

[0034] The memory 9 is any known data storage medium. In one example, the ultrasound diagnostic apparatus includes a non-transitory storage medium and a transient storage medium as memory. The ultrasound diagnostic apparatus may also include multiple memories. The non-transitory storage medium is a non-volatile storage medium such as a hard disk drive (HDD) or a read-only memory (ROM). The non-transitory storage medium may include a portable storage medium such as a compact disk (CD) or a digital versatile disk (DVD). The program executed by the processor 7 is stored in the non-transitory storage medium. The transient storage medium is a volatile storage medium such as a random access memory (RAM).

[0035] The memory 9 stores one or more instructions executable by the processor 7. The one or more instructions cause the processor 7 to perform operations described in the embodiments described below.

[0036] The processor 7 can also be configured to be connected to an external storage device via a wired or wireless connection. In this case, the instructions to be executed by the processor 7 can be stored in both the memory 9 and the external storage device.

[0037] The user interface 10 can accept user input. For example, the user interface 10 accepts input of instructions and information from the user. The user interface 10 includes a keyboard, hard keys, a trackball, a rotary control, soft keys, etc. The user interface 10 may also include a touch screen that displays soft keys, etc. The ultrasonic diagnostic apparatus 1 is configured as described above.

[0038] Although the ultrasound diagnostic device 1 is operated by a medical technician, doctors may also operate the ultrasound diagnostic device. In this case, ultrasound tomographic images can be drawn and diagnosed simultaneously, allowing doctors to expand the area of ​​observation at their discretion and collect detailed image data over a long period of time, enabling detailed observation and diagnosis, which is of great benefit to patients. However, one issue when doctors perform ultrasound imaging is the medical costs. Therefore, when doctors perform ultrasound examinations of subjects, they must perform medical procedures while considering the balance between the time required for diagnosis and medical costs.

[0039] Therefore, the inventors of the present invention have conducted extensive research and have devised a technology that enables contrast imaging of a subject while taking into consideration the balance between the time required for diagnosis and medical costs. This technology will be described below.

[0040] FIG. 2 is a diagram showing an example of the flow of a contrast examination according to this embodiment. In this embodiment, the processor is configured to execute a plurality of diagnostic support algorithms that output information necessary to support the diagnosis of the subject. A contrast examination is performed using these diagnostic support algorithms. The flow of the contrast examination is described below.

[0041] A contrast agent is administered to the subject, and block P1 is executed. Block P1 includes steps ST11 to ST13. Note that the left side of FIG. 2 shows a time axis t, which indicates how much time has passed since the start of the scan. In FIG. 2, the flow of the contrast examination will be explained while checking each time point on the time axis t as necessary.

[0042] In step ST11, the user continuously observes the imaging region of the subject and acquires ultrasound images from time t0 when the scan of the subject starts to a first time t1. The acquired ultrasound images may include B-mode images and contrast-enhanced images. In FIG. 2, the time from time t0 to time t1 is indicated by X1. Note that the start time t0 of the scan may be, for example, a time before the administration of a contrast agent starts, a time when the administration of a contrast agent starts, a time during the administration of a contrast agent, a time when the administration of a contrast agent ends, or a time after the administration of a contrast agent ends.

[0043] In step ST12, the user inputs a command to record images by operating the user interface 10. When this command is input, the processor records in the memory 9 the ultrasound images acquired between time t0 and time t1 (time X1).

[0044] In step ST13, the processor uses the first diagnostic assistance algorithm to analyze the ultrasound image recorded in step ST12 and output the analysis results. The candidate analysis results output by the first diagnostic assistance algorithm include the subject's diagnosis result and information for recommending to the user to continue the ultrasound examination.

[0045] If the analysis result in step ST13 indicates a diagnosis (for example, "metastatic tumor," "cancer," etc.), the diagnosis is deemed complete and the flow ends.

[0046] On the other hand, if it is determined that a diagnosis cannot be made by the analysis in step ST13 and that further testing of the subject is necessary to make a diagnosis, the first diagnostic assistance algorithm outputs information indicating that continuation of the test is recommended. If continuation of the test is recommended, the process proceeds to the next block P2. Block P2 includes steps ST21 to ST23.

[0047] In step ST21, the user continues observing the imaging region even after X1 minutes have passed since the start of the scan. Specifically, the user observes the imaging region at a second time point t2 (the time point when X2 minutes have passed since the scan start time point t0) that is later than the first time point t1, and acquires an ultrasound image. In step ST21, the user may not only observe the imaging region at time point t2, but may also observe the imaging region continuously (or intermittently) from time point t1 to time point t2, as necessary, or may observe the imaging region for only a few seconds before and after time point t2.

[0048] In step ST22, the user inputs a command to record an image by operating the user interface 10. When this command is input, the processor records the ultrasound image acquired in step ST21.

[0049] In step ST23, the processor uses a second diagnostic assistance algorithm to analyze the ultrasound image recorded in step ST22 and output the analysis result. Note that the second diagnostic assistance algorithm may analyze not only the ultrasound image recorded in step ST22 but also the ultrasound image recorded in step ST12 and output the analysis result. The candidate analysis results output by the second diagnostic assistance algorithm include the diagnosis result of the subject and information for recommending to the user to continue the ultrasound examination.

[0050] If the analysis result in step ST23 indicates a diagnosis result, the diagnosis is deemed to be complete and the flow ends.

[0051] On the other hand, if the analysis in step ST23 does not allow a diagnosis and it is determined that further testing of the subject is necessary to make a diagnosis, the second diagnostic assistance algorithm outputs information indicating that continuation of the test is recommended. If continuation of the test is recommended, the process proceeds to the next block.

[0052] Similarly, blocks including steps of observing the imaging region, recording images, and diagnostic assistance algorithm steps are repeatedly executed.

[0053] For example, at the (i-1)th time point t i-1 If the (i-1)th diagnostic assistance algorithm (i is an integer equal to or greater than 3) that analyzes the ultrasound image acquired in recommends that the user continue the examination, the flow proceeds to block Pi. Block Pi may be executed after block P2, or may be executed after one or more other blocks are executed after block P2. For example, when i=3, that is, when the (i-1)th diagnostic assistance algorithm is the second diagnostic assistance algorithm in block P2, if the second diagnostic assistance algorithm outputs information recommending that the user continue the ultrasound examination, the flow proceeds to block Pi. On the other hand, if the second diagnostic assistance algorithm in block P2 outputs a diagnosis result, the flow ends without proceeding to block Pi.

[0054] Block Pi includes steps STi1 to STi3. In step STi1, the user performs a step at the (i-1)th time point t i-1 At the i-th time t after i (at the time when Xi minutes have elapsed since the scan start time t0) and an ultrasound image is acquired. i observations at time t i-1 From time t i The imaging area may be observed continuously (or intermittently) until time t i It is acceptable to observe for a few seconds before and after the

[0055] In step STi2, the user inputs a command to record an image by operating the user interface 10. When this command is input, the processor records the ultrasound image acquired in step STi1.

[0056] In step STi3, the processor uses the i-th diagnostic assistance algorithm to analyze the ultrasound image recorded in step STi2 and output the analysis result. Note that the i-th diagnostic assistance algorithm may analyze not only the ultrasound image recorded in step STi2, but also ultrasound images recorded before step STi2 (e.g., ultrasound images recorded in step ST12, ultrasound images recorded in step ST22, etc.), and output the analysis result. The candidate analysis results output by the i-th diagnostic assistance algorithm include the subject's diagnosis result and information for recommending to the user to continue the ultrasound examination.

[0057] If the analysis result in step STi3 indicates a diagnosis result, the diagnosis is deemed to be complete and the flow ends.

[0058] On the other hand, if a diagnosis cannot be made through the analysis in step STi3 and it is determined that further examination of the subject is necessary to make a diagnosis, the i-th diagnostic assistance algorithm outputs information indicating that continuation of the examination is recommended. If continuation of the examination is recommended, the process proceeds to the next block, and similarly thereafter, blocks including a step of observing the imaging region, a step of recording the image, and a step of the diagnostic assistance algorithm are executed.

[0059] For example, at the (n-1)th time point t n-1 If the (n-1)th diagnostic assistance algorithm (n is an integer equal to or greater than i+1) that analyzes the ultrasound image acquired in recommends that the user continue the examination, the process proceeds to block Pn. Block Pn may be executed after block Pi, or block Pn may be executed after one or more other blocks are executed after block Pi. For example, when n=i+1, that is, when the (n-1)th diagnostic assistance algorithm is the ith diagnostic assistance algorithm of block Pi, the process proceeds to block Pn when the ith diagnostic assistance algorithm outputs information recommending that the user continue the ultrasound examination.

[0060] Block Pn includes steps STn1 to STn3. In step STn1, the user n-1 The nth time t after n (Xn minutes have elapsed since the scan start time t0) and an ultrasound image is obtained. n observations at time t n-1 From time t n The imaging area may be observed continuously (or intermittently) until time t n It is acceptable to observe for a few seconds before and after the

[0061] In step STn2, the user inputs a command to record an image by operating the user interface 10. When this command is input, the processor records the ultrasound image acquired in step STn1.

[0062] In step STn3, the processor uses the nth diagnostic assistance algorithm to analyze the ultrasound image recorded in step STn2 and output the analysis results. The candidate analysis results output by the nth diagnostic assistance algorithm include the subject's diagnosis result but do not include information for recommending to the user that the ultrasound examination continue. Note that the nth diagnostic assistance algorithm may analyze not only the ultrasound image recorded in step STn2, but also ultrasound images recorded before step STn2 (e.g., ultrasound images recorded in steps ST12, ST22, STi2, etc.) and output the analysis results. In this way, the flow of the contrast examination is completed.

[0063] In the flow shown in Figure 2, image analysis is performed using a diagnostic assistance algorithm each time an image is recorded. If a diagnosis can be made based on the recorded ultrasound image, the diagnostic result is output and the diagnosis is completed. On the other hand, if a diagnosis cannot be made even after analyzing the recorded ultrasound image, information is output to the user recommending that the examination continue. Therefore, the user only needs to continue acquiring ultrasound images if a diagnosis cannot be made using the recorded ultrasound image, thereby minimizing the medical costs associated with users' involvement in ultrasound examinations without reducing diagnostic accuracy.

[0064] Next, a method for realizing the diagnostic assistance algorithm used in the flow of FIG. 2 will be described. Various technologies have been developed as methods for realizing diagnostic assistance algorithms, but here we will explain a method that uses a learning model generated by machine learning.

[0065] FIG. 3 is an explanatory diagram of a method for generating a learning model used in the first diagnostic assistance algorithm (step ST13).

[0066] To generate a learning model, it is necessary to prepare learning data. The learning data can be prepared, for example, by the following procedure.

[0067] First, first learning images A1 to Aa are prepared. The learning images A1 to Aa are a plurality of ultrasound images obtained by actually performing a contrast-enhanced examination in a medical facility such as a hospital. The learning images A1 to Aa are images obtained from the start time t0 of the scan in the contrast-enhanced examination to the time t 11 The images are acquired during the time period from time t to time t (for X1 minutes). The ultrasound images acquired from the medical facility may be pre-processed as necessary to be suitable for the training images A1 to Aa. Examples of pre-processing include image extraction, standardization, normalization, image inversion, image rotation, magnification rate change, and image quality change. 11 It is desirable that the time t coincides with the time t1 in the flow of the contrast examination shown in FIG. 11is not required to strictly coincide with the time t1, and can be set to the time t2 as long as the reliability of the analysis result of the first diagnostic support algorithm (step ST13) can be sufficiently ensured. 11 may have a certain time lag from time t1.

[0068] Next, the diagnostic results are linked to the training images A1 to Aa as correct answer data. For example, if the diagnostic result for image A1 is a metastatic tumor, the diagnostic result "metastatic tumor" is linked to image A1, and if the diagnostic result for image A2 is cancer, the diagnostic result "cancer" is linked to image A2. Here, for convenience of explanation, it is assumed that three diagnostic results (G11, G12, G13) exist as correct answer data, and one of the three diagnostic results is linked to each of the training images A1 to Aa. In this way, the learning data LD1 can be prepared.

[0069] Then, the learning data LD1 is input to machine learning software and trained, thereby generating a learning model LM1 to be used in the first diagnostic assistance algorithm.

[0070] When an image with an unknown diagnostic result is input, the learning model LM1 predicts the probability that the input image corresponds to the diagnostic result G11, the probability that the input image corresponds to the diagnostic result G12, and the probability that the input image corresponds to the diagnostic result G13, and outputs the diagnostic result with the highest probability as the analysis result. However, even if the diagnostic result with the highest probability has a low probability, there is a risk that the diagnostic result obtained is not highly reliable. For example, if the probability that the diagnostic result corresponds to the diagnostic result G11 is 30%, the probability that the diagnostic result corresponds to the diagnostic result G12 is 30%, and the probability that the diagnostic result corresponds to the diagnostic result G13 is 30%, all of the diagnostic results are considered to be low in reliability. In this case, from time t0 to time t 11It is considered difficult to output a highly reliable diagnosis using only the images acquired during the period. Therefore, the learning model LM1 sets a threshold value TH for the probability of a diagnosis result, and is trained to recommend to the user that the ultrasound examination continue if the probability of any diagnosis result does not exceed the threshold value TH. Therefore, the continuation of the ultrasound examination is recommended only when further ultrasound images are necessary for diagnosis, so that the medical costs incurred by users (doctors, etc.) involved in ultrasound examinations can be minimized without reducing diagnostic accuracy.

[0071] In the above example, three diagnostic results (G11, G12, G13) are used as correct answer data, but the number of diagnostic results used as correct answer data is not limited to three, and one or two diagnostic results may be used as correct answer data, or four or more diagnostic results may be used as correct answer data. For example, if only one diagnostic result G11 is used as correct answer data, when an image with an unknown diagnostic result is input, the learning model LM1 predicts the probability that the input image corresponds to the diagnostic result G11, and if the predicted probability does not exceed the threshold TH, it can output information recommending the user to continue the ultrasound examination.

[0072] FIG. 4 is an explanatory diagram of a method for generating a learning model used in the second diagnostic assistance algorithm (step ST23).

[0073] First, second learning images B1 to Bb are prepared. The learning images B1 to Bb are a plurality of ultrasound images obtained by actually performing a contrast-enhanced examination in a medical facility such as a hospital. In the contrast-enhanced examination, the images are taken from the scan start time t0 to the time t 12 Therefore, the learning images B1 to Bb are images acquired during the time period from time t0 to time t 11 The training images include training images acquired at a later time point than the training images A1 to Aa (see FIG. 3) acquired between time t. Note that the ultrasound images acquired from the medical facility may be pre-processed as necessary to become images suitable for the training images B1 to Bb. 12It is desirable that the time t coincides with the time t2 in the flow of the contrast examination shown in FIG. 12 is not required to strictly coincide with time t2, and as long as the reliability of the analysis results of the second diagnostic support algorithm (step ST23) can be sufficiently ensured, it is possible to 12 In this embodiment, the learning images B1 to Bb are captured from time t0 to time t 12 However, if the reliability of the analysis result of the second diagnostic assistance algorithm (step ST23) can be sufficiently ensured, 12 Alternatively, the training images B1 to Bb may include only ultrasound images at time points t 12 If the ultrasound image includes the scan start time t0 to time t 12 The ultrasound images may include ultrasound images acquired continuously (or intermittently) up to time t 12 The ultrasound images may include ultrasound images acquired within a few seconds before and after the event.

[0074] Next, the diagnostic results are linked to the training images B1 to Bb as correct answer data. For ease of explanation, it is assumed here that three diagnostic results (G21, G22, G23) exist as correct answer data, and that one of the three diagnostic results (G21, G22, G23) is linked to each of the training images B1 to Bb. In this way, the learning data LD2 can be prepared.

[0075] The learning data LD2 is then input to machine learning software for learning. In this way, the learning model LM2 used in the diagnostic assistance algorithm 12 can be generated.

[0076] When an image with an unknown diagnostic result is input, the learning model LM2 is trained to predict the probability that the input image corresponds to diagnostic result G21, diagnostic result G22, or diagnostic result G23, and output the diagnostic result with the highest probability as the analysis result. However, even if the diagnostic result with the highest probability has a low probability, it may not be a highly reliable diagnostic result. For example, if the probability of diagnostic result G21 is 30%, the probability of diagnostic result G22 is 30%, and the probability of diagnostic result G23 is 30%, all of the diagnostic results are considered to be unreliable. In this case, it is considered difficult to output a highly reliable diagnostic result using only images acquired between the start of scanning t0 and time t12. Therefore, the learning model LM2 is trained to set a threshold value TH for the probability of the diagnostic result, and to recommend to the user to continue the ultrasound examination if the probability of any diagnostic result does not exceed the threshold value TH. Therefore, continuation of the ultrasound examination is recommended only when further ultrasound images are required for diagnosis, thereby minimizing medical costs incurred by users (doctors, etc.) engaging in ultrasound examinations without reducing diagnostic accuracy.

[0077] In the above example, three diagnostic results (G21, G22, G23) are used as correct answer data, but the number of diagnostic results used as correct answer data is not limited to three, and one or two diagnostic results may be used as correct answer data, or four or more diagnostic results may be used as correct answer data. For example, if only one diagnostic result G21 is used as correct answer data, when an image with an unknown diagnostic result is input, the learning model LM2 predicts the probability that the input image corresponds to the diagnostic result G21, and if the predicted probability does not exceed the threshold TH, it can output information recommending that the user continue with the ultrasound examination.

[0078] Similarly, a learning model to be used in each diagnostic assistance algorithm can be generated. For example, when generating a learning model to be used in the i-th diagnostic assistance algorithm in step STi3, i-th learning images C1 to Cc are prepared as shown in Fig. 5. The learning images C1 to Cc are a plurality of ultrasound images obtained by actually performing a contrast-enhanced examination in a medical facility such as a hospital, and are images obtained from the scan start time t0 to the time t 1i Until (X i Therefore, the learning images C1 to Cc are images acquired within the time range t0 to t 12 (See Figure 4) 1i The ultrasound images acquired at the medical facility may be pre-processed as necessary to be suitable for the learning images C1 to Cc. 1i is the time t i However, at time t 1i is the time t i However, if the reliability of the analysis results of the i-th diagnostic support algorithm (step STi3) can be sufficiently ensured, the time t 1i is the time t i In this embodiment, the learning images C1 to Cc are captured from time t0 to time t 1i However, if the reliability of the analysis results of the i-th diagnostic support algorithm (step STi3) can be sufficiently ensured, 1i Alternatively, the training images C1 to Cc may include only ultrasound images at time points t 1i If the ultrasound image includes an ultrasound image at time t0, the scan start time t 1i The ultrasound images may include ultrasound images acquired continuously (or intermittently) up to time t 1iThe training images C1 to Cc may include ultrasound images acquired within a few seconds before and after the training image C1. After the training images C1 to Cc are prepared, the i-th training data LDi is prepared by linking the training images C1 to Cc with the correct answer data as described above. Then, the i-th training model LMi can be generated by executing the training process.

[0079] Like learning models LM1 and LM2, learning model LMi sets a threshold value TH for the probability of a diagnosis result, and if the probability of any diagnosis result does not exceed the threshold value TH, learning model LMi is trained to recommend to the user that the ultrasound examination continue. Therefore, continuation of the ultrasound examination is recommended only when further ultrasound images are necessary for diagnosis, so that the medical costs incurred by users (doctors, etc.) involved in ultrasound examinations can be minimized without reducing diagnostic accuracy.

[0080] In the above example, three diagnostic results (Gi1, Gi2, Gi3) are used as the correct answer data, but the number of diagnostic results used as the correct answer data is not limited to three, and one or two diagnostic results may be used as the correct answer data, or four or more diagnostic results may be used as the correct answer data. For example, when only one diagnostic result Gi1 is used as the correct answer data, the learning model LMi predicts the probability that the input image corresponds to the diagnostic result Gi1 when an image with an unknown diagnostic result is input, and if the predicted probability does not exceed the threshold TH, it can output information recommending the user to continue the ultrasound examination.

[0081] Furthermore, when generating a learning model to be used in the n-th diagnostic assistance algorithm in step STn3, n-th learning images D1 to Dd are prepared as shown in Fig. 6. The learning images D1 to Dd are a plurality of ultrasound images obtained by actually performing a contrast-enhanced examination in a medical facility such as a hospital, and are taken from the scan start time t0 to the time t 1n Until (X n Therefore, the training images D1 to Dd are images acquired within the time range t0 to t 1i(See Figure 5) 1n The ultrasound images acquired at the medical facility may be pre-processed as necessary to be suitable for the training images D1 to Dd. 1n is the time t n However, at time t 1n is the time t n However, if the reliability of the analysis results of the nth diagnostic support algorithm (step STn3) can be sufficiently ensured, the time t 1n is the time t n In this embodiment, the learning images D1 to Dd are captured from time t0 to time t 1n However, if the reliability of the analysis results of the nth diagnostic support algorithm (step STn3) can be sufficiently ensured, 1n Alternatively, the training images D1 to Dd may include only ultrasound images at time points t 1n If the ultrasound image includes an ultrasound image at time t0, the scan start time t 1n The ultrasound images may include ultrasound images acquired continuously (or intermittently) up to time t 1n The training images D1 to Dd may include ultrasound images acquired within a few seconds before and after the training image D1. After the training images D1 to Dd are prepared, the n-th training data LDn is prepared by linking the training images D1 to Dd with the ground truth data, and the training process is performed to generate the n-th training model LMn.

[0082] The learning model LMn analyzes the ultrasound image acquired in the final observation (step STn1) of the contrast examination. Therefore, it is not assumed that the ultrasound examination will be continued after the analysis by the learning model LMn. Therefore, the learning model LMn differs from the previously described learning models LM1, LM2, and LMi in that it has not been trained to recommend to the user that the ultrasound examination be continued.

[0083] In this embodiment, learning models LD1 to LDn are generated as described above. The learning models LD1 to LDn output a diagnosis result if a diagnosis is possible with the acquired ultrasound images, and recommend the user to continue the ultrasound examination if a diagnosis is not possible with the acquired ultrasound images. Therefore, continuing the ultrasound examination is recommended only when further ultrasound images are necessary for diagnosis, so that medical costs incurred by users (such as doctors) engaging in ultrasound examinations can be minimized without reducing diagnostic accuracy.

[0084] Next, as an example of the flow shown in FIG. 2, a flow in which n=3, that is, observation is performed from the administration of a contrast agent until time t3, will be described.

[0085] FIG. 7 is a diagram showing an example of a flow when performing observation from the administration of a contrast agent until time t3. After administering the contrast agent, in step ST11, the user continuously observes the imaging site of the subject and acquires ultrasound images from time t0, when the scan of the subject begins, to a first time t1 (during time X1). The acquired ultrasound images may include B-mode images and contrast-enhanced images. Time X1 may be, for example, 1 minute. In step ST12, the images acquired in step ST11 are recorded. Then, in step ST13, the processor inputs the recorded ultrasound images (or ultrasound images preprocessed as necessary) as input images to the learning model LM1 of the first diagnostic assistance algorithm. The learning model LM1 analyzes the input images and, if a diagnosis can be made based on the input images alone, outputs a diagnosis result. On the other hand, if a diagnosis cannot be made based on the input images alone, the learning model LM1 outputs information indicating that continuation of the examination is recommended. This information is displayed, for example, on the display unit 8 (see FIG. 1). If information recommending continuation of the test is displayed on the display unit 8, the process proceeds to step ST21, where the user continues the test.

[0086] In step ST21, the user observes the imaging region from the first time point t1 to the second time point t2 and acquires ultrasound images. The second time point t2 represents the time point when time X2 has elapsed since the scan start time point t0. The time X2 can be, for example, X2=2 minutes. In step ST21, the user may observe only time point t2, or may observe the imaging region continuously (or intermittently) from time point t1 to time point t2, or may observe only a few seconds before and after time point t2. In step ST22, the images acquired in step ST21 are recorded. Then, in step ST23, the processor inputs the recorded ultrasound image (or an ultrasound image preprocessed as necessary) as an input image to the learning model LM2 of the second diagnostic assistance algorithm. The learning model LM2 analyzes the input image and, if a diagnosis can be made based on the input image alone, outputs a diagnosis result. The learning model LM2 may analyze not only the ultrasound image recorded in step ST22 but also ultrasound images recorded before step ST22 (for example, ultrasound images recorded in step ST12) and output the analysis results. If a diagnosis cannot be made based on the input image alone, the learning model LM2 outputs information indicating that it is recommended to continue the examination. If this information is output, the process proceeds to step ST31, where the user continues the examination.

[0087] In step ST31, the user observes the imaging region from the second time point t2 to the third time point t3 and acquires ultrasound images. The third time point t3 represents the time point when time X3 has elapsed since the scan start time point t0. Time X3 can be, for example, X3=10 minutes. In step ST31, the user may observe only time point t3, or may observe the imaging region continuously (or intermittently) from time point t2 to time point t3, or may observe only for a few seconds before and after time point t3. In step ST32, the image acquired in step ST31 is recorded. Then, in step ST33, the processor inputs the recorded ultrasound image (or an ultrasound image preprocessed as necessary) as an input image to the learning model LM3 of the third diagnostic assistance algorithm. The learning model LM3 analyzes the input image and outputs a diagnosis result. Furthermore, the learning model LM3 may analyze not only the ultrasound images recorded in step ST32, but also ultrasound images recorded before step ST32 (e.g., ultrasound images recorded in step ST12, ultrasound images recorded in step ST22, etc.) and output a diagnostic result. In this way, the diagnosis is completed.

[0088] As described above, in this embodiment, if a diagnosis is possible with the acquired ultrasound images, a diagnostic result is output. On the other hand, if a diagnosis is not possible with the acquired ultrasound images, the user is recommended to continue the ultrasound examination. Therefore, continuation of the ultrasound examination is recommended only when further ultrasound images are necessary for diagnosis, eliminating the need to spend 10 minutes on each contrast examination. As a result, detailed diagnostic information can be obtained while ensuring the time efficiency of the contrast examination.

[0089] In this embodiment, the diagnostic assistance algorithm uses machine learning technology to output a diagnostic result (or information recommending continuation of the examination). However, in the present invention, the diagnostic assistance algorithm is not limited to one that uses AI technology such as machine learning, and a diagnostic result (or information recommending continuation of the examination) may be output using an algorithm that does not use AI technology. For example, a region of interest may be set on an ultrasound image, and the pixel values ​​of pixels within the region of interest may be compared with the pixel values ​​of pixels in a region surrounding the region of interest, and a diagnostic result (or information recommending continuation of the examination) may be output based on the comparison result. [Explanation of symbols]

[0090] 1. Ultrasound diagnostic equipment 2 Ultrasound probes 2a Vibration element 3 Transmit beamformer 4 Transmitters 5 Receiver 6 Receive beamformer 7 processors 8 Display 9. Memory 10 User Interface

Claims

1. 1. An ultrasound diagnostic device that executes a plurality of diagnostic assistance algorithms to output information necessary to assist in the diagnosis of a subject, based on an ultrasound image obtained from an imaging region of the subject to which a contrast agent has been administered, The plurality of diagnostic assistance algorithms include: a first diagnostic assistance algorithm that analyzes first ultrasound images of the imaging region acquired from the start of scanning until a first time point has elapsed, and outputs an analysis result, wherein candidates for the analysis result of the first diagnostic assistance algorithm include a diagnosis result for the subject and information for recommending to a user that the ultrasound examination of the imaging region be continued; a second diagnostic assistance algorithm that, when the first diagnostic assistance algorithm outputs information recommending to a user to continue the ultrasound examination of the imaging region, analyzes a second ultrasound image of the imaging region acquired at a second time point after the first time point and outputs an analysis result, wherein candidates for the analysis result of the second diagnostic assistance algorithm include a diagnosis result of the subject and information recommending to a user to continue the ultrasound examination of the imaging region; 1. An ultrasound diagnostic device comprising:

2. The plurality of diagnostic assistance algorithms include: an i-th diagnostic assistance algorithm that analyzes an i-th ultrasound image of the imaging region acquired at a time ti after the (i-1)-th time point and outputs an analysis result when the (i-1)-th (i is an integer of 3 or more) diagnostic assistance algorithm outputs information recommending to a user to continue the ultrasound examination of the imaging region, wherein candidates for the analysis result of the i-th diagnostic assistance algorithm include a diagnosis result of the subject and information recommending to a user to continue the ultrasound examination of the imaging region; an n-th diagnostic assistance algorithm that analyzes an n-th ultrasound image of the imaging region acquired at an n-th time point after the (n-1)-th time point and outputs an analysis result when the (n-1)-th (n is an integer equal to or greater than i+1)-th diagnostic assistance algorithm outputs information recommending to a user to continue the ultrasound examination of the imaging region, and the candidate analysis results of the n-th diagnostic assistance algorithm include a diagnosis result of the subject; The ultrasonic diagnostic apparatus according to claim 1 ,

3. the first diagnostic assistance algorithm includes a first learning model generated based on first learning data including a first learning image; the second diagnostic assistance algorithm includes a second learning model generated based on second learning data including a second learning image; The ultrasound diagnostic apparatus according to claim 2 , wherein the second training images include training images acquired at a later time point than the first training images.

4. the i-th diagnostic assistance algorithm includes an i-th learning model generated based on i-th learning data including an i-th learning image; The ultrasound diagnostic apparatus according to claim 3 , wherein the i-th training image includes a training image acquired at a later time point than the second training image.

5. the nth diagnostic assistance algorithm includes an nth learning model generated based on nth learning data including an nth learning image; The ultrasound diagnostic apparatus according to claim 4 , wherein the nth training image includes a training image acquired at a later time point than the ith training image.

6. one diagnostic result is included in the candidates of the analysis results output by the first learning model, the second learning model, or the i-th learning model; 6. The ultrasound diagnostic device of claim 5, wherein when an image is input, the first learning model, the second learning model, or the i-th learning model predicts the probability that the input image corresponds to one of the diagnostic results, and if the predicted probability does not exceed a threshold, outputs information recommending to a user to continue ultrasound examination of the imaging area.

7. a plurality of diagnostic results are included in the candidates of the analysis results output by the first learning model, the second learning model, or the i-th learning model; 6. The ultrasound diagnostic device of claim 5, wherein when an image is input, the first learning model, the second learning model, or the i-th learning model predicts the probability that the input image corresponds to each diagnostic result, and if the probability of any diagnostic result does not exceed a threshold, outputs information recommending to the user to continue ultrasound examination of the imaging area.

8. The plurality of diagnostic assistance algorithms include: a third diagnostic assistance algorithm that analyzes a third ultrasound image of the imaging region acquired at a third time point after the second time point and outputs an analysis result when the second diagnostic assistance algorithm outputs information recommending to a user to continue the ultrasound examination of the imaging region, wherein candidates of the analysis result of the third diagnostic assistance algorithm include a diagnosis result of the subject. The ultrasonic diagnostic apparatus according to claim 1 ,

9. The ultrasound diagnostic device of claim 1 , wherein the second diagnostic assistance algorithm analyzes the first ultrasound image in addition to the second ultrasound image.

10. The ultrasound diagnostic device according to claim 2 , wherein the i-th diagnostic assistance algorithm analyzes an ultrasound image acquired before the i-th ultrasound image in addition to the i-th ultrasound image.

11. The ultrasonic diagnostic apparatus according to claim 2 , wherein the nth diagnostic assistance algorithm analyzes an ultrasonic image acquired before the nth ultrasonic image in addition to the nth ultrasonic image.

12. The ultrasound diagnostic apparatus according to claim 1 , wherein the first ultrasound image includes a B-mode image and a contrast-enhanced image.

13. 2. The ultrasound diagnostic device of claim 1, wherein the first diagnostic assistance algorithm analyzes the first ultrasound image based on pixel values ​​of pixels within a region of interest of the first ultrasound image and pixel values ​​of pixels in a region surrounding the region of interest.

14. 2. The ultrasound diagnostic device of claim 1, wherein the second diagnostic assistance algorithm analyzes the second ultrasound image based on pixel values ​​of pixels within a region of interest of the second ultrasound image and pixel values ​​of pixels in a region surrounding the region of interest.

15. A computer-executable non-transitory storage medium storing a plurality of diagnostic assistance algorithms that output information necessary to assist in the diagnosis of a subject based on an ultrasound image obtained from an imaging region of the subject to which a contrast agent has been administered, The instructions stored on the storage medium, when executed by one or more processors, the one or more processors a first diagnostic assistance algorithm that analyzes first ultrasound images of the imaging region acquired from the start of scanning until a first time point has elapsed, and outputs an analysis result, wherein candidates for the analysis result of the first diagnostic assistance algorithm include a diagnosis result for the subject and information for recommending to a user that the ultrasound examination of the imaging region be continued; a second diagnostic assistance algorithm that analyzes a second ultrasound image of the imaging region acquired at a second time point after the first time point and outputs an analysis result when the first diagnostic assistance algorithm outputs information recommending to a user to continue the ultrasound examination of the imaging region, wherein candidates for the analysis result of the second diagnostic assistance algorithm include a diagnosis result of the subject and information recommending to a user to continue the ultrasound examination of the imaging region; A storage medium that executes the above.

16. By having software learn first learning data including a first learning image obtained by performing a contrast examination, creating a first learning model; and Creating a second learning model by having software learn second learning data including second learning images obtained by performing a contrast examination, the second learning images including learning images acquired at a later time than the first learning images. Including, A method for generating a learning model, in which the first learning model and the second learning model receive as input images ultrasound images acquired from an imaging site of a subject to which a contrast agent has been administered, and output analysis results of the input images.

Images