Ultrasound diagnostic system for generating vascular path images, program for generating vascular path images, and non-temporary storage medium for storing the program for generating vascular path images

Abstract

Generates images of blood vessel pathways. [Solution] An ultrasound diagnostic system comprising the ultrasound probe of the present disclosure includes: an ultrasound probe that transmits an ultrasound signal at each of a plurality of locations in a subject where a plurality of blood vessels exist and receives a plurality of corresponding echo signals; an echo signal processing unit that obtains a plurality of ultrasound tomography images corresponding to each of a plurality of locations in a subject based on the plurality of echo signals; an analysis unit that analyzes the plurality of ultrasound tomography images to detect blood vessels in each of the plurality of ultrasound tomography images and analyzes information regarding the shape of the detected blood vessels and the type of artery and vein; a blood vessel image generation unit that processes the plurality of ultrasound tomography images and generates a blood vessel course image showing the course of the detected blood vessels and the type of artery and vein; a blood vessel state analysis unit that identifies abnormal locations in the shape of blood vessels based on information regarding the shape of blood vessels; and a superimposed image generation unit that superimposes the abnormal locations in shape onto the blood vessel course image.

Description

Technical Field

[0001] The present disclosure relates to an ultrasonic diagnostic system that generates a blood vessel course image using ultrasonic data, a program for generating a blood vessel course image, and a non-temporary storage medium that stores a program for generating a blood vessel course image.

Background Art

[0002] In order to perform cannulation such as artificial dialysis, administration of anticancer drugs, and intravenous drip, puncture of blood vessels for inserting a metal injection needle or a plastic catheter into the blood vessel has been conventionally performed. In addition, an invasive treatment to a blood vessel called an arteriovenous fistula (AVF) for performing hemodialysis (HD) has been conventionally used. This treatment surgically directly anastomoses an artery and a vein in the arm or leg (forms a blood flow path called "autogenous shunt") so that arterial blood with high pressure flows into the vein, the vein is dilated and strengthened to withstand dialysis puncture, and a sufficient blood flow rate is ensured at the location where blood is extracted, thereby improving dialysis efficiency. The radial artery and the radial cutaneous vein in the forearm or the brachial artery and the median cubital cutaneous vein or the ulnar cutaneous vein in the upper arm are often selected as the artery / vein for performing autogenous shunt.

[0003] When forming an autogenous shunt, it may be necessary to obtain an image in which the blood vessels existing in the arm or leg where the surgery is performed are depicted and evaluate the state of the blood vessels included in the image. However, evaluation of blood vessels included in ultrasonic images requires skill and cannot be easily performed by an observer with little experience. Further, not limited to the formation of autogenous shunt, there may be cases where it is desired to confirm the state of blood vessels for reasons such as performing puncture.

[0004] Therefore, a technique is needed to visualize blood vessels in a simple and easy-to-understand manner, without necessarily requiring the observer's expertise. In particular, understanding the vascular anatomy and blood flow status of the upper and lower extremities is important for treatment planning, such as identifying puncture sites for dialysis and apheresis (plasma separation and exchange therapy). In addition to vascular anatomy and blood flow status, information on the type and / or depth of blood vessels may also be required.

[0005] To confirm the course of blood vessels and their blood flow status, ultrasound, X-rays, and near-infrared light are currently used to recognize the course of blood vessels from the body surface, and the course of blood vessels is manually drawn along with the puncture and shunt sites. Methods for representing the course of blood vessels include devices using near-infrared light (JP 2009-95516, JP 2010-218) and methods using PDI image information from ultrasound (JP Hei 9-243342), but in both cases, the shape and blood flow information in the depth direction are added or averaged, and it is only possible to represent the course of blood vessels as seen from the body surface. Photoacoustic imaging is similar, and it is difficult to visualize deep areas. Furthermore, it has not been possible to clearly display the difference between arteries and veins. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2009-95516 [Patent Document 2] Japanese Patent Publication No. 2010-218 [Patent Document 3] Japanese Patent Application Publication No. 9-243342 [Patent Document 4] Japanese Patent Publication No. 2008-237670 [Overview of the project] [Problems that the invention aims to solve]

[0007] Therefore, there is a need for new methods that clearly distinguish between arteries and veins and allow for the confirmation of vascular anatomy and blood flow conditions. [Means for solving the problem]

[0008] In a first embodiment of the present invention, an ultrasound diagnostic system is provided. The ultrasound diagnostic system includes an ultrasound probe that transmits an ultrasound signal at each of a plurality of locations in a subject where a plurality of blood vessels exist and receives a plurality of corresponding echo signals; an echo signal processing unit that obtains a plurality of ultrasound tomography images corresponding to each of the plurality of locations in the subject based on the plurality of echo signals; an analysis unit that analyzes the plurality of ultrasound tomography images to detect blood vessels in each of the plurality of ultrasound tomography images and analyzes information regarding the shape of the detected blood vessels and the type of artery and vein; a blood vessel image generation unit that processes the plurality of ultrasound tomography images and generates a blood vessel course image showing the course and type of artery and vein of the detected blood vessels; a blood vessel state analysis unit that identifies abnormal locations in the shape of the blood vessels based on the information regarding the shape of the blood vessels; and a superimposed image generation unit that superimposes the abnormal locations in shape onto the blood vessel course image.

[0009] In a second embodiment of the present invention, a program is provided for processing a plurality of ultrasound tomography images collected by an ultrasound diagnostic system. The ultrasound diagnostic system includes an ultrasound probe that transmits an ultrasound signal at each of a plurality of locations in a subject where a plurality of blood vessels exist and receives a corresponding plurality of echo signals. The program causes a processor to perform the following steps: obtain a plurality of ultrasound tomography images corresponding to each of a plurality of locations in the subject based on the plurality of echo signals; analyze the plurality of ultrasound tomography images to detect blood vessels in each of the plurality of ultrasound tomography images and analyze information regarding the shape of the detected blood vessels and the type of artery or vein; process the plurality of ultrasound tomography images to generate a vascular course image showing the course and type of artery or vein of the detected blood vessels; identify locations of morphological abnormalities in the blood vessels based on the information regarding the shape of the blood vessels; and superimpose the locations of morphological abnormalities onto the vascular course image. [Effects of the Invention]

[0010] According to the invention of the above embodiment, it is possible to provide images that clearly show the difference between arteries and veins, and that allow for confirmation of the course of blood vessels and their blood flow state. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram showing the configuration of an ultrasound diagnostic system in an embodiment of the present invention. [Figure 2] This is a conceptual diagram showing a functional block in an embodiment of the present invention. [Figure 3] This is a flowchart that includes multiple processing steps for presenting the condition of blood vessels to the observer. [Figure 4] This diagram illustrates the process of acquiring B-mode images. [Figure 5] This diagram illustrates how the blood vessel shape analysis unit uses a pre-trained deep learning model to analyze the shape and type of blood vessels contained in B-mode tomographic images. [Figure 6] This figure shows an analysis results management table for managing the results of blood vessel analysis. [Figure 7] This diagram illustrates how the vascular condition analysis unit uses a pre-trained deep learning model to analyze the state of blood vessels. [Figure 8] This is a conceptual diagram illustrating the process by which superimposed images are generated in a particular embodiment of the present invention. [Figure 9] This shows a composite image created by combining a superimposed image with a real-time image of the subject being photographed. [Modes for carrying out the invention]

[0012] Embodiments of the present invention will be described below with reference to the drawings. The ultrasound diagnostic system 100 shown in Figure 1 includes a transmitting beamformer 103 that drives a plurality of vibrating elements 102a arranged in an ultrasound probe 102 to generate a pulsed ultrasound signal, and a transmitter 104 that radiates the generated pulsed ultrasound signal to a subject (not shown). The pulsed ultrasound signal generates echoes that are reflected within the subject and return to the vibrating elements 102a. In a particular embodiment, the subject is a human or a non-human mammal, and the structure being observed is a limb or neck, etc. The echoes are converted into electrical signals by the vibrating elements 102a, and the electrical signals are received by a receiver 105. The electrical signals representing the received echoes, i.e., the echo signals, are amplified by the required gain in the receiver 105 and then input to a receiving beamformer 106, where receiving beamforming is performed. The receiving beamformer 106 outputs ultrasound data after receiving beamforming.

[0013] The receiving beamformer 106 may be a hardware beamformer or a software beamformer. If the receiving beamformer 106 is a software beamformer, it may comprise one or more processors 107, including any one or more of the following: a graphics processing unit (GPU), a microprocessor, a central processing unit (CPU), a digital signal processor (DSP), or other types of processors capable of performing logical operations. The processors comprising the receiving beamformer 106 may consist of processors other than the processors 107 described below, or they may consist of processors 107. The echo signal before receiving beamforming and the ultrasonic data after receiving beamforming are stored in the memory 109. One or more of the multiple processors 107 may reside on the network to which the communication interface is connected.

[0014] When the echo signal is received, the processor 107 can process the data in real time during the scanning session. For the purposes of this disclosure, the term "real time" is defined to include procedures that are performed without any intentional delay.

[0015] Alternatively, the data can be temporarily stored in a buffer (not shown) during the ultrasonic scan and processed not in real time but in a live operation or an offline operation. In the present disclosure, the term "data" can be used to refer to one or more data sets obtained using an ultrasonic diagnostic system.

[0016] The ultrasonic data can be processed by the processor 107 in other or different mode-related modules (e.g., B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, contrast mode, elastography, TVI, strain, strain rate, etc.) to create data for an ultrasonic image. For example, one or more modules can generate ultrasonic images such as B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, contrast mode, elastography, TVI, strain, strain rate, and combinations thereof.

[0017] A video processor module may be provided that reads an image frame from the memory and displays the image frame in real time while a treatment is being performed on a subject. The video processor module can store the image frame in an image memory, and the ultrasonic image is read from the image memory and displayed on the display device 108.

[0018] Note that, as used herein, the term "image" broadly refers to both visible images and data representing visible images. Also, the term "data" can include raw data, which is ultrasonic data (echo signal or sound beam signal) before scan conversion operation, and image data, which is data after scan conversion operation.

[0019] If processor 107 includes multiple processors, the processing tasks described above that are handled by processor 107 may be handled by multiple processors. For example, the first processor can be used to demodulate and decimate the RF signal, and the second processor can be used to further process the data and then display the image. Also, if, for example, the receiving beamformer 106 is a software beamformer, its processing functions may be performed by a single processor or by multiple processors.

[0020] Display devices 108 include LED (Light Emitting Diode) displays, LCD (Liquid Crystal Display), microLED displays, and organic EL (Electro-Luminescence) displays.

[0021] Memory 109 is any known data storage medium. For example, the ultrasound diagnostic system 100 includes multiple memory 109s, including non-transient and transient storage media. Non-transient storage media are non-volatile storage media such as HDDs (Hard Disk Drives) and ROMs (Read Only Memory). Non-transient storage media may also include portable storage media such as CDs (Compact Disks), DVDs (Digital Versatile Disks), and Blu-ray Discs (registered trademarks). Programs executed by the processor 107 are stored in the non-transient storage media. Protocols described later are also stored in the non-transient storage media. Transient storage media can be volatile storage media such as RAM (Random Access Memory). These may all be stored in the same memory 109, or at least one of them may be stored in a different memory 109. Memory 109 may also be multiple data storage media deployed on the cloud.

[0022] The user interface 110 can accept input from an operator. For example, the user interface 110 accepts instructions and information input from an operator. The user interface 110 is composed of a keyboard, hard keys, and soft keys, etc. The user interface 110 may also include pointing devices such as a mouse, touch panel, pen tablet, touchpad, trackball, joystick, and various input devices such as eye tracking and voice input.

[0023] The speaker 111 outputs sound controlled by the processor 107. Specifically, the speaker 111 outputs sound based on a signal input from the processor 107. The ultrasound diagnostic system 100 shown in Figure 1 can function as an ultrasound data processing device that generates B-mode images, color Doppler images, vascular pathway images, and superimposed images displayed on the display device 108.

[0024] In certain embodiments, the ultrasonic probe 102 incorporates a magnetic sensor 123. This magnetic sensor 123 is composed of, for example, a Hall element, a magnetoresistive element, a magnetohistoelectric element, a GSR (GHz-Spin-Rotation) element, or a Faraday element. The magnetic sensor 123 detects the magnetism generated from the transmitter 121. The transmitter 121 and the magnetic sensor 123 are provided to detect the position and tilt of the ultrasonic probe 102.

[0025] The detection signal from the magnetic sensor 123 is input to the processor 107. The detection signal from the magnetic sensor 123 may be input to the processor 107 via a cable (not shown) or wirelessly.

[0026] The processor 107 determines the position of the magnetic sensor 123 and the directions of the three mutually orthogonal axes set on the magnetic sensor 123 based on the magnetic detection signal from the magnetic sensor 123. Specifically, the magnetic sensor 123 determines its position and direction using the change in the magnetic flux line signal due to the influence of the magnetic field of the transmitter 121. This allows the processor 107 to determine the position and direction of the magnetic sensor 123 (directions relative to the three mutually orthogonal axes) in a coordinate system in three-dimensional space with the transmitter 121 as the origin.

[0027] Figure 2 shows a functional block 300 that is realized when a program 112 stored in memory 109 is executed by processor 107. In a preferred embodiment of the present invention, the functional block 300 includes an echo signal processing unit 302 which includes a B-mode image processing unit 304 and a color Doppler image processing unit 306, an acquisition location identification unit 310, a blood vessel shape analysis unit 312, a blood vessel state analysis unit 314, and an image generation unit / superimposed image generation unit 320.

[0028] The B-mode image processing unit 304 processes the echo signal received from the receiving beamformer 106 to generate a B-mode image 710 (Figure 5). The color Doppler image processing unit 306 processes the echo signal for the color Doppler image received from the receiving beamformer 106 to determine whether the blood vessels included in the B-mode image 710 (Figure 5) are veins or arteries. It is possible to use power Doppler or other methods instead of color Doppler, but for simplicity of explanation, color Doppler will be used as an example. The Doppler mode may be any of the following: pulsed Doppler mode, continuous wave Doppler mode, color Doppler mode, power Doppler mode, high PRF Doppler mode, duplex mode, or triplex mode.

[0029] The data collection location identification unit 310 obtains location information of the location where each B-mode image was collected and tags the location information with the corresponding B-mode image. The blood vessel shape analysis unit 312 analyzes the shape of the blood vessels contained in the B-mode image with or without the trained deep learning model 322 (Figure 5). The blood vessel state analysis unit 314 analyzes the state of the blood vessels based on the shape of the blood vessels analyzed by the blood vessel shape analysis unit 312. The blood vessel state analysis unit 314 analyzes the state of the blood vessels with or without the trained deep learning model 344 (Figure 7).

[0030] The image generation unit / superimposed image generation unit 320 processes the image generated by the echo signal processing unit 302 and adds the information generated by the acquisition position identification unit 310, the blood vessel shape analysis unit 312, and the blood vessel condition analysis unit 314 to the output image.

[0031] Figure 3 is a flowchart showing a plurality of processing steps for presenting the state of blood vessels to an observer according to the procedure of the present invention. The plurality of processing steps in Figure 3 are executed by the processor 107 when the program 112 stored in the memory 109 shown in Figure 1 is executed. The procedure starts at step 401. In certain embodiments, the procedure is started by the operator making a keyboard input to signal the start of this procedure. In other embodiments, the operator can start the processing routine by double-clicking an icon provided for this procedure. In other embodiments, the processing routine is started by voice command.

[0032] After the procedure begins in step 401, the counter is reset in step 403. Specifically, each B-mode ultrasound image is identified by its tomographic image number. In this example, the tomographic image number counter N is set to 1.

[0033] In step 405, the B-mode image processing unit 304 (Figure 2) is activated and B-mode images are acquired. The ultrasound probe 102 transmits an ultrasound signal at each of several locations in the subject where multiple blood vessels exist, and receives multiple corresponding echo signals. Based on the multiple echo signals, the B-mode image processing unit 304 obtains multiple ultrasound tomographic images corresponding to each of the multiple locations in the subject. Figure 4 shows the process of acquiring B-mode images. As shown in Figure 4, the operator performs a sweep scan (sweep) at a constant speed in the longitudinal direction of the arm from the forearm to the upper arm. During the sweep scan, the probe 102 is moved in the elevation direction 205 such that the elevation direction of the probe 102 coincides with the longitudinal direction of the arm and the azimuth direction of the probe 102 is perpendicular to the longitudinal direction of the arm. The sweep scan of the ultrasound probe 102 may also be performed in the reverse direction, from the upper arm side of the subject to be examined to the vicinity of the wrist. During a sweep scan, the operator can be guided on how to perform the sweep scan using audio from speaker 111 (Figure 1) and a tutorial using display device 108 (Figure 1). In this example, the target of the sweep scan is the subject's arm 201 and the blood vessels 203 contained therein, but the target of the sweep scan may be, for example, the neck or legs and the blood vessels contained therein. It may also be a part of the forearm, a part of the upper arm, etc. Furthermore, the subject of the examination may be an animal other than a human (livestock such as cows and pigs, or companion animals such as dogs and cats).

[0034] Returning to Figure 3 and continuing the explanation, the position identification unit 310 acquires positional information to determine the location where the B-mode tomographic images collected in step 405 were acquired (step 407). In the example in Figure 4, cross-sectional images are continuously obtained by moving the probe 102 in a direction perpendicular to the image. At this time, the positional information of the probe 102 can be determined by the magnetic sensor 123 and acceleration sensor built into the probe 102. The positional information is tagged to each B-mode tomographic image along with the tomographic image number by the acquisition position identification unit 310. The positional information can include three-dimensional coordinates and the orientation of the probe. The positional information of the probe 102 can also be obtained by detecting the probe position using a camera or by using a jig that mechanically moves the probe. Furthermore, even if accurate positional information cannot be obtained, it can be assumed that the operator moves the probe as parallel and at a constant speed as possible, and that the obtained B-mode tomographic images were obtained at equal intervals.

[0035] In step 421, a color Doppler transmitted ultrasound signal is irradiated, and the color Doppler image processing unit 306 generates a color Doppler image based on the echo signal (step 421). The irradiation of the color Doppler transmitted ultrasound signal can be limited to the same range as the B-mode tomography image, or to a range that covers all of the multiple blood vessels previously detected by the blood vessel shape analysis unit 312.

[0036] In certain embodiments, the blood vessel shape analysis unit 312 is activated in step 423 to detect blood vessels and analyze their shape for each B-mode image. In other embodiments, blood vessel analysis is not performed for all B-mode images, but only for some frames.

[0037] The blood vessel shape analysis unit 312 can analyze the shape of blood vessels contained in B-mode tomography images using various methods. In a particular embodiment, as shown in Figure 5, the blood vessel shape analysis unit 312 inputs the B-mode tomography image 710 into a trained deep learning model 322 and obtains the output 330 from the trained deep learning model 322 to analyze the location and shape of multiple blood vessels contained in the B-mode tomography image. The trained deep learning model 322 can also be input with the B-mode tomography image 710 and a color Doppler image, and the color Doppler image can be used to identify the location of the blood vessels. The trained deep learning model 322 can be generated by training an existing machine learning model such as YOLO™ or U-Net™, which is capable of object detection and object boundary extraction (segmentation). During training, it can be created by training the machine learning model with many blood vessel images and their location and shape information (annotations). In other embodiments, the vascular shape analysis unit 312 identifies the location and shape of multiple blood vessels included in the B-mode tomography using known methods such as segmentation and pattern matching. In specific embodiments, a trained deep learning model 322 can identify the location of nerves and muscles included in the B-mode tomography.

[0038] Figure 6 shows the analysis result management table 600 for managing the analysis results of blood vessels. When the blood vessel shape analysis unit 312 detects a blood vessel depicted in the B-mode tomographic image, it assigns the blood vessel number 610 to that blood vessel. In the example of image 710 in Figure 5, two blood vessels 711 and 713 are detected, and corresponding blood vessel numbers are assigned to each. Also, a plaque 715 is present in one of the blood vessels. The blood vessel shape analysis unit 312 identifies a function that defines the shape of the plaque 715.

[0039] In the example shown in Figure 6, for each detected blood vessel, the arteriovenous type 620, outer contour 630, and internal structure contour 640 are registered in the analysis result management table 600. The arteriovenous type 620 is the result of processing by the color Doppler image processing unit 306. Details of the processing by the color Doppler image processing unit 306 will be described later. The outer contour 630 registers a function that approximates the shape of the detected blood vessel contour. This function may be a function that identifies an ellipse. The outer contour 630 may be used in conjunction with the inner contour of the blood vessel, or it may be replaced by the inner contour of the blood vessel. The outer contour 630 may be used in conjunction with the contour of the center of the blood vessel wall, or it may be replaced by the contour of the center of the blood vessel wall.

[0040] The internal structure contour 640 registers a function that defines the shape of internal structures within the blood vessel that deviate from the circular or elliptical shape appearing in the cross-section of the blood vessel, such as plaque within the blood vessel. If there are no abnormalities such as plaque within the blood vessel, information indicating the absence of abnormalities (such as a blank) is registered in the internal structure contour 640. In certain embodiments, in step 405, in addition to the B-mode tomography, a color Doppler image is acquired. In this case, the blood vessel shape analysis unit 312 can detect the location of the blood vessel by referring to both the B-mode tomography and the color Doppler image, and can register all the information in Figure 6. The blood vessel shape analysis unit 312 can operate a blood vessel detection and classification algorithm in conjunction with power Doppler to identify the type of artery and vein and identify the boundaries of the blood vessel. By analyzing the ultrasound tomography image obtained in color Doppler mode, the blood vessel shape analysis unit 312 can detect the location and shape of the blood vessel by acquiring blood flow information. Note that if a flash occurs in Doppler mode, areas other than blood vessels may be colored. To prevent this and limit the coloring to within the blood vessels, the blood vessel region can be masked by detecting blood flow velocity and power information. In this way, all vessel numbers 610, arteriovenous types 620, lateral contours 630, and internal structure contours 640 are registered.

[0041] In step 423, the image analysis unit 311 analyzes not only B-mode tomography images but also color Doppler images to identify whether each blood vessel in the B-mode tomography images is a vein or an artery. The hemangioma analysis by the image analysis unit 311 can be performed using or without the pre-trained deep learning model 322. As a result, all items shown in Figure 6 are registered. The image analysis unit 311 can also input color Doppler images in addition to B-mode tomography images into the pre-trained deep learning model 322 to identify the shape and type of blood vessel (vein or artery). Note that if classification between arteries and veins can be performed simply by inputting B-mode tomography images into the pre-trained deep learning model, step 421 can be omitted. A processing flow can also be adopted in which color Doppler images are collected only when a determination cannot be made from B-mode tomography images.

[0042] Continuing to refer to Figure 3, in step 425, the vascular condition analysis unit 314 performs an analysis of the vascular condition. Figure 7 illustrates how the vascular condition analysis unit 314 performs the analysis of the vascular condition. The vascular condition analysis unit 314 can analyze the state of blood vessels included in the set of B-mode tomography images 720 using various methods. In a particular embodiment, as shown in Figure 7, the vascular condition analysis unit 314 analyzes the state of multiple blood vessels included in the set of B-mode tomography images 720 by inputting the set of B-mode tomography images 720 into a trained deep learning model 344 and obtaining an output 340 from the trained deep learning model 344. In this example, the information in the analysis result management table 600 in Figure 6 is input into the trained deep learning model 344 in association with the corresponding B-mode tomography image frames. The outer contour 630 and internal structure contour 640 of the analysis result management table 600 in Figure 6 are added as shapes to the corresponding B-mode tomographic image frames, and the arteriovenous type 620 of the analysis result management table 600 in Figure 6 is added as the color of the outer contour 630 shape, and these are input into the trained deep learning model 344. The trained deep learning model 344 can be generated by training an existing machine learning model such as YOLO™ or U-Net™ that is capable of object detection and object boundary extraction (segmentation). During training, it can be created by having the machine learning model learn a large number of vascular images and their location, shape information, and shape anomaly information (annotation). The set of B-mode tomographic images 720 may be all B-mode tomographic images collected after step 401 in Figure 3 has started, or it may be a selection of the most recent B-mode tomographic images.

[0043] In other embodiments, the vascular condition analysis unit 314 can analyze the shape of blood vessels using known analysis methods. In other embodiments, the trained deep learning model 344 receives information from the analysis result management table 600, but does not receive the B-mode tomographic images themselves. In other embodiments, the vascular condition analysis unit 314 analyzes the information from the analysis result management table 600 without using the trained deep learning model 344. For example, the vascular condition analysis unit 314 can analyze the change in the circularity ratio of the cross-section of each blood vessel over multiple frames. The circularity ratio can be calculated using the following formula. Circularity C = 4πA / P 2 Here, A is the area of ​​the ellipse and P is the circumference of the ellipse. For a perfect circle, the degree of circularity C is 1, and the value decreases as the shape becomes more distorted. The sweep scan described above can be performed independently of the subject's heartbeat, or only at specific timings of the subject's heartbeat. Arteries expand and contract in accordance with the heartbeat cycle, but when the sweep scan is performed independently of the subject's heartbeat, the influence of the heartbeat can be reduced by correcting the size of the artery according to the blood flow velocity.

[0044] In the example in Figure 7, the trained deep learning model 344 outputs which blood vessels and which sections have what kind of abnormalities. The numerical value of the abnormal section corresponds to the value of counter N of the tomographic image number explained in Figure 3. In this example, it is shown that stenosis (abnormality type 1) exists in blood vessel number 1 from frame 164 to frame 239. Areas with intimal thickening, areas with thin vessel walls, areas with aneurysms, areas of strong tortuosity, areas of fibrosis, branching points, and areas where multiple blood vessels are close together are also assigned corresponding abnormality type numbers.

[0045] The vascular anatomy image and superimposed image described in Figure 8 can clearly show the observer areas of shape abnormalities that cannot be identified in B-mode tomography. In Figure 8, Figure 510, which shows the actual state of the blood vessels, a collection of multiple B-mode tomography images 530 collected at multiple locations 501 to 507, a collection of images 550 in which the location, shape, and type of blood vessels were detected in each B-mode tomography image, a vascular anatomy image 570, and a superimposed image 590 are displayed side by side. The superimposed image 590 includes the vascular anatomy image 570 and additional information superimposed on it.

[0046] In the example shown in Figure 510, which illustrates the actual structure of blood vessels, there is a relatively large vein 513, two relatively small veins 515 and 517 that merge with the relatively large vein 513, and an artery 511 that extends close to the relatively large vein 513.

[0047] In the B-mode tomography image 531 acquired at position 501, veins 513 and 515 are depicted as cross-sections 543 and 545, and artery 511 is depicted as cross-section 541. Cross-sections of each blood vessel are also depicted in B-mode tomography images 533, 535, and 537. In this example, a plaque 549 is present in vein 513 in B-mode tomography image 533. In a particular embodiment, the blood vessel in which the plaque 549 is present is defined by a part of the elliptical function that defines the main outer circumference of the blood vessel (outer contour 630 in Figure 6) and a part of the elliptical function that defines the plaque 549 (internal structure contour 640 in Figure 6).

[0048] Images 551 and 553, which detect the location, shape, and type of blood vessels, show a significant change in the cross-section of vein 513. The vascular condition analysis unit 314 determines that there is an abnormality in the shape of the blood vessel at that location if there is a significant change in the cross-section of the blood vessel and certain conditions are met. That is, the vascular condition analysis unit 314 identifies the location of the abnormal blood vessel shape (step 425). The vascular condition analysis unit 314 is configured to analyze changes in one or more of the following in each of the multiple B-mode tomographic images: the ratio of height to width of the blood vessels, the circumference of the blood vessels, and the thickness of the blood vessel walls. In this case, the height is defined by the depth direction of the B-mode tomographic image, the width is defined by the left-right direction of the B-mode tomographic image, the circumference and the thickness of the blood vessel walls are defined by the cross-sectional shape of the blood vessel in the plane of the B-mode tomographic image, and the longitudinal direction of the blood vessel extends in the direction intersecting the B-mode tomographic image.

[0049] The vascular condition analysis unit 314 identifies abnormalities in the shape of blood vessels by detecting artificial objects placed in the blood vessels, narrowed areas of blood vessels, treatment scars, and plaque. In certain embodiments, the vascular condition analysis unit 314 can also identify abnormal blood vessels if muscle is present between them and the body surface, or if nerves are located close to the blood vessels. Here, "close" can mean a distance of 1 cm or less, more preferably 5 mm or less, between the blood vessel and the nerve. Furthermore, if a vein is located close to an artery, the risk of vein puncture increases, so this vein can also be identified as an abnormal blood vessel. Here, "close" can mean a distance of 5 mm or less, more preferably 3 mm or less, between the vein and the artery.

[0050] Artificial objects placed in blood vessels include, for example, stents to widen narrowed blood vessels and ensure blood flow, coils to embolize abnormal blood vessel lumens such as aneurysms, filters to prevent blood clots from traveling to vital organs (such as the lungs), and artificial blood vessels. In all cases, their shape and material are known, so the vascular condition analysis unit 314 can detect the artificial objects placed in the blood vessels using the trained deep learning model 344.

[0051] For example, artificial blood vessels are linear and have a different shape from natural blood vessels. Also, after bypass surgery, blood vessels that should not normally be there are often present in other pathways, and their location and direction of course can be inferred. Furthermore, their wall structure is uniform and they appear highly luminous (biological blood vessels have a multilayer structure (intima, media, adventitia) and appear somewhat layered). In addition, they may contain Teflon®-based or polyester-based materials, and since these materials easily reflect ultrasound, a unique acoustic shadow may occur around them. For this reason, the vascular condition analysis unit 314 can perform detection using the trained deep learning model 344.

[0052] The vascular condition analysis unit 314 can be equipped with a vascular dimension tracker, which tracks changes in geometric information (width, height, circumference, etc.) across multiple frames for all detected blood vessels. The vascular dimension tracker compares the detected blood vessel region in the previous frame to determine if it is the same blood vessel. This allows the vascular condition analysis unit 314 to analyze changes in the geometric dimensions of the same blood vessel and detect vascular stenosis, treatment scars, and plaque. Because it tracks changes in geometric information (width, height, circumference, etc.) across multiple frames, it becomes possible to determine whether a blood vessel whose cross-section is depicted as a flattened ellipse in the B-mode tomography image is due to a shape anomaly, or whether it is a blood vessel that extends at a large inclination relative to the plane of the B-mode tomography image. Shape anomalies can include any of the following: stenosis of at least one blood vessel, areas with intimal thickening, areas with thin blood vessel walls, areas with aneurysms, areas of strong tortuosity, areas of fibrosis, branching points, or areas where multiple blood vessels are in close proximity.

[0053] Returning to Figure 3 and continuing the explanation, step 427 generates the vascular pathway image 570. The generation of the vascular pathway image 570 in step 427 can also be performed before the analysis of the vascular state in step 425. The vascular pathway image 570 can be plotted by plotting the left and right ends of each vessel, as shown in Figure 8.

[0054] As shown in image 551 of Figure 8, the vascular shape analysis unit 312 can identify the left end 561 and right end 562 of artery 511 and the left end 563, 565 and right end 564, 566 of veins 513 and 515. By sequentially plotting multiple points corresponding to these, it can generate a vascular course image 570 as seen from the body surface. When plotting, the spacing between points to be plotted can be adjusted using the position information tagged by the acquisition position identification unit 310, thereby preventing the vascular course image 570 from deviating from the actual position and shape of the blood vessels. As shown in image 555, in areas where two or more blood vessels overlap, only the blood vessels located at a shallow position can be drawn, and the blood vessels located at a deeper position can not be drawn. In this example, it is clearly shown that part of artery 571 is located below vein 573. Veins 573-577 are drawn without being hidden by other blood vessels.

[0055] Returning to Figure 3 and continuing the explanation, in step 429, the superimposed image 590 is generated. In the example in Figure 8, artery 591 is displayed in red. Veins 593, 595, and 597 are displayed in blue. Plaque 549 present in vein 513 is shown as a pattern 599 in the corresponding area of ​​vein 593 in the superimposed image 590. The pattern of the abnormal shape can be changed according to the type of abnormality. That is, the image generation unit / superimposed image generation unit 320 displays a vascular course image showing the detected course of the blood vessel and the type of artery or vein, and superimposes the display of the abnormal shape onto the vascular course image. The pattern 599 can also be a different color or has a different brightness from the color of the corresponding blood vessel. The display of the abnormal shape can be changed to blink. Furthermore, for example, a display method can be adopted that allows for intuitive distinction between mild and severe stenosis (for example, a slow blink or a color close to the original blood vessel color for mild cases, and a rapid blink or a color completely different from the original blood vessel for severe cases). When identifying and displaying blood vessels that should not be punctured, the color can be changed according to the reason why puncture should not be performed (proximity to artery, stenosis, presence of plaque, fibrosis). The specific location and nature of the problem can be displayed in real time. By projecting the blood vessel course from the body surface onto images obtained from the ultrasound diagnostic device in real time, and further overlaying information about the local blood vessel, the efficiency of the examination can be expected to improve. In addition, by displaying the superimposed image 590, the efficiency of processing and the accuracy of procedures can be improved in various situations such as monitoring treatment surgery, postoperative observation, and preoperative diagnosis.

[0056] The vascular anatomy image 570 and / or superimposed image 590 can be displayed in three dimensions and rotated around the longitudinal axis. Furthermore, icons can be selectively displayed to indicate the depth of multiple locations of each vessel in numerical or color form. In certain embodiments, the superimposed image 590 displays identifiable areas of normal vascular shape. Areas of normal vascular shape can be suitable for puncturing or forming an autologous shunt of at least one vessel.

[0057] Returning to Figure 3 and continuing the explanation, once step 429 is completed, step 433 determines whether or not a termination event has occurred. The operator can generate a termination event through various operations. For example, a termination event can be generated by pressing a dedicated button to signal the end of the vascular sweep scan, pressing a dedicated button to signal the start of the vascular sweep scan (i.e., it is set to start when pressed once and stop when pressed a second time), selecting another operating mode (e.g., M mode, power Doppler mode), or moving the ultrasound probe 102 away from the subject for a certain period of time or longer.

[0058] If a termination event is detected in step 433, the process ends (step 435). If no termination event is detected in step 433, the value of the tomographic image number counter N is incremented by 1 in preparation for acquiring the next B-mode tomographic image (step 437).

[0059] In certain embodiments, the relationship between the examination area and the superimposed image 590 is more clearly shown. For example, the examination area may be imaged in real time using a camera (not shown), and the superimposed image 590 of Figure 8 may be superimposed on the captured real image. Alternatively, the superimposed image 590 can be projected onto the examination area in real time using a projector (not shown). That is, if the examination area is located on the upper limb, lower limb, or neck of the subject, the superimposed image 590 is displayed superimposed on the image of such an examination area or the examination area itself. Figure 9 shows an image obtained by combining the superimposed image 590 with a real-time image 701 of the target 201. The acquisition position identification unit 310 (Figure 2) can identify the current position and orientation of the ultrasound probe 102. Furthermore, by analyzing the real-time image 702 of the ultrasound probe (real-time image analysis), the current position and orientation of the ultrasound probe 102 in the real-time image can be identified. This allows the two images to be aligned, and the superimposed image 590 to be combined with the real-time image 701 of the subject 201 to generate the composite image 700 shown in Figure 9. The composite image 700 includes the superimposed image 703 and a display 705 of the abnormal shape areas contained therein. Therefore, the observer can easily determine the location of abnormal blood vessels in the subject.

[0060] Although the above description has focused on the optimal embodiment of the present invention, as will be apparent to those skilled in the art, the present invention can be implemented by making various changes and modifications to the embodiment within its technical scope. [Explanation of Symbols]

[0061] 100 Ultrasound Diagnostic Systems 102 Ultrasound probe 102a Vibration element 103 Transmitting beamformer 104 Transmitter 105 Receiver 106 Receiving beamformer 107 Processor 108 Display device 109 memory 110 User Interface 111 Speakers 112 Programs 121 Transmitter 123 Magnetic Sensor 201 Subject to Photograph 203 Blood vessels 300 Functional Blocks 302 Echo signal processing unit 304 B-mode image processing unit 306 Color Doppler Image Processing Unit 310 Collection location identification unit 311 Image Analysis Department 312 Blood vessel shape analysis department 314 Vascular Condition Analysis Department 320 Image generation unit / Superimposed image generation unit 322 Pre-trained Deep Learning Models 344 pre-trained deep learning models 501, 503, 505, 507 Multiple locations 510 Diagram showing the actual appearance of blood vessels 511 Arteries Veins 513, 515, 517 531, 533, 535, 537 B-mode tomographic images 541, 543, 545 cross-sections 549 Plaque Images 551, 553, 555, 557: Images in which the location, shape, and type of blood vessels were detected. 570 Vascular path images 571 Arteries Veins 573, 575, 577 590 superimposed images 591 Arteries 593, 595, 597 Veins 595 patterns 600 Analysis Result Management Table 610 Blood vessel number 620 Types of arteries and veins 630 Outer contour 640 Internal structural contour 700 composite images 701 Real-time image of the subject being photographed 702 Real-time images of an ultrasound probe 703 Superimposed image 705 Display of abnormal shape 710 B-mode tomographic images 711, 713 blood vessels 715 Plaque Set of 720 B-mode tomographic images

Claims

1. An ultrasound probe that transmits an ultrasound signal at each of multiple locations where multiple blood vessels exist in a subject, and receives multiple corresponding echo signals, An echo signal processing unit that obtains multiple ultrasound tomographic images corresponding to each of multiple locations in the subject based on the multiple echo signals, An image analysis unit analyzes the plurality of ultrasound tomography images to detect blood vessels in each of the plurality of ultrasound tomography images, and analyzes information regarding the shape of the detected blood vessels and the type of artery or vein. A vascular image generation unit processes the plurality of ultrasound tomographic images and generates a vascular image showing the direction of the detected blood vessels and the type of artery and vein. A vascular condition analysis unit that identifies abnormal locations in the shape of the blood vessels based on the information regarding the shape of the blood vessels, A superimposed image generation unit that superimposes the aforementioned abnormal shape area onto the blood vessel path image, Includes, The vascular condition analysis unit uses a trained deep learning model to identify abnormalities in the shape of the blood vessels in real time, even before all of the multiple ultrasound images have been collected, when only a portion of the multiple ultrasound images have been collected. An ultrasound diagnostic system in which a portion of the vascular path image overlaid with the aforementioned abnormal shape is displayed in real time when only a portion of the plurality of ultrasound tomographic images has been acquired, before all of the plurality of ultrasound tomographic images have been acquired.

2. The ultrasound diagnostic system according to claim 1, wherein the abnormal shape location includes any of the following: a stenosis of at least one of the multiple blood vessels, a location with intimal thickening, a location with a thin blood vessel wall, a location with an aneurysm, a location with severe tortuosity, a location with fibrosis, a location where multiple blood vessels are in close proximity.

3. The vascular condition analysis unit is configured to identify the normal shape of at least one blood vessel, The ultrasound diagnostic system according to claim 2, wherein the normal-shaped portion of the at least one blood vessel includes a portion suitable for puncturing or forming an autologous shunt of the at least one blood vessel.

4. The ultrasound diagnostic system according to claim 2, wherein the abnormal shape of the at least one blood vessel is identifiable in the image of the blood vessel included in the blood vessel course image.

5. The vascular path image includes an image of at least one blood vessel that extends intersecting the cross-sections of the plurality of ultrasound tomographic images. The vascular condition analysis unit is configured to identify the normal shape of at least one blood vessel, The ultrasound diagnostic system according to claim 3, wherein the normal-shaped portion of the at least one blood vessel includes a portion suitable for the formation of an autologous shunt of the at least one blood vessel.

6. The ultrasound diagnostic system according to claim 5, wherein the vascular condition analysis unit identifies the normal shape of the at least one blood vessel based on one or more of the following: an artificial object placed in the at least one blood vessel, a stenotic area of ​​the at least one blood vessel, a treatment scar in the at least one blood vessel, and the location of plaque present in the at least one blood vessel.

7. The ultrasound diagnostic system according to claim 1, wherein the image analysis unit inputs the plurality of ultrasound tomographic images into a machine learning model and obtains output from the machine learning model to identify at least one blood vessel from the plurality of blood vessels.

8. The machine learning model is configured to identify the location of nerves and / or muscles included in the plurality of ultrasound images. The ultrasound diagnostic system according to claim 7, wherein the vascular condition analysis unit identifies the location of the shape abnormality based on the location of the nerve and / or muscle.

9. The vascular path image includes an image of at least one blood vessel that extends intersecting the cross-sections of the plurality of ultrasound tomographic images. The aforementioned ultrasound probe is capable of operating in Doppler mode. The ultrasound diagnostic system according to claim 1, wherein the image analysis unit obtains blood flow information by analyzing a plurality of ultrasound tomographic images obtained in Doppler mode and identifies at least one blood vessel from the plurality of blood vessels.

10. The ultrasound diagnostic system according to claim 9, wherein the vascular image generation unit is configured to identify the boundary of at least one vein and / or at least one artery in the vascular course image by analyzing a plurality of ultrasound tomographic images obtained in Doppler mode.

11. The ultrasound diagnostic system according to claim 9, wherein the Doppler mode includes any of pulsed Doppler mode, continuous wave Doppler mode, color Doppler mode, powered Doppler mode, high PRF Doppler mode, duplex mode, or triplex mode.

12. The ultrasound diagnostic system according to claim 1, wherein the plurality of locations are located in the upper limb, lower limb, or neck of the subject, and the vascular path image is displayed superimposed on the image of the upper limb, lower limb, or neck or the upper limb, lower limb, or neck itself.

13. The system further includes a collection position identification unit that identifies the collection position of the ultrasound probe that collected each of the plurality of ultrasound tomography images, The ultrasound diagnostic system according to claim 12, wherein the vascular path image is generated based on the identified acquisition location.

14. The ultrasonic diagnostic system according to claim 13, wherein the collection position identification unit includes any of the following: a magnetic sensor and / or an acceleration sensor disposed on the ultrasonic probe, a camera for photographing the ultrasonic probe, and a jig for mechanically moving the ultrasonic probe.

15. The ultrasound diagnostic system according to claim 1, wherein the vascular path image includes a two-dimensional or three-dimensional image.

16. The plurality of ultrasound tomography images are a plurality of B-mode tomography images, The ultrasound diagnostic system according to claim 1, wherein the vascular condition analysis unit identifies the location of abnormalities in the shape of the blood vessels by inputting the plurality of B-mode tomographic images into the trained deep learning model.

17. The vascular condition analysis unit is configured to analyze changes in each of the plurality of ultrasound tomographic images with respect to one or more of the following: the ratio of height to width of at least one blood vessel included in the plurality of blood vessels, the circumference of the at least one blood vessel, and the thickness of the blood vessel wall of the at least one blood vessel. The height is defined by the depth direction of the plurality of ultrasound tomographic images. The width is defined by the left-right direction of the plurality of ultrasound tomographic images. The circumference length and the thickness of the vessel wall are determined by the cross-sectional shape of at least one vessel in the plane of the plurality of ultrasound tomographic images. The ultrasound diagnostic system according to claim 1, wherein the longitudinal direction of at least one blood vessel extends in a direction that intersects the plurality of ultrasound tomographic images.

18. The ultrasound diagnostic system according to claim 1, wherein the plurality of ultrasound tomographic images are collected by sweeping the ultrasound probe over the plurality of locations independently of the heartbeat of the subject.

19. A program for processing multiple ultrasound tomographic images collected by an ultrasound diagnostic system, wherein the ultrasound diagnostic system includes an ultrasound probe that transmits an ultrasound signal at each of multiple locations where multiple blood vessels exist in a subject and receives a corresponding plurality of echo signals. The aforementioned program, The steps include obtaining a plurality of ultrasound tomographic images corresponding to each of a plurality of locations in the subject based on the plurality of echo signals, The steps include analyzing the plurality of ultrasound tomography images to detect blood vessels in each of the plurality of ultrasound tomography images, and analyzing information regarding the shape of the detected blood vessels and the type of artery or vein, The steps include processing the plurality of ultrasound tomographic images to generate a vascular course image showing the direction and type of artery and vein of the detected blood vessels, A step of identifying a location of abnormality in the shape of the blood vessel based on the information regarding the shape of the blood vessel, The steps include: superimposing the aforementioned abnormal shape onto the image of the blood vessel's course; Make the processor execute it, The abnormalities in the shape of the blood vessels are identified in real time using a trained deep learning model, even before all of the multiple ultrasound images have been collected, when only a portion of the multiple ultrasound images have been collected. A program that displays in real time a portion of the vascular path image on which the aforementioned abnormal shape is superimposed, when only a portion of the multiple ultrasound tomography images has been acquired, before all of the multiple ultrasound tomography images have been acquired.

20. A non-temporary storage medium for storing the program described in claim 19.

Images