Ultrasonic diagnostic apparatus, medical information processing apparatus, information processing method, and program

Abstract

To provide a technology to acquire high resolution blood flow information.SOLUTION: An ultrasonic diagnostic apparatus includes: a first acquisition unit for acquiring measurement data including tissue-derived information and blood flow-derived information on the basis of an ultrasonic signal reflected in a living body; a second acquisition unit for acquiring an amount of deformation between a plurality of frames on the basis of blood flow data that extracts or emphasizes the blood flow-derived information of the measurement data; and a generation unit for generating display image data by synthesizing feature points in the blood flow-derived information for a plurality of frames after performing positioning between the frames on the basis of the amount of deformation.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an ultrasonic diagnostic apparatus, a medical information processing apparatus, an information processing method, and a program. [Background technology]

[0002] Ultrasound diagnostic devices are widely used to observe and diagnose blood flow in living bodies. Ultrasound diagnostic devices generate and display blood flow information from reflected ultrasonic waves using the Doppler method based on the Doppler effect. Examples of blood flow information generated and displayed by ultrasound diagnostic devices include color Doppler images and Doppler waveforms (Doppler spectra).

[0003] Color Doppler images are captured using the Color Flow Mapping (CFM) method. In the CFM method, ultrasonic waves are transmitted and received multiple times on multiple scanning lines. Then, by applying an MTI (Moving Target Indicator) filter to the data sequence at the same position, signals (clutter signals) originating from stationary or slow-moving tissues are suppressed and signals originating from blood flow are extracted. In the CFM method, blood flow information such as blood flow speed, blood flow dispersion, and blood flow power are estimated from this blood flow signal, and the distribution of the estimated results is displayed as a Doppler image.

[0004] It is known that the resolution of B-mode images and Doppler images decreases due to the point spread function (PSF), which is determined by the wavelength of the transmitted ultrasound, the transmit / receive aperture width, etc. Although there are solutions such as increasing the frequency of the transmitted ultrasound, there is a limit to the resolution of the images that can be obtained because there is also a limit to the frequency band of the probe.

[0005] Non-Patent Document 1 describes a super-resolution technology for blood flow images that achieves a resolution of about 1 / 5 of the wavelength of the transmitted ultrasound. A blood flow image with improved resolution is generated by extracting and accumulating high pixel values ​​from multiple blood flow images obtained in time series. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] Jorgen Arendt Jensen et al., "Fast super resolution ultrasound imaging using the erythrocytes," Proc. SPIE 12038, Medical Imaging 2022: Ultrasonic Imaging and Tomography, 120380E (4 April 2022) Summary of the Invention [Problem to be solved by the invention]

[0007] In Non-Patent Document 1, first, the movement of tissues in an ultrasound image is estimated by speckle tracking, and based on the estimation result, the ultrasound images of all frames are aligned with the image of a reference frame. Then, tissue components are removed from the aligned ultrasound images to obtain a blood flow image, and the parts with high pixel values ​​in the blood flow image of each frame are extracted and integrated.

[0008] However, for example, in ultrasound images that do not contain many characteristic structures in the tissue or in ultrasound images where speckle is dominant, the estimation accuracy of speckle tracking may decrease, and the deviation of the blood vessel position may not be corrected sufficiently. In such cases, the method of Non-Patent Document 1 cannot obtain high-resolution blood flow information.

[0009] The present invention has been made in consideration of the above-mentioned problems, and has an object to provide a technique for obtaining high-resolution blood flow information. [Means for solving the problem]

[0010] The present disclosure includes an ultrasound diagnostic device having a first acquisition unit that acquires measurement data including tissue-derived information and blood flow-derived information based on an ultrasound signal reflected within a living body, a second acquisition unit that acquires a deformation amount between multiple frames based on blood flow data in which the blood flow-derived information from the measurement data has been extracted or emphasized, and a generation unit that generates display image data by aligning feature points in the blood flow-derived information between frames based on the deformation amount and synthesizing the multiple frames. Effect of the Invention

[0011] According to the present invention, high-resolution blood flow information can be obtained. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of an ultrasound diagnostic apparatus. [Diagram 2] FIG. 4 is a block diagram showing an example of functions of a reception signal processing block. [Diagram 3] FIG. 4 is a flowchart showing a super-resolution image generation process according to the first embodiment. [Figure 4] 5A to 5C are diagrams for explaining frame setting for generating a blood flow image in the first embodiment. [Diagram 5] 5A to 5C are views for explaining generation of a super-resolution image in the first embodiment. [Figure 6] 6A to 6C are diagrams for explaining correction of a deformation amount involving nonlinear deformation in a blood flow image. [Figure 7] 5A and 5B are diagrams for explaining selection of a reference frame when calculating a displacement amount. [Figure 8] FIG. 1 is a diagram illustrating a Rayleigh distribution. [Figure 9] FIG. 11 is a flowchart showing a super-resolution image generation process according to the second embodiment. [Figure 10] 13A to 13C are diagrams for explaining frame setting for generating a blood flow image according to the second embodiment. [Figure 11] FIG. 11 is a flowchart showing a super-resolution image generating process according to the third embodiment. [Figure 12]13A to 13C are diagrams for explaining frame setting for generating a blood flow image according to the third embodiment. [Figure 13] FIG. 13 is a flowchart showing a super-resolution image generating process according to the fourth embodiment. [Figure 14] 13A to 13C are diagrams for explaining frame setting for generating a blood flow image according to the fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] As mentioned above, in conventional super-resolution technology, the deviation of blood vessel position between frames is corrected by aligning ultrasound images with each other using speckle tracking. Ultrasound images contain information derived from tissues (hereinafter also simply referred to as "tissue components") and information derived from blood flow (hereinafter also simply referred to as "blood flow components"). Speckle tracking focuses on the similarity of speckles appearing in the tissue components, and estimates the correspondence and displacement of tissues between frames by comparing and tracking speckle patterns between frames. This method can be expected to achieve a certain degree of estimation accuracy when characteristic structures are captured in the ultrasound images, such as the myocardium in echocardiograms. However, as mentioned above, the estimation accuracy decreases in ultrasound images that do not contain many characteristic structures or in ultrasound images where speckles are dominant.

[0014] Therefore, the inventors came up with a new approach of aligning blood vessels using image features of blood flow components, rather than using image features of tissue components in ultrasound images. By using image features of the blood vessels (blood flow) themselves, which are the object of alignment, it is expected that higher accuracy can be obtained than in the conventional method (method of using image features of tissue components). However, in general ultrasound images, image features of tissue components are overwhelmingly more numerous than those of blood flow components, so even if this approach is simply applied to ultrasound images, it is difficult to accurately capture the deformation amount of the blood vessel region for each frame.

[0015] In order to solve such technical problems, the ultrasonic diagnostic apparatus according to this embodiment employs the following procedure.

[0016] First, the first acquisition unit acquires measurement data including information derived from tissue (tissue components) and information derived from blood flow (blood flow components) based on an ultrasonic signal reflected within a living body. The "measurement data" may be, for example, a received signal obtained by transmitting and receiving an ultrasonic signal, or data obtained by performing phasing and summation and quadrature detection processing on the received signal. Note that the information derived from blood flow may include not only information derived from blood, but also information derived from a contrast agent in blood. A plurality of frames of time-series measurement data are acquired at a predetermined frame rate.

[0017] Next, the second acquisition unit acquires the amount of deformation between multiple frames based on blood flow data in which information derived from blood flow of the measurement data is extracted or emphasized. "Extracting information derived from blood flow" is, for example, an operation of selectively extracting image features of blood flow components from the measurement data. "Emphasizing information derived from blood flow" is, for example, an operation of making image features of blood flow components stand out relatively more than image features of tissue components. The second acquisition unit may acquire blood flow data by performing a process of extracting or emphasizing information derived from blood flow, or conversely, may acquire blood flow data by performing a process of removing or reducing information derived from tissue. For example, the process of the second acquisition unit may include a process of calculating information such as the velocity, variance value, and power value of blood flow by the Doppler method (also called "blood flow information"). The "amount of deformation" is an index that indicates a relative displacement or deformation with respect to a reference position or shape. The second acquisition unit may acquire the amount of deformation based on a blood vessel region (for example, the position or shape of the blood vessel region) in the blood flow data. The method of calculating the amount of deformation and the method of holding the parameters are arbitrary, and may be appropriately designed to match the algorithm of the subsequent correction process (processing for aligning the positions of blood vessel regions between frames).

[0018] Next, the generating unit generates display image data by aligning the feature points in the information derived from blood flow based on the amount of deformation and synthesizing the information for multiple frames. The "information derived from blood flow" may be extracted from the blood flow data acquired by the second acquiring unit, or may be extracted from the measurement data or data generated from the blood flow data. The "feature points" may be parts in the information derived from blood flow where the features of blood flow are particularly strongly expressed (i.e., parts that are highly likely to indicate the location of blood flow). For example, if feature points are extracted from an image having information derived from blood flow (such as the speed, variance, and power value of blood flow) as pixel values, pixels having the maximum pixel value in the image or pixels having pixel values ​​equal to or greater than a predetermined threshold value may be extracted as feature points. The range of pixel values ​​(extraction range) from which feature points are extracted can be set arbitrarily. "Synthesis" is an operation of combining information on multiple feature points extracted from each of multiple frames to generate one blood flow image, and may be, for example, a process of accumulating (accumulating; plotting) multiple feature points on one image. The "display image data" is data that visualizes high-resolution blood flow information and is displayed as a processing result, and is also called an ultra-high-resolution blood flow image.

[0019] According to the above-mentioned procedure, the deformation amount of the blood vessel region for each frame can be accurately captured by calculating the deformation amount based on the blood flow data in which the information derived from the blood flow is dominant. Therefore, even in the case of an ultrasound image that does not include many characteristic structures or an ultrasound image in which speckle is dominant, the positioning of the blood vessel region between frames can be performed with high accuracy, and a high-quality and high-resolution blood flow image can be obtained.

[0020] Although the present invention has been described so far as applied to an ultrasonic diagnostic device, the present invention may be applied to modalities (medical information processing devices) other than ultrasonic diagnostic devices. For example, in a medical information processing device that acquires measurement data including tissue-derived information and blood flow-derived information from a living body in the same manner as an ultrasonic image, high-resolution blood flow information can be obtained.

[0021] Next, some specific embodiments of the present invention will be described as examples.

[0022] <First embodiment> 1 is a block diagram showing an example of a hardware configuration of an ultrasonic diagnostic apparatus according to this embodiment. The ultrasonic diagnostic apparatus 1 includes an ultrasonic probe 102, a probe connection unit 103, a transmission electric circuit 104, a reception electric circuit 105, a reception signal processing unit 106, an image processing unit 107, a display device 108, and a system control unit 109. The ultrasonic diagnostic apparatus 1 is a system for transmitting a pulsed ultrasonic signal from the ultrasonic probe 102 to a subject 100, receiving the ultrasonic signal reflected within the living body, and generating image information of the inside of the subject 100. Ultrasonic images such as B-mode images and blood flow images obtained by the ultrasonic diagnostic apparatus 1 are used in various clinical examinations.

[0023] The ultrasonic probe 102 is an electronic scanning type probe, and has a plurality of transducers 101 arranged one-dimensionally or two-dimensionally at its tip. The transducer 101 is an electromechanical conversion element that converts between an electric signal (voltage pulse signal) and an ultrasonic wave (acoustic wave). The ultrasonic probe 102 transmits ultrasonic waves from the plurality of transducers 101 to the subject 100, and receives reflected ultrasonic waves from the subject 100 by the plurality of transducers 101. The reflected acoustic waves reflect the difference in acoustic impedance within the subject 100. When the transmitted ultrasonic pulse is reflected by the surface of a moving blood flow or a heart wall, the reflected ultrasonic signal undergoes a frequency shift due to the Doppler effect depending on the velocity component of the moving body in the ultrasonic transmission direction.

[0024] The transmission electric circuit 104 is a transmission unit that outputs a pulse signal (drive signal) to the multiple transducers 101. By applying a pulse signal to the multiple transducers 101 with a time difference, ultrasonic waves with different delay times are transmitted from the multiple transducers 101, and a transmitted ultrasonic beam is formed. The direction and focus of the transmitted ultrasonic beam can be controlled by selectively changing the transducer 101 to which the pulse signal is applied (i.e., the transducer 101 to be driven) or by changing the delay time (application timing) of the pulse signal. By sequentially changing the direction and focus of the transmitted ultrasonic beam, the observation area inside the subject 100 is scanned. In addition, by changing the delay time of the pulse signal, a transmitted ultrasonic beam that is a plane wave (focus is far away) or a diverging wave (focus point is the opposite ultrasonic transmission direction for the multiple transducers 101) may be formed. Alternatively, a transmitted ultrasonic beam may be formed using one transducer or a part of the multiple transducers 101. The transmission electric circuit 104 transmits a pulse signal of a predetermined drive waveform to the transducer 101, causing the transducer 101 to generate a transmitted ultrasonic wave having a predetermined transmission waveform. The receiving electrical circuit 105 is a receiving section that inputs, as a received signal, an electrical signal output from the transducer 101 that has received a reflected ultrasonic wave. The received signal is input to a received signal processing section .

[0025] The operations of the transmitting electric circuit 104 and the receiving electric circuit 105, i.e., the transmission and reception of ultrasonic waves, are controlled by a system control unit 109. The system control unit 109 changes the position at which the voltage signal or the transmitted ultrasonic wave is formed in accordance with the generation of a B-mode image and a blood flow image, which will be described later, for example.

[0026] When generating a B-mode image, the reflected ultrasound reception signal obtained by scanning the observation area is acquired and used for image generation. When generating a blood flow image, ultrasound transmission and reception is performed multiple times on one or multiple scanning lines in the observation area to obtain multiple frames of reflected ultrasound reception signals, which are used for image generation, i.e., extraction of blood flow information. Scanning for generating a blood flow image may be a method of performing multiple transmissions and receptions on one scanning line and then transmission and reception on the next scanning line, or a method of performing transmission and reception once on each scanning line multiple times. In addition, B-mode images and blood flow images may be generated so as to reduce the number of scanning lines, that is, by transmitting plane waves or diverging waves to transmit ultrasound over a wide range of the observation area. In addition, the transmission angle of the plane waves or diverging waves, the range of the observation area to be transmitted, etc. may be changed, transmission and reception may be performed multiple times over a wide range of the observation area, and the received signals may be added and used.

[0027] In this specification, both the analog signal output from the transducer 101 and the digital data sampled (digitally converted) from the analog signal are referred to as the received signal without any particular distinction. However, depending on the context, the received signal may be referred to as the received data or the measured data in order to clearly indicate that it is digital data.

[0028] The received signal processing unit 106 is an image generating unit that generates image data based on the received signal obtained from the ultrasonic probe 102. The image processing unit 107 performs image processing such as brightness adjustment, interpolation, and filter processing on the image data generated by the received signal processing unit 106. The display device 108 is a display unit for displaying image data and various information, and is configured, for example, with a liquid crystal display or an organic EL display. The system control unit 109 is a control unit that generally controls the transmitting electric circuit 104, the receiving electric circuit 105, the received signal processing unit 106, the image processing unit 107, the display device 108, etc.

[0029] (Configuration of the receiving signal processing block) 2 is a block diagram showing an example of functions of the received signal processing unit 106. The received signal processing unit 106 includes a received signal storage unit 200, a delay-and-sum processing unit 201, a signal storage unit 202, a B-mode processing unit 203, a Doppler processing unit 204, a deformation amount calculation unit 205, a deformation correction unit 206, and a super-resolution processing unit 207.

[0030] The received signal storage unit 200 stores the received signal received by the receiving electrical circuit 105. Note that, depending on the device configuration and the type of received signal, the received signal may not be stored in the received signal storage unit 200, but may be stored in a signal storage unit 202 after a phasing and summing processing unit 201, which will be described later. The received signal storage unit 200 may be formed of a block common to the signal storage unit 202, which will be described later, and may store the received signal from the receiving electrical circuit 105 and the received signal after the phasing and summing processing unit 201.

[0031] The delay and sum processing unit 201 performs delay and sum processing and quadrature detection processing on the reception signal obtained by the reception electric circuit 105, and stores the processed reception signal in the signal storage unit 202. The delay and sum processing is a processing to form a reception ultrasonic beam by changing the delay time and weight for each transducer 101 and adding up the reception signals of the multiple transducers 101, and is also called delay and sum (DAS) beamforming. The quadrature detection processing is a processing to convert the reception signal into an in-phase signal (I signal) and a quadrature signal (Q signal) of the baseband. The delay and sum processing and quadrature detection processing are performed based on the element arrangement and various conditions for image generation (aperture control, signal filter) input from the system control unit 109. The reception signal after the delay and sum processing and quadrature detection processing is stored in the signal storage unit 202. Here, a typical example of DAS beamforming is shown, but any processing to form a reception ultrasonic beam, such as adaptive beamforming, model-based processing, and processing using machine learning, may be used.

[0032] The reception signal (digital data) obtained by the receiving electrical circuit 105 is also called RAW data. The reception signal (digital data) obtained by the phasing addition processing unit 201 is also called RF data. In this embodiment, the RAW data and the RF data are examples of measurement data acquired based on an ultrasonic signal reflected within a living body, and the receiving electrical circuit 105 and the phasing addition processing unit 201 are examples of a first acquisition unit that acquires the measurement data.

[0033] The B-mode processing unit 203 performs envelope detection processing, logarithmic compression processing, etc. on the received signals for generating B-mode images stored in the signal storage unit 202, and generates image data in which the signal intensity at each point within the observation area is represented as brightness intensity.

[0034] The processing contents of the Doppler processing unit 204 will be described in detail. The Doppler processing unit 204 performs frequency analysis on the received signals for generating a blood flow image stored in the signal storage unit 202, thereby obtaining a scanning range The blood flow information based on the Doppler effect of an object within the subject is extracted. In this embodiment, an example in which the object is blood will be mainly described, but the object may be an object such as internal tissue or a contrast agent. Examples of blood flow information include at least one of velocity, variance, and power. The Doppler processing unit 204 may obtain blood flow information at one point (one position) within the subject, or may obtain blood flow information at multiple positions in the depth direction. The Doppler processing unit 204 may obtain blood flow information at multiple points in time in a time series so that the time change of the blood flow information can be displayed.

[0035] In generating a blood flow image using the Doppler method, a series of received signal data of a plurality of frames is acquired in the time direction for the same measurement position. The Doppler processing unit 204 applies an MTI (Moving Target Indicator) filter to the received signal data series. This reduces information (clutter components) derived from tissues that are stationary between frames or tissues with little movement, and extracts information derived from blood flow (blood flow components). The Doppler processing unit 204 then calculates blood flow information such as blood flow velocity, blood flow dispersion, and blood flow power from the blood flow components. Data representing the calculated blood flow information in the form of a two-dimensional image is called a blood flow image.

[0036] The MTI filter is a Butterworth type IIR (infinite impulse Alternatively, a filter with fixed filter coefficients, such as a multi-level time response filter or a polynomial regression filter, may be used. The MTI filter may be an adaptive filter that changes coefficients according to an input signal using eigenvalue decomposition or singular value decomposition. Alternatively, the Doppler processing unit 204 may decompose the received signal into one or more bases using eigenvalue decomposition or singular value decomposition, extract only a specific base, and remove clutter components. The Doppler processing unit 204 may also use a method such as a vector Doppler method, a speckle tracking method, or a vector flow mapping method to obtain a velocity vector for each coordinate in the image and obtain a blood flow vector representing the magnitude and direction of the blood flow. In addition to the methods exemplified here, any method may be used as long as it can extract or emphasize information derived from blood flow contained in the received signal (measurement data) or remove or reduce information derived from tissue.

[0037] The deformation amount calculation unit 205 calculates the difference in position and shape of the vascular region between frames (deformation amount of the vascular region) using the blood flow images of multiple frames output by the Doppler processing unit 204. In this embodiment, the blood flow images are an example of blood flow data in which information derived from blood flow is extracted or emphasized, and the Doppler processing unit 204 and the deformation amount calculation unit 205 are an example of a second acquisition unit that acquires the deformation amount between multiple frames based on the blood flow images (blood flow data).

[0038] Prior to the process of calculating the deformation amount, the deformation amount calculation unit 205 may perform preprocessing on the blood flow image to emphasize or clarify the blood vessel structure in the blood flow image. As the preprocessing, for example, a filter that extracts or emphasizes blood vessel-like structures (e.g., columnar, linear, mesh-like, etc.) contained in the image may be applied to the blood flow image. Conversely, the blood vessel structure may be relatively emphasized by applying a filter that removes or reduces the background (tissue parts other than the blood vessel structure) in the image or noise contained in the background to the blood flow image. For example, the deformation amount calculation unit 205 may perform multi-resolution decomposition such as pyramid transformation or wavelet transformation on the blood flow image, apply an appropriate filter to each image, and then perform an inverse transformation of the multi-resolution decomposition. This allows structures different from the background to be clearly extracted.

[0039] The deformation amount calculation unit 205 may also apply line enhancement processing to the blood flow image using eigenvalue analysis of a Hessian matrix to enhance linear structures (i.e., blood vessels) in the image. The deformation amount calculation unit 205 may also extract only the blood vessel structure from the blood flow image using image extraction processing such as template matching using a template of the blood vessel structure.

[0040] The method for calculating the amount of deformation may be rigid registration or non-rigid registration. Alternatively, the amount of deformation may be calculated using a trained model based on deep learning or the like. In addition, the method is not limited to these, and any method may be used as long as it can calculate the amount of deformation for correcting the position or shape of the blood flow image that has changed between frames due to body movement or the like. The user may be allowed to select a desired method from a plurality of calculation methods prepared in advance.

[0041] Rigid body registration is a method of performing registration using a transformation function that includes only elements of translation and rotation in an image, assuming that the object (blood vessel in this embodiment) is a rigid body, and has the advantage of low calculation cost and high speed processing. In the case of rigid body registration, one deformation amount may be calculated for the image (i.e., the entire image may be translated and rotated), or the image may be divided into multiple regions and the deformation amount may be calculated for each region. In the latter case, since the amount of movement or rotation may differ for each region, the boundary portion is smoothly connected so that artifacts do not occur at the boundary between adjacent regions. For example, a predetermined width of overlap may be created at the boundary portion, and the pixel values ​​of the two regions may be averaged or blended at the overlap portion. The blending ratio may be gradually changed between the two regions. In addition, the deformation amount may be gradually changed by interpolating the deformation amounts of the two regions at the overlap portion.

[0042] Non-rigid registration includes a method of assuming that the object (blood vessel in this embodiment) is non-rigid and using affine transformation such as scaling and shearing, and a method of deforming an image with a deformation element such as local non-linear deformation. Non-rigid registration is also called DIR (deformable image registration). Note that in biological images such as blood flow images, movement is often accompanied by non-linear deformation, so it is desirable to use non-rigid registration. Examples of algorithms used for non-rigid registration include the FFD (free form deformation) method and the Demons algorithm. The FFD method is a method of deforming an image using a B-spline function by setting control lattice points (sampling points) for an image to be deformed and determining a control point matrix from a target movement amount. The Demons algorithm is a method of generating a deformation vector based on a signal difference or gradient at the same position of two images.

[0043] When using deep learning such as deep learning, it is advisable to use a trained model that has been trained to, for example, input a blood flow image and output the amount of deformation in that blood flow image. In this case, the reference data (the position and shape of the reference blood vessel) may be fixed (i.e., pre-trained in the trained model), or the reference image may be given as an input together with the blood flow image to be processed. When machine learning a model, supervised learning may be performed using a set of the reference image, the blood flow image after deformation, and the amount of deformation (true value) as training data.

[0044] Alternatively, a trained model may be used that is trained to output a corrected image in which the deformation of blood vessels is corrected when a blood flow image is input. In this case, the trained model serves both the functions of the deformation amount calculation unit 205 and the deformation correction unit 206. In the case of such a trained model, the reference data (position and shape of the reference blood vessel) may be fixed (i.e., learned in advance in the trained model), or the reference image may be given as an input together with the blood flow image to be processed. When machine learning the model, supervised learning may be performed using a set of the reference image (true value) and the pre-correction image as training data.

[0045] The deformation correction unit 206 corrects the position and shape of the blood vessel in the image based on the deformation amount output by the deformation amount calculation unit 205, that is, aligns the blood vessel between frames. The deformation correction unit 206 may apply correction to the received signal (measurement data) stored in the received signal storage unit 200 or the signal storage unit 202, or may apply correction to the blood flow image (blood flow data) obtained by the Doppler processing unit 204. 2. Alternatively, the deformation correction unit 206 may apply correction to secondary data generated from the received signal or blood flow image (for example, an image obtained by averaging blood flow images of multiple frames). In this embodiment, after the image is corrected by the deformation correction unit 206, the corrected image is passed to the super-resolution processing unit 207 in the subsequent stage. However, instead of explicitly performing the deformation correction, the positions of the feature points may be corrected at the same time as the accumulation processing in the super-resolution processing unit 207. In the latter case, the super-resolution processing unit 207 also serves as the deformation correction unit 206, so the deformation correction unit 206 in FIG. 2 may not be provided.

[0046] The super-resolution processor 207 generates super-resolution blood flow image data, which is a blood flow image with improved resolution, from multiple frames of blood flow image data that have been aligned (blood vessel deformation corrected) by the deformation corrector 206. A method for acquiring a super-resolution blood flow image will be described when explaining the processing flow. In this embodiment, the deformation corrector 206 and the super-resolution processor 207 are an example of a generator that generates display image data.

[0047] The image data output from the B-mode processing unit 203, the Doppler processing unit 204, and the super-resolution processing unit 207 are subjected to processing by the image processing unit 107 shown in Fig. 1, and then finally displayed on the display device 108. The respective image data may be displayed in a superimposed manner, may be displayed in parallel, or only a portion of the image data may be displayed.

[0048] The received signal processing unit 106 may be configured with one or more processors and memories. In this case, the functions of the units 201 to 207 shown in FIG. 2 are realized by a computer program. For example, the functions of the units 201 to 207 can be provided by the CPU reading and executing a program stored in the memory. In addition to the CPU, the received signal processing unit 106 may also include a processor (GPU, FPGA, etc.) that handles the calculations of the B-mode processing unit 203, the Doppler processing unit 204, the deformation amount calculation unit 205, the deformation correction unit 206, and the super-resolution processing unit 207. The memory may include a memory for non-temporarily storing a program, a memory for temporarily storing data such as a received signal, a working memory used by the CPU, etc.

[0049] The above has described the overall configuration of the ultrasound diagnostic apparatus 1 according to the first embodiment. Next, a processing flow for acquiring a super-resolution blood flow image will be described.

[0050] (Processing flow for obtaining super-resolution blood flow images) A processing flow for acquiring a super-resolution blood flow image in the first embodiment will be described with reference to FIG.

[0051] A measurement position is set so as to include a region in which blood flow information is to be observed, and ultrasonic transmission and reception are repeated while fixed at the same measurement position to obtain a reception signal including a data sequence of multiple frames in the time direction (S310). The number of frames of the reception signal when generating a super-resolution blood flow image according to this embodiment is generally greater than that when generating a normal Doppler image. For example, the number of frames (number of packets) of the reception signal when generating a normal Doppler image is about 5 to 20, whereas the number of frames of the reception signal when generating a super-resolution blood flow image is at least 100 frames or more, preferably about several hundred to tens of thousands. The frame rate of the ultrasound diagnostic device 1 is about several hundred Hz to several thousand Hz, depending on the model and measurement method. Therefore, the measurement time of the reception signal to generate a super-resolution blood flow image is generally about several hundred msec to several tens of sec. The reception signal including the data sequence of multiple frames is subjected to a delay-and-sum processing unit 201 to a delay-and-sum processing and a quadrature detection processing, and is stored in the signal storage unit 202. In addition, one frame in the data sequence may be a received signal obtained by transmitting and receiving ultrasound once to include the observation area, or may be a sum of received signals transmitted and received multiple times to include the observation area. For example, a plane wave or diverging wave may be transmitted to include the observation area, and multiple signals may be transmitted at different transmission angles. The received signals are added together to create one frame of data.

[0052] The Doppler processing unit 204 applies an MTI filter to the received signal including a data sequence of multiple frames to reduce clutter components and extract components derived from blood flow to generate blood flow images of multiple frames (S320). A method of setting frames when generating blood flow images is shown in FIG. 4. By extracting a part of the data sequence of the received signal to design an MTI filter, blood flow images (before averaging) in the same number as the number of extracted frames (number of packets) can be obtained. Averaging these (additive averaging, square root sum of squares, etc.) produces a blood flow image (after averaging). In a normal Doppler, this blood flow image (after averaging) is displayed. Further blood flow images can be obtained by changing the frames to be extracted in the time direction. At this time, the frames to be extracted may be set to overlap or not overlap. An example of the obtained blood flow image is shown in FIG. 5A. It is preferable that the parameters of the MTI filter (cutoff frequency, number of packets / number of overlaps of the frames to be extracted, etc.) can be adjusted automatically or by the user according to the nature of the received signal, the subject to be measured, etc.

[0053] The deformation amount calculation unit 205 calculates the amount of deformation of the blood vessel region caused by body movement between frames from the blood flow image (S330). The blood flow image (after averaging) or the blood flow image (before averaging) may be used to calculate the amount of deformation. Next, the deformation correction unit 206 corrects the corresponding blood flow image based on the calculated amount of deformation so that the blood flow (blood vessel) positions between frames that undergo integration processing in S360 (described later) are aligned (S340).

[0054] An example of correction involving nonlinear deformation in a blood flow image will be described with reference to Figs. 6A to 6C. Figs. 6A to 6C show blood flow images in time series, each of which is a blood flow image at a different frame (time point). The dashed lines in Figs. 6B and 6C show the blood flow (blood vessel) position in Fig. 6A. The blood flow (blood vessel) position is not only translated or rotated but also has a local nonlinear deformation. When such a nonlinear deformation occurs, it is advisable to apply the above-mentioned non-rigid registration or correction using a learned model. For example, the deformation amount calculation unit 205 selects a reference image (reference frame) from the blood flow images in time series, and calculates the relative displacement or deformation with respect to the position or shape of the blood vessel region in the reference frame for the blood flow images of other frames. In the illustrated example, Fig. 6A is set as the reference frame, and the deformation amount is calculated for each of the frames in Figs. 6B and 6C. When a local nonlinear deformation occurs, a local deformation amount is calculated for each of a plurality of coordinates in the blood flow image. Then, the deformation correction unit 206 applies non-rigid registration correction to the image of each frame so as to cancel the amount of deformation, so that the position and shape of the vascular region in the blood flow images of Figures 6B and 6C coincide with the vascular region in the blood flow image of Figure 6A.

[0055] The reference frame for calculating the deformation amount may be only one frame in the entire data sequence, or the reference frame may be changed depending on the time direction position (i.e., multiple reference frames may be provided), or the previous frame in the adjacent frames may be used as the reference frame. The reference frame may be selected arbitrarily from the data sequence, and may be, for example, any of the first frame, the middle frame, and the last frame. However, it is preferable to select, as the reference frame, the frame that contains the most information derived from blood flow, for example, the frame in which the most blood vessels (blood flow) are visualized, among the multiple frames that make up the data sequence. By selecting the reference frame in this way, it becomes possible to align more blood flows (blood vessels) in the image, and the accuracy of the alignment is also improved. Figures 7A to 7B are examples of blood flow images, and while the blood flow branched into two is clearly visualized in Figure 7A, there is a lot of noise and the entire blood flow is unclear, and in Figure 7C, the thin blood flow is unclear. In the case of such a data sequence, the image of Figure 7A, which has the best quality, may be selected as the reference frame. In addition, as an index for evaluating whether many blood flows (blood vessels) are visualized, for example, the SNR (signal-to-noise ratio) or CNR (contrast-to-noise ratio) of the image may be used. Alternatively, the average brightness value of the image, a histogram, the number of pixels exceeding a predetermined threshold, the number of linear structures, or the like may be used as the evaluation index.

[0056] The blood flow image used in the calculation of the deformation amount in S330 may be either before or after averaging, but the blood flow image after averaging has a better SNR of the blood flow structure and the accuracy of the deformation amount calculation is higher. The blood flow image to which the correction used in S340 is applied may also be either before or after averaging, but the blood flow image before averaging has a larger pixel value difference in the blood flow, making it easier to extract the local maximum value of the blood flow image in S350 described later. When calculating the deformation amount using the blood flow image after averaging and correcting the blood flow image before averaging, the blood flow image (before averaging) may be a part or all of the blood flow image in the packet, and the frame used in the accumulation process in S360 described later is corrected.

[0057] The super-resolution processor 207 extracts feature points in the information derived from blood flow from each corrected blood flow image (S350). The blood flow image of this embodiment is an image in which information derived from blood flow is expressed by pixel values ​​(brightness; density), and pixels with more (stronger) information derived from blood flow (i.e., pixels corresponding to locations where blood flow exists) have a larger pixel value. Therefore, the super-resolution processor 207 extracts points (pixels) with large pixel values ​​as feature points from the blood flow image. By the operation of S350, an image is obtained in which only points where information derived from blood flow is particularly strongly expressed are extracted from the blood flow image, as shown in FIG. 5B.

[0058] The upper and lower limits of the range of pixel values ​​for extracting feature points (hereinafter referred to as the "predetermined range" or "extraction range") may be fixed values, or may be dynamically set according to the pixel value distribution (histogram) or maximum pixel value of the blood flow image. For example, the upper limit may be the maximum pixel value of the blood flow image, and the lower limit may be a value obtained by multiplying the maximum pixel value by a predetermined coefficient (e.g., 0.8). In this case, if the coefficient is set to 1.0, only the pixel having the maximum pixel value in the blood flow image is extracted, and the coefficient can be adjusted so that more pixels are extracted as the coefficient is made smaller. Alternatively, the extraction range may be set to include the top N pixels of the histogram (N is an integer that is set arbitrarily).

[0059] The super-resolution processor 207 may extract a pixel that is a local maximum value as a feature point. For example, the blood flow image is divided into a plurality of local regions, and feature points are extracted for each local region. In this case, the super-resolution processor 207 may fix the extraction range of each local region, or may set the extraction range for each local region according to the local maximum pixel value. In addition, the local maximum value may be extracted by extracting a pixel that has a larger pixel value than adjacent pixels (for example, when the pixel value of the central pixel is the maximum in a local region of 3 pixels x 3 pixels), i.e., a pixel that is a maximum. Here, an example of a representative method for extracting a local maximum value is shown, but any process that extracts a local maximum value may be used. By setting an extraction range for each local region, more pixels can be extracted as feature points than when a uniform extraction range is set for the entire image, and the number of pixels used in the accumulation process in S360 described later can be increased. Note that when an extraction range is set for each local region, there is a possibility that not only pixels in the blood flow but also pixels in the tissue (i.e., noise) are extracted. However, since the pixels (noise) in tissues tend to have relatively small pixel values ​​compared to the pixels in blood flow, the influence of the pixels in tissues is reduced by accumulating the pixel values ​​of the pixel groups extracted in S360 (described later). Therefore, in most cases, a super-resolution image with an SNR equal to or higher than that of the original blood flow image is ultimately obtained, and this does not pose any particular problem.

[0060] Alternatively, before performing the accumulation process of S360, the super-resolution processor 207 may perform a process of actively removing pixels in tissue (noise) from the group of pixels extracted in S350. Specifically, a threshold may be set for the pixel value distribution (histogram) of the group of pixels extracted in S350, and pixels with pixel values ​​smaller than the threshold may be regarded as noise and removed. The threshold may be set by the user. For example, the threshold may be input or selected on a setting screen displayed on the display device 108, or may be specified by a GUI such as a slider bar. In this case, the pixel value distribution (histogram) of the group of pixels extracted in S350 may be set to 100%. may be displayed on the setting screen, and auxiliary information such as the position of Nσ in the pixel value distribution and the position of the valley may also be displayed. Such a display can assist the user in selecting the threshold. Alternatively, the super-resolution processor 207 may automatically set the threshold. In this case, a fixed threshold may be used, or the threshold may be dynamically determined based on the variance, bimodality, outliers, etc. of the pixel value distribution. Alternatively, the bottom M pixels (M is an integer that is set arbitrarily) with the smallest pixel value from the extracted pixel group may be mechanically removed as noise.

[0061] In addition, the pixel value of the blood flow image varies depending on the depth from the body surface due to attenuation by the living body and diffusion attenuation of ultrasonic waves. Therefore, in the process of S350, the super-resolution processing unit 207 may set the above-mentioned extraction range according to the "depth from the body surface" of each local region in the blood flow image. In other words, when a first local region and a second local region having different depths from the body surface are included in the blood flow image, the extraction range of the first local region and the extraction range of the second local region are made different in consideration of the depth from the body surface. For example, when the second local region is deeper from the body surface than the first local region and the pixel value of the second local region is smaller due to the influence of ultrasound attenuation, it is advisable to make the extraction range of the second local region wider than that of the first local region (for example, lower the lower limit). Alternatively, instead of changing the extraction range, a process for correcting the living body attenuation or diffusion attenuation may be applied to the blood flow image before the process for extracting feature points.

[0062] The super-resolution processor 207 may accumulate blood flow images of multiple frames to generate a blood flow image with improved SNR, and then extract feature points from the accumulated blood flow image in S350. The super-resolution processor 207 may not use blood flow images of all frames in the feature point extraction process in S350 or the accumulation process in S360. For example, frames with poor SNR may be excluded from the processing targets of S350 and S360.

[0063] In addition, the super-resolution processing unit 207 may apply the feature point extraction process of S350 after performing a process of increasing the resolution (up-conversion) of the blood flow image. At this time, the blood flow image itself may be up-converted, or the original received signal may be up-converted and the blood flow image may be generated using the received signal of multiple frames after up-conversion. The up-conversion method is arbitrary, and for example, an interpolated pixel may be generated by nearest neighbor, linear interpolation, spline interpolation, or the like. Alternatively, the up-conversion may be performed by applying a filter generated by machine learning. Since the resolution (number of pixels) of the super-resolution blood flow image obtained by the accumulation process of S360 is the same as the resolution (number of pixels) of the original image used to extract the feature points, when the resolution of the original image is low, the effect of improving the resolution (resolution) by the super-resolution process is small. Therefore, by increasing the resolution (number of pixels) of the original image in advance, the resolution of the final super-resolution blood flow image can be increased. In addition, in images obtained by a convex probe or a sector probe, the pixel size (resolution) differs between the deep and shallow regions in the image. By increasing the resolution of shallow regions in advance through up-conversion, it is expected that the resolution of the final super-resolution blood flow image will also be improved.

[0064] In the feature point extraction by S350, only the parts with high pixel values ​​are extracted. If we assume that the point spread function is a Gaussian distribution, this means that only the sharpest point (peak) at the center of the distribution is extracted. It is generally known that the amplitude distribution characteristics of ultrasonic reception signals from a uniform scattering medium can be approximated by a probability distribution called Rayleigh distribution shown in Figure 8. Extracting the parts with high pixel values ​​in S350 means extracting only the reflection signals from a small number of red blood cells and making them spatially sparse by extracting parts with a small probability in the Rayleigh distribution but large amplitude values ​​from the reflection signals from a large number of red blood cells. Only points or pixels in a small range are extracted from one frame of image, but since the parts with high pixel values ​​move between frames, by extracting feature points from each of many frames, it is possible to obtain comprehensive feature points from almost the entire blood flow range.

[0065] Finally, the super-resolution processing unit 207 generates a super-resolution blood flow image by accumulating pixel values ​​of feature points extracted from each of the multiple frames (S360). Since pixels with large pixel values ​​are concentrated in the blood flow range by the accumulation process, the blood flow is visualized (imaged) with higher resolution and clarity in the super-resolution blood flow image compared to the original blood flow image. The super-resolution processing unit 207 displays the generated super-resolution blood flow image on the display device 108.

[0066] All the frames stored in the signal storage unit 202 may be used for feature point extraction and integration processing to generate one super-resolution blood flow image, or only frames in a certain time range may be used to generate a super-resolution blood flow image. In the latter case, the user may select the time range used to generate the super-resolution blood flow image. Also, multiple super-resolution blood flow images in time series may be generated while shifting the time range used to generate the super-resolution blood flow image in sequence. Furthermore, multiple super-resolution blood flow images may be displayed in sequence on the display device 108 to display a moving image of the super-resolution blood flow image. Alternatively, multiple super-resolution blood flow images may be displayed side by side on the display device 108 to allow comparison of changes over time.

[0067] According to the configuration of this embodiment described above, even if there is movement or deformation between frames, the blood flow position can be accurately aligned, thereby making it possible to generate a high-resolution, high-quality super-resolution blood flow image.

[0068] <Second embodiment> In the first embodiment, feature points are extracted from a blood flow image after deformation correction. In the second embodiment, a plurality of feature separation images having different image features are generated from the blood flow image after deformation correction, and feature points are extracted from each of the feature separation images.

[0069] Consider a feature space formed by all image features contained in a blood flow image. For example, by separating a partial space from the feature space and reconstructing an image using only the image features contained in that partial space, an image in which only a partial image feature of the original blood flow image is separated (called a feature separated image) can be generated. In this case, by varying the image feature of interest (i.e., the partial space separated from the feature space), multiple feature separated images with different image features can be generated from one blood flow image. Note that the partial spaces of each feature separated image may be completely separated, or the partial spaces may overlap.

[0070] The device configuration may be the same as that of the first embodiment. A process flow for acquiring a super-resolution blood flow image according to the second embodiment will be described with reference to Fig. 9. The processes from S710 to S740 are the same as those from S310 to S340 in the first embodiment. The super-resolution processor 207 generates a plurality of feature separated images having different image features from each of the deformation-corrected blood flow images (S750).

[0071] A frame setting method for generating multiple feature separation images from a blood flow image that has been subjected to deformation correction is shown in Fig. 10. When the super-resolution processor 207 acquires blood flow images after deformation correction, it generates multiple feature separation images having different image features of interest from each blood flow image. Fig. 10 shows a case where deformation correction is performed on blood flow images before averaging, but deformation correction may also be performed on blood flow images after averaging as in the first embodiment, in which case multiple feature separation images can be generated from the blood flow images after deformation correction (after averaging).

[0072] Here, the effect of dividing a blood flow image into multiple feature separation images will be described. In general, in a blood flow image, when comparing "thick blood vessels with fast flow speeds" and "thin blood vessels with slow flow speeds," the pixel value of the thick blood vessels with fast flow speeds is larger. Therefore, when a pixel group having a pixel value within a predetermined range is extracted from a blood flow image in which a thick blood vessel with fast flow speeds and a thin blood vessel with slow flow speeds are adjacent to each other, There is a possibility that pixels of slow and thin blood vessels will not be extracted. It is possible to widen the extraction range of pixel values ​​to be extracted as feature points so that blood vessels with small pixel values, i.e. blood vessels with slow flow speeds, can also be extracted. However, when considering extraction for the above-mentioned Gaussian distribution, the range of the distribution to be extracted is widened, so the effect of improving the resolution is reduced. To solve this problem, it is advisable to generate a feature separation image using image features related to the blood flow speed. That is, from the original blood flow image, multiple feature separation images with different flow speed ranges are generated, such as a first feature separation image with image features corresponding to a first flow speed range, a second feature separation image with image features corresponding to a second flow speed range, etc. Then, an appropriate extraction range is set for each feature separation image, and feature points are extracted from each feature separation image. With this method, pixels of thick blood vessels with fast flow speeds and pixels of thin blood vessels with slow flow speeds can be extracted as feature points.

[0073] Specifically, a discrete Fourier transform is performed on the change in pixel value of each pixel in the time direction using some frames of blood flow images of multiple frames. The discrete Fourier transform is a sine wave basis and represents frequency components. Therefore, the frequency components of the change between frames of each pixel of the blood flow image, i.e., flow velocity components, can be separated by the discrete Fourier transform, and a feature separation image having blood flow information in the flow velocity range corresponding to each basis can be generated. The feature separation image having blood flow information in the flow velocity range corresponding to each basis may be a frequency domain image after the discrete Fourier transform, or a time domain image obtained by performing an inverse discrete Fourier transform on the frequency domain image. By changing some frames used for the discrete Fourier transform among the blood flow images of multiple frames in the time direction, a feature separation image with a different time series corresponding to each basis can be obtained. In this case, the blood flow image is separated into multiple feature separation images by focusing on image features related to flow velocity (frequency components of the change in pixel value over time), i.e., by using the flow velocity as a basis. By increasing the number of separations, i.e., the number of frames used for the discrete Fourier transform, the flow velocity range corresponding to each base becomes smaller, so that the blood flow image can be separated for each finer flow velocity range. Note that any base other than the discrete Fourier transform based on a sine wave may be used as long as it can be separated for each flow velocity range, and the blood flow image may be separated into images for each flow velocity range by polynomial regression such as the Legendre polynomial. In addition, when the energy level of the pixel values ​​of a certain blood vessel (for example, the standard deviation of the pixel values) is close to the energy of noise, the noise energy is dispersed to each base by generating a feature separation image separated for each flow velocity by the processing of S750. This reduces the noise energy compared to the image information of the blood flow component, so that the SNR (signal-to-noise ratio) or CNR (contrast-to-noise ratio) of each feature separation image can be improved. Therefore, there is also the advantage that blood vessels that could not be distinguished from noise in the blood flow image can be depicted in the feature separation image. Furthermore, more feature points can be extracted by the processing of S760 described later, and more feature points can be extracted with the same number of frames.This improves the quality of the final super-resolution blood flow image, and since a super-resolution blood flow image of equivalent quality can be generated from fewer frames than in the first embodiment, the measurement time required to generate a super-resolution blood flow image can be shortened.

[0074] Image features are not limited to those related to flow velocity. For example, a feature separation image may be generated by focusing on image features related to the shape or thickness of blood vessels. That is, the blood vessel diameter extracted differs in each basis. For example, the spatial frequency of each blood flow image is calculated using a two-dimensional Fourier transform. A high spatial frequency, i.e., a spatially fine structure, is an image feature of a thin blood vessel, and a low spatial frequency, i.e., a spatially coarse structure, is an image feature of a thick blood vessel. Therefore, for example, a feature separation image having image features corresponding to a desired blood vessel diameter can be generated by extracting only the spatial frequency components corresponding to a desired blood vessel diameter from the two-dimensional Fourier transform of the blood flow image and performing an inverse Fourier transform. Then, by appropriately changing the spatial frequency components to be extracted, a plurality of feature separation images corresponding to different blood vessel diameters can be obtained, such as a feature separation image corresponding to a blood vessel diameter of 1 mm to 2 mm, a feature separation image corresponding to a blood vessel diameter of 2 mm to 3 mm, etc. Also, by applying line enhancement processing based on eigenvalue analysis of the Hessian matrix to the blood flow image, a blood vessel shape (linear, columnar) of a desired thickness can be generated. A feature separation image in which image features corresponding to a desired blood vessel diameter are emphasized may be generated by selectively emphasizing the brightness or contrast of the structure. In this case, a plurality of feature separation images corresponding to different blood vessel diameters can be generated by appropriately changing the thickness to be emphasized by the line emphasis process. A similarity search process such as template matching may also be used. For example, a similarity map (a two-dimensional map representing the similarity with the template image for each local region) is generated by searching the entire blood flow image using a template image of a blood vessel pattern of a desired thickness. This similarity map can be regarded as a two-dimensional image filter in which a larger coefficient is set for pixels corresponding to the blood vessel diameter of the template image. Therefore, by multiplying the blood flow image by the similarity map, a feature separation image in which image features corresponding to the blood vessel diameter of the template image are emphasized can be obtained. In this case, a plurality of template images with different blood vessel diameters can be prepared to generate a plurality of feature separation images corresponding to different blood vessel diameters. A feature separation image may be generated by using any method other than those listed here as long as it is a method for selectively emphasizing or extracting image features corresponding to the thickness or shape of a blood vessel.

[0075] A feature separation image may be generated based on two image features, namely, "temporal change and spatial structure", i.e., "flow velocity and shape". For example, a discrete Fourier transform (three-dimensional discrete Fourier transform) is performed on some frames of a blood flow image of a plurality of frames in both the time direction of the pixel value of each pixel and the spatial direction within the image. Then, by extracting only the spatial frequency components corresponding to the desired flow velocity range and the desired blood vessel diameter and performing an inverse Fourier transform, a feature separation image having image features corresponding to the desired flow velocity range and the desired blood vessel diameter can be generated. By appropriately changing the spatial frequency components to be extracted, a plurality of feature separation images having different combinations of flow velocity range and blood vessel diameter can be obtained. Alternatively, it can be realized by converting each frame of some frames of a blood flow image of a plurality of frames into a one-dimensional vector, and applying singular value decomposition to the matrix data in which each frame is arranged in the column direction. This allows blood vessels of different sizes in space to be further divided into images with different flow velocity ranges.

[0076] A feature separation image may be generated by focusing on image features related to the direction of blood flow. In addition to blood flow images obtained by the Doppler method, a velocity vector for each coordinate in the image is obtained by using methods such as a vector Doppler method, a speckle tracking method, and a vector flow mapping method. The direction of this velocity vector represents the direction of blood flow. To generate a feature separation image corresponding to a desired blood flow direction, for example, a coincidence map is generated that indicates the degree of coincidence between the desired blood flow direction and the velocity vector for each coordinate. This coincidence map can be regarded as a two-dimensional image filter in which a larger coefficient is set for a coordinate having a velocity vector closer to the desired blood flow direction. Therefore, by multiplying the coincidence map by the blood flow image, a feature separation image in which image features corresponding to the desired blood flow direction are emphasized can be obtained. By generating a coincidence map for each blood flow direction and multiplying it by the blood flow image, multiple feature separation images corresponding to different blood flow directions can be generated.

[0077] The super-resolution processor 207 extracts feature points from each of the multiple feature separation images generated in S750 (S760). In the first embodiment, the target was a blood flow image that has been subjected to deformation correction, whereas in the second embodiment, the target is a feature separation image generated from a blood flow image. The method of determining the extraction range of the feature points may be the same as in the first embodiment, but an appropriate extraction range is determined for each feature separation image.

[0078] Finally, the super-resolution processor 207 generates a super-resolution blood flow image by integrating pixel values ​​of feature points extracted from each of the multiple feature separated images (S760). The super-resolution processor 207 displays the generated super-resolution blood flow image on the display device 108.

[0079] All the feature separation images generated from the blood flow images of all the frames stored in the signal storage unit 202 may be used for feature point extraction and integration processing to generate one super-resolution blood flow image, or only frames in a part of the time range may be used to generate the super-resolution blood flow image. In the latter case, the user may select the time range used to generate the super-resolution blood flow image. Also, multiple super-resolution blood flow images in time series may be generated while shifting the time range used to generate the super-resolution blood flow image in sequence. Furthermore, multiple super-resolution blood flow images may be displayed in sequence on the display device 108 to display a moving image of the super-resolution blood flow image. Alternatively, multiple super-resolution blood flow images may be displayed side by side on the display device 108 to allow comparison of changes over time.

[0080] All feature points extracted from a plurality of feature separation images corresponding to different image features may be simply integrated, or only some feature points may be selected and integrated, or weighted according to the image features may be added. For example, by simply integrating all feature points extracted from a plurality of feature separation images with different flow velocity ranges, all blood vessels (blood flows) from blood vessels with slow flow velocities to blood vessels with fast flow velocities can be imaged. By selecting only feature points of a feature separation image corresponding to a certain flow velocity range, or by increasing the weight of the feature points and then integrating, an image in which blood vessels in the certain flow velocity range are emphasized can be obtained. A GUI may be provided to the user for arbitrarily selecting or setting the range of image features (feature separation images) used for integration or the weighting according to the image features. By using such a GUI, a suitable super-resolution blood flow image can be generated according to the object to be observed (blood vessel thickness, shape, flow velocity, direction, etc.).

[0081] <Third embodiment> In the third embodiment, the deformation correction is applied to the received signal instead of the blood flow image. A blood flow image is generated from the received signal after the deformation correction, and the blood flow image is used for the super-resolution processing, thereby generating a high-resolution and high-quality super-resolution blood flow image.

[0082] The device configuration may be the same as that of the first embodiment. A process flow for acquiring a super-resolution blood flow image according to the third embodiment will be described with reference to FIG. 11. The processes from S910 to S930 are the same as those from S310 to S330 of the first embodiment. That is, in the third embodiment, the deformation amount calculation unit 205 calculates the deformation amount of the blood vessel region caused by body movement between frames from the blood flow image (S930). The blood flow image used for calculating the deformation amount may be either before or after averaging, but the blood flow image after averaging has a better SNR of the blood flow structure and the accuracy of the deformation amount calculation is higher. The specific method for calculating the deformation amount is as described in the first embodiment.

[0083] Next, the deformation correction unit 206 corrects the received signal corresponding to the blood flow image based on the deformation amount calculated from the blood flow image in S930 (S940). The specific correction method may be the same as that described in the first embodiment, and any of correction by rigid registration, correction by non-rigid registration, and correction by a learned model may be used.

[0084] Next, the Doppler processing unit 204 removes clutter from the deformation-corrected received signal to generate a plurality of blood flow images (S950). The method of generating the blood flow images may be the same as that described in the first embodiment.

[0085] Fig. 12 shows the frame setting method when generating a blood flow image from the deformation-corrected received signal. The MTI filter parameters (cutoff frequency, number of packets / overlap of extracted frames, etc.) should be adjustable automatically or by the user according to the nature of the received signal, the subject to be measured, etc. The MTI filter parameters may be the same for S920 and S950, or may be changed. In general, the larger the number of packets in the MTI filter (i.e., the better the image quality). Although a longer measurement time makes it possible to observe blood flows with low flow rates, the performance of the MTI filter decreases if the image position shifts between frames in a packet due to effects such as body movement. Therefore, as shown in Figure 12, it is desirable to have a larger number of packets in the MTI filter for S950 than for S920. That is, since the received signal before deformation correction is likely to include effects such as body movement between frames, the number of packets is set to be small for S920. On the other hand, since the received signal after deformation correction has been corrected for effects such as body movement, the number of packets is set to be large so that blood flows with low flow rates can be observed. This improves the ability to visualize blood flows.

[0086] The super-resolution processor 207 extracts feature points from each of the multiple frames of blood flow images generated in S950 (S960). The method of determining the extraction range of feature points may be the same as in the first embodiment. The blood flow image from which feature points are extracted may be either before or after averaging as in the first embodiment, but the blood flow image before averaging has a larger pixel value difference within the blood flow, making it easier to extract feature points.

[0087] Finally, the super-resolution processor 207 generates a super-resolution blood flow image by integrating pixel values ​​of the feature points extracted from the blood flow images of multiple frames (S970). The super-resolution processor 207 displays the generated super-resolution blood flow image on the display device 108.

[0088] In this way, by setting MTI filters appropriate for deformation correction and blood flow extraction, it is possible to generate a super-resolution blood flow image with improved resolution and high accuracy.

[0089] <Fourth embodiment> In the fourth embodiment, similar to the third embodiment, blood vessel alignment (deformation correction) is applied to the received signal instead of the blood flow image, and an MTI filter is applied to the corrected received signal to generate a blood flow image. Then, similar to the second embodiment, a plurality of feature separation images are generated from the blood flow image, and feature points are extracted from each of the feature separation images. That is, the fourth embodiment is a combination of the second and third embodiments.

[0090] The device configuration may be the same as that of the first embodiment. A process flow for acquiring a super-resolution blood flow image according to the fourth embodiment will be described with reference to FIG. 13. The processes from S1110 to S1150 are the same as those from S910 to S950 of the third embodiment. The super-resolution processing unit 207 generates a plurality of feature separation images having different image features from each of the plurality of frames of blood flow images generated in S1150 (S1160). FIG. 14 shows a frame setting method for generating a plurality of feature separation images from a blood flow image based on a deformation-corrected received signal. After deformation correction is performed on the received signal, a blood flow image is generated by applying an MTI filter, and a plurality of feature separation images are generated from each blood flow image. The method for generating the feature separation images may be the same as that of the second embodiment.

[0091] The super-resolution processing unit 207 extracts feature points from each of the multiple feature separated images generated in S1160 (S1170). The method of determining the extraction range of the feature points may be the same as in the first and second embodiments.

[0092] Finally, the super-resolution processor 207 generates a super-resolution blood flow image by accumulating pixel values ​​of feature points extracted from each of the multiple feature separation images (S1180). The super-resolution processor 207 displays the generated super-resolution blood flow image on the display device 108. In this embodiment, as in the second embodiment, the frames used for accumulation may be appropriately selected, and the range of image features used for accumulation and weights at the time of accumulation may be appropriately set.

[0093] In this way, we set MTI filters suitable for deformation correction and blood flow extraction, and by extracting blood flows with a wide range of flow speeds and thicknesses from multiple blood flow images, we improved the resolution. It is possible to generate the above-mentioned super-resolution blood flow images.

[0094] <Other embodiments> The above-described embodiments merely show specific examples of the present invention. The scope of the present invention is not limited to the configurations of the above-described embodiments, and various embodiments can be adopted without changing the gist of the present invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as in the scope and gist of the invention.

[0095] The disclosed technology can be embodied as, for example, a system, an apparatus, a method, a program, or a recording medium (storage medium), etc. Specifically, the technology may be applied to a system composed of multiple devices (for example, a host computer, an interface device, an imaging device, a web application, etc.), or may be applied to an apparatus composed of a single device.

[0096] Needless to say, the object of the present invention can be achieved by the following: That is, a recording medium (or storage medium) on which is recorded a program code (computer program) of software that realizes the functions of the above-mentioned embodiments is supplied to a system or device. Needless to say, such a storage medium is a computer-readable storage medium. Then, a computer (or a CPU or MPU) of the system or device reads and executes the program code stored in the recording medium. In this case, the program code itself read from the recording medium realizes the functions of the above-mentioned embodiments, and the recording medium on which the program code is recorded constitutes the present invention.

[0097] The disclosure of this specification includes the following configurations, methods, and programs.

[0098] (Configuration 1) a first acquisition unit that acquires measurement data including information derived from tissue and information derived from blood flow based on an ultrasonic signal reflected within a living body; a second acquisition unit that acquires a deformation amount between a plurality of frames based on blood flow data obtained by extracting or emphasizing information derived from the blood flow of the measurement data; and a generation unit that generates display image data by synthesizing a plurality of frames of feature points in the information derived from the blood flow after aligning the frames based on the amount of deformation.

[0099] (Configuration 2) the second acquisition unit acquires the deformation amount based on a blood vessel region in the blood flow data. 2. The ultrasonic diagnostic apparatus according to claim 1.

[0100] (Configuration 3) the second acquisition unit acquires the deformation amount based on a position or a shape of a blood vessel region in the blood flow data. 3. The ultrasonic diagnostic apparatus according to claim 2.

[0101] (Configuration 4) the second acquisition unit selects a frame including the largest amount of information derived from blood flow from among the blood flow data of the plurality of frames as a reference frame, and calculates the deformation amount of the other frames based on a blood vessel region in the blood flow data of the reference frame; 3. The ultrasonic diagnostic apparatus according to claim 2.

[0102] (Configuration 5) The second acquisition unit acquires the deformation amount by using a learned model that has been trained to input blood flow data and output the deformation amount of the blood flow data. 2. The ultrasonic diagnostic apparatus according to claim 1.

[0103] (Configuration 6) The generation unit defines a blood vessel region as a non-rigid body and performs registration between frames using non-rigid registration. The ultrasonic diagnostic apparatus according to any one of configurations 1 to 5.

[0104] (Configuration 7) The blood flow data is data obtained by reducing clutter components derived from tissues in the measurement data. The ultrasonic diagnostic apparatus according to any one of configurations 1 to 6.

[0105] (Configuration 8) the second acquisition unit averages the blood flow data of two or more frames and acquires the deformation amount from the averaged blood flow data. The ultrasonic diagnostic apparatus according to any one of configurations 1 to 7.

[0106] (Configuration 9) the generating unit extracts pixels having pixel values ​​within a predetermined extraction range as the feature points from each of a plurality of frame images having information derived from blood flow as pixel values; The ultrasonic diagnostic apparatus according to any one of configurations 1 to 8.

[0107] (Configuration 10) The generation unit sets the extraction range according to a maximum pixel value of the image. 10. The ultrasound diagnostic apparatus according to configuration 9.

[0108] (Configuration 11) the generation unit divides the image into a plurality of local regions, and sets the extraction range for each of the local regions according to a maximum pixel value for each of the local regions. 11. The ultrasound diagnostic apparatus according to configuration 9 or 10.

[0109] (Configuration 12) the generating unit divides the image into a plurality of local regions, and sets the extraction range for each of the local regions according to a depth from a body surface of each of the local regions. The ultrasonic diagnostic apparatus according to any one of configurations 9 to 11.

[0110] (Configuration 13) the generating unit removes noise from pixel groups extracted as the feature points from the images of the plurality of frames, and generates the display image data by synthesizing the pixel groups after the noise removal. The ultrasonic diagnostic apparatus according to any one of configurations 9 to 12.

[0111] (Configuration 14) The generation unit up-converts the images of the multiple frames and extracts the feature points from the up-converted images. The ultrasonic diagnostic apparatus according to any one of configurations 9 to 13.

[0112] (Configuration 15) the generation unit performs correction for alignment of the images of the plurality of frames based on the amount of deformation, and extracts the feature points from each of the corrected images of the plurality of frames. The ultrasonic diagnostic apparatus according to any one of configurations 9 to 14.

[0113] (Configuration 16) the generation unit performs correction for alignment based on the amount of deformation on the measurement data of the plurality of frames, generates images of the plurality of frames having information derived from blood flow as pixel values ​​based on the corrected measurement data of the plurality of frames, and extracts the feature points from each of the generated images of the plurality of frames. The ultrasonic diagnostic apparatus according to any one of configurations 9 to 14.

[0114] (Configuration 17) 17. The ultrasound diagnostic apparatus of claim 15, wherein the generation unit generates a plurality of feature separation images having different image features from each of the plurality of frame images, and extracts the feature points from each of the plurality of feature separation images.

[0115] (Configuration 18) The image features include at least one of an image feature related to a blood flow velocity, an image feature related to a shape or a thickness of a blood vessel, and an image feature related to a blood flow direction. 18. The ultrasound diagnostic apparatus according to claim 17.

[0116] (Configuration 19) the generating unit, when generating the display image data, combines the feature points extracted from each of the plurality of feature separated images by weighting them according to the image features. 19. The ultrasonic diagnostic apparatus according to claim 17 or 18.

[0117] (Configuration 20) Obtaining a deformation amount between a plurality of frames based on blood flow data obtained by extracting or emphasizing information derived from blood flow from measurement data including information derived from tissues and information derived from blood flow in a living body; a generating unit that generates display image data by synthesizing a plurality of frames of feature points in the information derived from the blood flow after aligning the frames based on the deformation amount; A medical information processing device having the above configuration.

[0118] (Method 21) A method for processing information by a computer, comprising the steps of: acquiring measurement data including information derived from tissue and information derived from blood flow based on an ultrasonic signal reflected within a living body; acquiring a deformation amount between a plurality of frames based on blood flow data obtained by extracting or emphasizing information derived from the blood flow of the measurement data; generating display image data by synthesizing a plurality of frames of feature points in the information derived from the blood flow, after aligning the frames based on the deformation amount; An information processing method comprising:

[0119] (Program 22) acquiring measurement data including information derived from tissue and information derived from blood flow based on an ultrasonic signal reflected within a living body; acquiring a deformation amount between a plurality of frames based on blood flow data obtained by extracting or emphasizing information derived from the blood flow of the measurement data; generating display image data by synthesizing a plurality of frames of feature points in the information derived from the blood flow, after aligning the frames based on the deformation amount; A program for causing a computer to execute an information processing method including the steps of: [Explanation of symbols]

[0120] 1: Ultrasound diagnostic equipment

Claims

1. a first acquisition unit that acquires measurement data including information derived from tissue and information derived from blood flow based on an ultrasonic signal reflected within a living body; a second acquisition unit that acquires a deformation amount between a plurality of frames based on blood flow data obtained by extracting or emphasizing information derived from the blood flow of the measurement data; and a generation unit that generates display image data by synthesizing a plurality of frames of feature points in the information derived from the blood flow after aligning the frames based on the amount of deformation.

2. the second acquisition unit acquires the deformation amount based on a blood vessel region in the blood flow data. The ultrasonic diagnostic apparatus according to claim 1 .

3. the second acquisition unit acquires the deformation amount based on a position or a shape of a blood vessel region in the blood flow data. The ultrasonic diagnostic apparatus according to claim 2 .

4. the second acquisition unit selects a frame including the largest amount of information derived from blood flow from among the blood flow data of the plurality of frames as a reference frame, and calculates the deformation amount of the other frames based on a blood vessel region in the blood flow data of the reference frame; The ultrasonic diagnostic apparatus according to claim 2 .

5. The second acquisition unit acquires the deformation amount by using a learned model that has been trained to input blood flow data and output the deformation amount of the blood flow data. The ultrasonic diagnostic apparatus according to claim 1 .

6. The generation unit defines a blood vessel region as a non-rigid body and performs registration between frames using non-rigid registration. The ultrasonic diagnostic apparatus according to claim 1 .

7. The blood flow data is data obtained by reducing clutter components derived from tissues in the measurement data. The ultrasonic diagnostic apparatus according to claim 1 .

8. the second acquisition unit averages the blood flow data of two or more frames and acquires the deformation amount from the averaged blood flow data. The ultrasonic diagnostic apparatus according to claim 1 .

9. the generating unit extracts pixels having pixel values ​​within a predetermined extraction range as the feature points from each of a plurality of frame images having information derived from blood flow as pixel values; The ultrasonic diagnostic apparatus according to claim 1 .

10. The generation unit sets the extraction range according to a maximum pixel value of the image. The ultrasonic diagnostic apparatus according to claim 9.

11. the generation unit divides the image into a plurality of local regions, and sets the extraction range for each of the local regions according to a maximum pixel value for each of the local regions. The ultrasonic diagnostic apparatus according to claim 9.

12. the generating unit divides the image into a plurality of local regions, and sets the extraction range for each of the local regions according to a depth from a body surface of each of the local regions. The ultrasonic diagnostic apparatus according to claim 9.

13. the generating unit removes noise from pixel groups extracted as the feature points from the images of the plurality of frames, and generates the display image data by synthesizing the pixel groups after the noise removal. The ultrasonic diagnostic apparatus according to claim 9.

14. The generation unit up-converts the images of the multiple frames and extracts the feature points from the up-converted images. The ultrasonic diagnostic apparatus according to claim 9.

15. the generation unit performs correction for alignment based on the deformation amount on the images of the plurality of frames, and extracts the feature points from each of the corrected images of the plurality of frames. The ultrasonic diagnostic apparatus according to claim 9.

16. the generation unit performs correction for alignment based on the amount of deformation on the measurement data of the plurality of frames, generates images of the plurality of frames having information derived from blood flow as pixel values ​​based on the corrected measurement data of the plurality of frames, and extracts the feature points from each of the generated images of the plurality of frames. The ultrasonic diagnostic apparatus according to claim 9.

17. 17. The ultrasound diagnostic apparatus according to claim 15, wherein the generating unit generates a plurality of feature separated images having different image features from each of the plurality of frame images, and extracts the feature points from each of the plurality of feature separated images.

18. The image features include at least one of an image feature related to a blood flow velocity, an image feature related to a shape or a thickness of a blood vessel, and an image feature related to a blood flow direction. The ultrasonic diagnostic apparatus according to claim 17.

19. the generating unit, when generating the display image data, combines the feature points extracted from each of the plurality of feature separated images by weighting them according to the image features. The ultrasonic diagnostic apparatus according to claim 17.

20. obtaining a deformation amount between a plurality of frames based on blood flow data obtained by extracting or emphasizing information derived from blood flow from measurement data including information derived from tissues and information derived from blood flow in a living body; a generating unit that generates display image data by synthesizing a plurality of frames of feature points in the information derived from the blood flow after aligning the frames based on the deformation amount; A medical information processing device having the above configuration.

21. A method for processing information by a computer, comprising the steps of: acquiring measurement data including information derived from tissue and information derived from blood flow based on an ultrasonic signal reflected within a living body; acquiring a deformation amount between a plurality of frames based on blood flow data obtained by extracting or emphasizing information derived from the blood flow of the measurement data; generating display image data by synthesizing a plurality of frames of feature points in the information derived from the blood flow, after aligning the frames based on the deformation amount; An information processing method comprising:

22. acquiring measurement data including information derived from tissue and information derived from blood flow based on an ultrasonic signal reflected within a living body; acquiring a deformation amount between a plurality of frames based on blood flow data obtained by extracting or emphasizing information derived from the blood flow of the measurement data; generating display image data by synthesizing a plurality of frames of feature points in the information derived from the blood flow, after aligning the frames based on the deformation amount; A program for causing a computer to execute an information processing method including the steps of:

Images