Ultrasonic diagnostic apparatus, control method of ultrasonic diagnostic apparatus, and recording medium

Abstract

The present application provides an ultrasonic diagnostic apparatus capable of improving the accuracy of a likelihood image representing a target region in ultrasonic image diagnosis using a spatial compounding method. An ultrasonic diagnostic apparatus (1) according to the present disclosure includes a transmission / reception unit (11) that causes an ultrasonic probe (20) to perform transmission / reception of an ultrasonic beam; a signal processing unit (12) that generates an ultrasonic image based on a reception signal acquired from the ultrasonic probe (20); a target recognition unit (13c) that performs a segmentation process based on a structure type on the ultrasonic image to generate a likelihood image representing a region where a target exists in the ultrasonic image; and a likelihood image synthesis unit (13e) that synthesizes the likelihood images of a plurality of ultrasonic images generated by ultrasonic scanning using ultrasonic beams having mutually different deflection angles to generate a spatially compounded likelihood image.

Description

Technical Field

[0001] This invention relates to an ultrasonic diagnostic device, a control method for the ultrasonic diagnostic device, and a control program for the ultrasonic diagnostic device. Background Technology

[0002] Previously, as a type of medical imaging diagnostic device, ultrasound diagnostic devices were known that visualize the internal shape, properties, or dynamics of the subject as ultrasound images by sending ultrasound waves toward the subject, receiving the reflected waves, and performing prescribed signal processing on the received signals. Ultrasound diagnostic devices can acquire ultrasound images through simple operations such as placing the ultrasound probe against the body surface or inserting it into the body; therefore, they are safe and exert minimal stress on the subject.

[0003] Ultrasound diagnostic devices are used, for example, to treat a target area by inserting a puncture needle into the body of a patient under ultrasound guidance. In such treatments, the surgeon or other personnel can identify the target area by observing ultrasound images obtained through the ultrasound diagnostic device while inserting the puncture needle and performing the treatment.

[0004] In ultrasound-guided treatment, to accurately determine the location and extent of the treatment area, it is preferable to ensure that the target area is clearly reflected in the ultrasound image (B-mode image). For example, in nerve blocks where local anesthesia is administered via direct or peripheral puncture of peripheral nerves, the nerve into which the anesthetic is injected, and blood vessels into which the anesthetic must not be mistakenly injected, can be the target. Furthermore, while in nerve blocks, the surgeon can visually distinguish nerves and blood vessels on the ultrasound image and avoid puncturing blood vessels, this requires a high level of skill and extensive experience.

[0005] Against this background, in recent years, the following technologies have also been proposed: providing a display image of an ultrasound image to a person performing surgery (hereinafter also referred to as "user") in a way that identifies a target within an ultrasound image and is able to identify the area of ​​such a target (for example, see Patent Document 1 and Patent Document 2).

[0006] Figure 1 This figure illustrates an example of an image processing method for ultrasound images as described in the prior art.

[0007] In existing image processing methods, for example, a recognition model trained through machine learning is used to identify targets (e.g., neural tissue) within an ultrasound image, generating a likelihood image (also known as segmentation processing) that distinguishes regions with high likelihood (i.e., certainty factor) from regions with low likelihood (i.e., certainty factor) within the ultrasound image as areas where targets exist. Furthermore, in existing image processing methods, a color map is used to add color information (hue, saturation, brightness) to each pixel of the ultrasound image based on the pixel values ​​of the ultrasound image and the likelihood image, causing at least one change in the hue, saturation, or brightness of the ultrasound image to generate a display image for the user.

[0008] Furthermore, the "likelihood" of a target is an indicator of the probability of it becoming a target; the likelihood is high in areas where a target exists and low in areas where a non-target exists. Additionally, a "likelihood image" is an image representing the distribution of the likelihood of a target corresponding to the overall ultrasound image (i.e., the area where the target exists).

[0009] Existing technical documents

[0010] Patent documents

[0011] Patent Document 1: Japanese Patent Publication No. 2019-508072

[0012] Patent Document 2: Japanese Patent Application Publication No. 2021-058232 Summary of the Invention

[0013] The problem that the invention aims to solve

[0014] However, the inventors of this application are researching the application of spatial compositing in such an ultrasound diagnostic device, with the aim of providing the user with high-quality ultrasound images and more accurately highlighting the target area (including not only puncture objects such as nerve tissue, but also objects that draw the user's attention to the area, such as puncture needles; the same applies hereinafter) within the ultrasound image. Spatial compositing is a method of generating multiple frames of images by sending ultrasound beams from different directions toward the same part of the body being examined, and then combining these multiple frames to generate a single spatial composite image.

[0015] Figure 2 This diagram illustrates the typical spatial composition method.

[0016] In spatial composite methods, for example, such as Figure 2As shown, ultrasonic images B (generated by an ultrasonic beam with a deflection angle of 0 degrees), A (generated by an ultrasonic beam with a deflection angle of -θ degrees), and C (generated by an ultrasonic beam with a deflection angle of +θ degrees) are repeatedly generated in a 3-frame period and in the same order. Each time one frame of received data is acquired, it is combined with the ultrasonic images from the immediately preceding two frames to form a spatial composite image Sy, representing three frames of ultrasonic images. Thus, the spatial composite image Sy, synthesized from the ultrasonic images A, B, and C corresponding to the three deflection angles, is continuously updated.

[0017] According to this spatial composite method, by synthesizing multi-frame images generated from ultrasonic beams transmitted from different directions, it is possible to reduce speckle noise caused by scattered waves from numerous scattering sources present in the subject body, and also reduce acoustic noise such as shadows.

[0018] However, the spatial compositing method involved in the prior art has several problems that make the existence region of the target uncertain.

[0019] Figure 3 This diagram illustrates motion artifacts, one of the problems with spatial composite methods in existing technologies.

[0020] Typically, in order to locate therapeutic targets (e.g., nerve tissue) within the body during an ultrasound examination, the user moves the ultrasound probe along the surface of the subject. This results in frame images of the object being synthesized in various directions during spatial compositing, becoming images with a shifted shooting position. Consequently, the spatial composite image generated by synthesizing frame images from various directions is unclear, making it difficult to identify the target based on this composite image. Figure 3 In addition to the movement of the ultrasound probe, such motion artifacts are also caused by the movement of the tissues within the subject (e.g., the heart) themselves.

[0021] Figure 4 This diagram illustrates the increasing difficulty in identifying structures that exhibit acoustic anisotropy (hereinafter simply referred to as "anisotropy") relative to ultrasonic beams, which are other problem points in the spatial composite method involved in the prior art.

[0022] Typically, as a target for alerting the user to the location of an ultrasound examination, it is mixed with substances such as nerve tissue. Figure 4 Structures that are not anisotropic relative to ultrasound beams, such as HT (in the context of ultrasound), and puncture needles ( Figure 4Structures like the QT (Quicksweep) exhibit anisotropy relative to the ultrasound beam. Ultrasound waves are typically reflected from boundaries with differences in acoustic impedance, but the closer the angle of illumination to 90 degrees relative to the boundary, the stronger the reflection, resulting in clear reflected ultrasound waves. Therefore, structures like nerve tissue that induce reflected ultrasound waves in various directions relative to the incident ultrasound beam do not depend on the beam direction, eliminating concerns about unclear depiction in spatial composite images. However, in the case of a puncture needle, when the ultrasound beam direction is orthogonal to the needle's extension direction, the needle is clearly depicted in the ultrasound image; but when the ultrasound beam direction is parallel to the needle's extension direction, the needle becomes almost invisible in the ultrasound image.

[0023] In other words, in methods that simply average frame images in all directions to generate spatial composite images, such as the spatial composite method involved in the prior art, the images of anisotropic structures such as puncture needles are unclear as a result of image synthesis, and it is more difficult than usual to identify such structures in spatial composite images.

[0024] Figure 5 This figure illustrates the increased difficulty in recognizing structures at the edges of an image, which is another problem with spatial compositing methods in the prior art.

[0025] Typically, in spatial composite processing, the ultrasonic image generated by ultrasonic scanning using an ultrasonic beam with a deflection angle of 0 degrees is trimmed to match the image region of the ultrasonic image generated by ultrasonic scanning using an ultrasonic beam with an outer transmission direction. Figure 2 In the process, after ultrasound image A and ultrasound image C, these images are synthesized.

[0026] Typically, the goal ( Figure 3 In ultrasound images, the more extensive the nerve tissue (HT) is within the surrounding area, the easier it is to identify the target. Conversely, the difficulty of target identification increases when the target is located at the edge of the image and depicted in a partially incomplete state. In other words, the difficulty of identification increases when an ultrasound image is generated using an ultrasound beam transmitted from the outer direction (see [reference]). Figure 5 In ultrasound images (A), although they can be identified, they are more difficult to identify than usual in spatial composite images generated by simply averaging frame images in all directions.

[0027] This disclosure was made in view of the above-mentioned problems, and aims to provide an ultrasound diagnostic apparatus, a control method for the ultrasound diagnostic apparatus, and a control program for the ultrasound diagnostic apparatus that can improve the accuracy of the likelihood image representing the target region in ultrasound image diagnosis using the spatial composite method.

[0028] Methods for solving problems

[0029] The present disclosure, which addresses the aforementioned issues, is an ultrasound diagnostic apparatus comprising: a transmitting and receiving unit that enables an ultrasound probe to transmit and receive an ultrasound beam; a signal processing unit that generates an ultrasound image based on a received signal acquired from the ultrasound probe; a target recognition unit that performs segmentation processing on the ultrasound image based on structure type to generate a likelihood image representing the presence area of ​​a target in the ultrasound image; and a likelihood image synthesis unit that synthesizes the likelihood images of multiple ultrasound images generated by ultrasound scanning using ultrasound beams with different deflection angles to generate a spatial composite likelihood image.

[0030] Another aspect is a control method for an ultrasound diagnostic device, comprising the following processes: causing an ultrasound probe to transmit and receive an ultrasound beam; generating an ultrasound image based on the received signal obtained from the ultrasound probe; performing segmentation processing on the ultrasound image based on structure type to generate a likelihood image representing the presence area of ​​a target in the ultrasound image; and combining the likelihood images of multiple ultrasound images generated by ultrasound scanning using ultrasound beams with different deflection angles to generate a spatial composite likelihood image.

[0031] Another aspect is a control program for an ultrasound diagnostic device, which causes a computer to perform the following processes: causing an ultrasound probe to transmit and receive an ultrasound beam; generating an ultrasound image based on the received signal obtained from the ultrasound probe; performing segmentation processing on the ultrasound image based on structure type to generate a likelihood image representing the area where a target exists in the ultrasound image; and combining the likelihood images of multiple ultrasound images generated by ultrasound scanning using ultrasound beams with different deflection angles to generate a spatial composite likelihood image.

[0032] The effects of the invention

[0033] According to the ultrasound diagnostic apparatus disclosed herein, it is possible to improve the accuracy of the likelihood image representing the target region in ultrasound image diagnosis using the spatial composite method. Attached Figure Description

[0034] Figure 1This figure illustrates an example of an image processing method for ultrasound images as described in the prior art.

[0035] Figure 2 This diagram illustrates the typical spatial composition method.

[0036] Figure 3 This diagram illustrates motion artifacts, one of the problems with spatial composite methods in existing technologies.

[0037] Figure 4 This figure illustrates the increased difficulty in identifying structures with acoustic anisotropy relative to ultrasonic beams, which are other problems with spatial composite methods involved in the prior art.

[0038] Figure 5 This figure illustrates the increased difficulty in recognizing structures at the edges of an image, which is another problem with spatial compositing methods in the prior art.

[0039] Figure 6 This is a diagram showing an example of the appearance of an ultrasonic diagnostic apparatus according to an embodiment of the present invention.

[0040] Figure 7 This is a block diagram showing the main parts of the control system of an ultrasound diagnostic device.

[0041] Figure 8 This is a diagram showing the detailed structure of the image processing unit.

[0042] Figure 9 This diagram illustrates the processing performed by the target recognition unit.

[0043] Figure 10 This diagram illustrates the processing performed by the likelihood image synthesis unit.

[0044] Figure 11 This is a diagram representing an example of an image synthesis method corresponding to a recognition object, stored in an image synthesis method data table. Detailed Implementation

[0045] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Furthermore, in this specification and the accompanying drawings, structural elements having substantially the same function are labeled with the same reference numerals, thereby omitting repeated descriptions.

[0046] [Overall structure of the ultrasound diagnostic device]

[0047] The following is for reference Figures 5-6The overall structure of an ultrasound diagnostic apparatus (hereinafter, "ultrasound diagnostic apparatus 1") according to an embodiment of the present invention will be described. Furthermore, the ultrasound diagnostic apparatus 1 according to this embodiment is used, for example, to visualize the shape, properties, or dynamics of a subject as an ultrasound image for image-based diagnostic purposes.

[0048] Figure 6 This is a diagram showing an example of the appearance of the ultrasonic diagnostic apparatus 1 according to this embodiment. Figure 7 This is a block diagram showing the main parts of the control system of the ultrasonic diagnostic apparatus 1 according to this embodiment.

[0049] The ultrasound diagnostic device 1 visualizes the shape, characteristics, or dynamics of the body within the patient as an ultrasound image for image-based diagnostic purposes. For example, the ultrasound diagnostic device 1 has the function of visually indicating the area where the target is located as puncture aid information and superimposed on a B-mode image when performing nerve block by injecting an anesthetic into or around a nerve during puncture of the patient.

[0050] Furthermore, in this embodiment, for example, the nerve tissue used to determine the area where the puncture needle should be inserted, and the puncture needle itself, can serve as "targets" for drawing the user's attention to the area. The target setting can be arbitrarily changed depending on how the user uses the ultrasound diagnostic device. Nerves can be treated as targets, while structures other than nerves, such as blood vessels, bones, and muscle fibers, can be treated as non-targets. Nerves and blood vessels that cannot be punctured can be treated as targets, while other structures can be treated as non-targets.

[0051] The ultrasonic diagnostic device 1 includes an ultrasonic diagnostic device body 10 and an ultrasonic probe 20. The ultrasonic diagnostic device body 10 and the ultrasonic probe 20 are connected, for example, via a cable 30.

[0052] The ultrasonic probe 20 transmits ultrasonic waves to the subject and receives the ultrasonic echoes reflected within the subject, converting them into received signals and transmitting them to the main body 10 of the ultrasonic diagnostic device. The ultrasonic probe 20 can be any type of probe, such as convex, linear, or fan-shaped probes.

[0053] The ultrasonic probe 20 has an array of piezoelectric vibrators 21 arranged in an array, and a channel switching unit (not shown) for individually switching the on and off states of the piezoelectric vibrators that make up the array of piezoelectric vibrators 21.

[0054] The array of piezoelectric vibrators 21 is, for example, composed of a plurality of piezoelectric vibrators arranged in an array along the scanning direction. Furthermore, through the control of the channel switching unit based on the control unit 16, the driving states of the plurality of piezoelectric vibrators constituting the array of piezoelectric vibrators 21 are switched on and off sequentially along the scanning direction, either individually or in blocks. In other words, the plurality of piezoelectric vibrators, individually or in blocks, convert the voltage pulses generated by the transmitting / receiving unit 11 into ultrasonic beams and transmit them into the subject body, and receive the reflected wave beams generated by the ultrasonic beams reflected within the subject body and convert them into electrical signals, which are then output to the transmitting / receiving unit 11. Thus, in the ultrasonic probe 20, the transmission and reception of ultrasonic waves are performed in a manner that scans the subject body.

[0055] The main body 10 of the ultrasound diagnostic device includes a transmitting and receiving unit 11, a signal processing unit 12, an image processing unit 13, a display unit 14, an operation input unit 15, and a control unit 16.

[0056] The transmitting and receiving unit 11 is a transmitting and receiving circuit that enables the ultrasonic probe 20 to perform ultrasonic transmitting and receiving.

[0057] The transmitting and receiving unit 11 includes: a transmitting unit 11a that generates voltage pulses (hereinafter referred to as "drive signals") and sends them to each piezoelectric vibrator of the ultrasonic probe 20; and a receiving unit 11b that receives and processes the electrical signals (hereinafter referred to as "receive signals") involved in the receiving beam generated by each piezoelectric vibrator of the ultrasonic probe 20. Furthermore, the transmitting unit 11a and the receiving unit 11b, under the control of the control unit 16, respectively execute the operation of transmitting and receiving ultrasonic waves by the ultrasonic probe 20.

[0058] The transmitting unit 11a is configured, for example, to include a pulse oscillator and a pulse setting unit for each channel connected to the ultrasonic probe 20. The transmitting unit 11a adjusts the voltage pulses generated by the pulse oscillator to the voltage amplitude, pulse width, and timing set by the pulse setting unit, and sends them to the array of vibrators 21. Furthermore, the transmitting unit 11a supplies a drive signal to each piezoelectric vibrator by appropriately setting a delay time for each channel, such that the ultrasonic waves output from each piezoelectric vibrator of the ultrasonic probe 20 are converged in a beam-like manner in a predetermined direction.

[0059] The receiving unit 11b is configured, for example, to include a preamplifier, an analog-to-digital converter (ADC), and a receiving beamformer. The preamplifier and ADC are provided for each channel connected to the ultrasonic probe 20, amplifying the weak received signal and converting the amplified received signal (analog signal) into a digital signal. The receiving beamformer concentrates multiple received signals into one by inverting and adding the received signals (digital signals) from each channel, and outputs them to the signal processing unit 12. In the receiving beamformer, for example, a delay time is appropriately set for each channel to converge ultrasonic echoes from a predetermined direction, converging multiple received signals into one and outputting it to the signal processing unit 12. Furthermore, in the receiving beamformer, dynamic receiving focus control is performed to continuously move the receiving focus point from the vicinity of the ultrasonic radiation surface of the ultrasonic probe 20 towards a deeper direction.

[0060] The signal processing unit 12 acquires a signal by detecting (envelope detection) the acoustic data input from the receiving unit 11b. Furthermore, it performs logarithmic amplification, filtering (e.g., low-pass, smoothing), and enhancement processing as needed. The signal processing unit 12 sequentially stores the received signals at each scanning position in a frame memory, generating two-dimensional data composed of sampled data (e.g., the signal strength of the received signal) at each position within a cross-section along the scanning and depth directions. The signal processing unit 12, for example, converts the signal strength of the received signal at each position of this two-dimensional data into pixel values, generating one frame of ultrasound image data for B-mode display (hereinafter simply referred to as "ultrasound image"). Moreover, the signal processing unit 12 generates such an ultrasound image whenever the transmitting and receiving unit 11 scans the subject.

[0061] In addition, the signal processing unit 12 may also include an orthogonal detection processing unit, an autocorrelation calculation unit, etc., in a manner that enables the generation of ultrasonic images involved in Doppler images.

[0062] The image processing unit 13 performs spatial composite processing on the ultrasonic image generated by the signal processing unit 12, and combines multiple ultrasonic images generated by ultrasonic scanning using ultrasonic beams with different deflection angles to generate an ultrasonic image for display (hereinafter referred to as "spatial composite ultrasonic image").

[0063] Furthermore, the image processing unit 13 performs segmentation processing based on the structure category on the ultrasonic image generated by the signal processing unit 12 to generate a likelihood image representing the area where the target exists. Moreover, the image processing unit 13 combines the likelihood images of multiple ultrasonic images generated by ultrasonic scanning using ultrasonic beams with different deflection angles to generate a single likelihood image (hereinafter referred to as a "spatial compound likelihood image") for display purposes.

[0064] Furthermore, the transmitting / receiving unit 11, the signal processing unit 12, and the image processing unit 13 are constructed, for example, by dedicated or general-purpose hardware (i.e., electronic circuits) corresponding to the processing of ASICs (Application Specific Integrated Circuits) and FPGAs (Field-Programmable Gate Arrays), and cooperate with the control unit 16 to realize their respective functions. Some or all of them may also be implemented by performing calculations according to a program using DSPs (Digital Signal Processors), CPUs (Central Processing Units), or GPGPUs (General-Purpose Graphics Processing Units).

[0065] The display unit 14 is, for example, a display such as an LCD (Liquid Crystal Display). The display unit 14 acquires display image data from the image processing unit 13 and displays the display image data.

[0066] The operation input unit 15 is, for example, a keyboard or mouse, and acquires the operation signals input by the user. The operation input unit 15 can, for example, set the type of ultrasound probe 20, the type of the subject (i.e., the type of biological tissue), the depth of the object being imaged within the subject, or the imaging mode (e.g., mode B, mode C, or mode E) based on the user's operation input.

[0067] The control unit 16 controls the transmission and reception unit 11, signal processing unit 12, image processing unit 13, display unit 14 and operation input unit 15 according to their respective functions, and performs overall control of the ultrasound diagnostic device 1.

[0068] The control unit 16 includes, for example, a CPU (Central Processing Unit) as an arithmetic / control device, a ROM (Read Only Memory) as main storage, and RAM (Random Access Memory) as main storage. The ROM stores basic programs and basic setting data. The CPU reads the program corresponding to the processing content from the ROM, expands it in the RAM, and executes the expanded program, thereby centrally controlling the operation of each functional module of the main body 10 of the ultrasound diagnostic device.

[0069] Furthermore, in this embodiment, the functions of each functional module are realized through cooperation between the hardware components constituting the functional modules and the control unit 16. Alternatively, the control unit 16 may execute programs to realize some or all of the functions of each functional module.

[0070] Furthermore, the control unit 16 determines the transmission and reception conditions of the ultrasonic waves in the ultrasonic probe 20 (e.g., aperture condition, convergence point, transmission waveform, center frequency, band, and apodization) based on the type of ultrasonic probe 20 set in the operation input unit 15 (e.g., convex, fan-shaped, or linear type), the depth of the object being imaged within the subject, and the imaging mode (e.g., mode B, mode C, or mode E). Moreover, the control unit 16 activates the transmission and reception unit 11 according to the transmission and reception conditions of the ultrasonic waves in the ultrasonic probe 20.

[0071] [Detailed Structure of Image Processing Unit 13]

[0072] Figure 8 This is a diagram showing the detailed structure of the image processing unit 13 according to this embodiment.

[0073] The image processing unit 13 in this embodiment includes a first DSC (Digital Scan Converter) 13a, an ultrasonic image synthesis unit 13b, a target recognition unit 13c, a second DSC (Digital Scan Converter) 13d, a likelihood image synthesis unit 13e, and a display image generation unit 13f.

[0074] The first DSC13a performs coordinate transformation processing and pixel interpolation processing corresponding to the type of ultrasonic probe 20 on the ultrasonic image generated by the signal processing unit 12, and converts the ultrasonic image data into display image data according to the scanning mode of the television signal of the display unit 14.

[0075] Ultrasonic image synthesis unit 13b as shown in reference Figure 2 As described above, multiple ultrasonic images generated by ultrasonic scanning using ultrasonic beams with different deflection angles are combined to create a spatial composite ultrasonic image.

[0076] In this embodiment, as an example, under the control of the control unit 16, the deflection angle of the ultrasonic beam transmitted from the ultrasonic probe 20 is controlled, such as... Figure 2As shown, ultrasonic images B (generated by ultrasonic scanning with an ultrasonic beam at a deflection angle of 0 degrees), A (generated by ultrasonic scanning with an ultrasonic beam at a deflection angle of -θ degrees), and C (generated by ultrasonic scanning with an ultrasonic beam at a deflection angle of +θ degrees) are repeatedly generated in the same order and at a 3-frame cycle. Each time the ultrasonic image synthesis unit 13b acquires one frame of ultrasonic image, it synthesizes it with the preceding two frames of ultrasonic images to generate a spatial composite ultrasonic image Sy, comprising three frames of ultrasonic images.

[0077] At this time, the ultrasonic image synthesis unit 13b, for example, trims the outer regions of ultrasonic images A and C generated by ultrasonic scanning using an ultrasonic beam with a deflection angle of -θ degrees and ultrasonic images C generated by ultrasonic scanning using an ultrasonic beam with a deflection angle of +θ degrees, which do not overlap with ultrasonic images B generated by ultrasonic scanning using an ultrasonic beam with a deflection angle of 0 degrees. After unifying the coordinate systems of the ultrasonic images A, B, and C, the ultrasonic images A, B, and C are synthesized by averaging the overlapping portions of the B-mode image signals obtained from the same location.

[0078] Furthermore, the number of frames of the ultrasonic image of the object synthesized by the ultrasonic image synthesis unit 13b can also be more than three.

[0079] Target recognition unit 13c as shown in reference Figure 1 As described above, using the recognition model D1, the ultrasound image generated by the signal processing unit 12 is subjected to structure category-based segmentation processing to generate a likelihood image (an image representing the distribution of the likelihood of the target in the ultrasound image) representing the area where the target exists in the ultrasound image. That is, the target recognition unit 13c identifies the target (e.g., nerve tissue, puncture needle) in the ultrasound image.

[0080] Here, the recognition model D1 is, for example, a neural network (e.g., a convolutional neural network), and is pre-learned using a known machine learning algorithm (e.g., backpropagation) in a manner that extracts features from the input ultrasonic image and outputs the likelihood distribution of the target in the ultrasonic image, and is pre-stored in the storage unit of the image processing unit 13. Typically, such a recognition model D1 is constructed using supervised learning of training data consisting of a dataset corresponding to the likelihood distribution of the ultrasonic image and the target. Furthermore, for an example of the learning process for the recognition model D1, reference may be made to Patent Document 2, an earlier application filed by the applicant of this application.

[0081] The recognition model D1 is trained, for example, to identify at least one structure category from ultrasound images, including nerve tissue, vascular tissue, muscle tissue, fascia tissue, tendon tissue, or a puncture needle. The recognition model D1 can also be prepared separately for each structure category, or it can be configured to recognize multiple structure categories. Furthermore, the target recognition unit 13c can switch the type of recognition model D1 according to the type of the target being identified.

[0082] In other words, the recognition model D1 calculates the likelihood of the target for each pixel or pixel block (a group of pixels) corresponding to each pixel region in the ultrasound image, and outputs the likelihood distribution of the target corresponding to the entire input ultrasound image (i.e., a likelihood image). The recognition model D1 in this embodiment is configured to output the likelihood of the target corresponding to the pixel block at the center of the input ultrasound image of a predetermined size. Furthermore, the target recognition unit 13c switches the input image for the recognition model D1 by scanning the entire ultrasound image in increments of a predetermined size using raster scanning, thereby outputting the likelihood distribution of the target for the entire ultrasound image (i.e., a likelihood image). At this time, the target recognition unit 13c outputs the likelihood distribution of the target from the input image, for example, through forward propagation processing of the recognition model D1 (neural network).

[0083] The likelihood image generated by the target recognition unit 13c is, for example, likelihood data of a value in the range of 0 to 1 calculated for each pixel region corresponding to each pixel region of the ultrasound image (see reference). Figure 1 Such a likelihood image can represent, for example, the likelihood distribution of a single target (e.g., neural tissue) in the overall ultrasound image, or the likelihood distribution of multiple targets (e.g., neural tissue and puncture needle) in the overall ultrasound image. Furthermore, the size of the likelihood image (i.e., the number of pixels) can be the same as the size of the ultrasound image, or it can be scaled down.

[0084] Furthermore, the recognition model D1 used for the target recognition unit 13c can be any recognition model other than a neural network, such as SVM (Support Vector Machine), k-nearest neighbor method, random forest, or a combination thereof. This recognition model extracts features of the object's pattern through learning processing and autonomously optimizes itself in a way that allows it to correctly identify the object's pattern even from noise-laden data. Therefore, it is effective in constructing a highly robust recognizer.

[0085] The second DSC13d performs coordinate transformation processing and pixel interpolation processing corresponding to the type of ultrasonic probe 20 on the likelihood image generated by the target recognition unit 13c, and converts the data of the likelihood image into data of a display image according to the scanning mode of the television signal of the display unit 14.

[0086] The likelihood image synthesis unit 13e synthesizes multiple likelihood images obtained from ultrasonic scanning using ultrasonic beams with different deflection angles to generate a spatial composite likelihood image.

[0087] The likelihood image synthesis unit 13e is essentially the same as the synthesis process of the ultrasonic image synthesis unit 13b. After unifying the coordinate systems of multiple likelihood images, it synthesizes multiple likelihood images by averaging the likelihoods at the same location in the multiple likelihood images. Specifically, the likelihood image synthesis unit 13e refers to the image synthesis method data table D2 pre-stored in the storage unit (not shown) of the image processing unit 13, and uses the image synthesis method set according to each structure category to synthesize multiple likelihood images (see reference). Figure 11 ).

[0088] Figure 9 This diagram illustrates the processing performed by the target identification unit 13c according to this embodiment. Figure 10 This diagram illustrates the processing performed by the likelihood image synthesis unit 13e according to this embodiment. Figure 11 This is a diagram representing an example of an image synthesis method corresponding to the recognized object stored in the image synthesis method data table D2.

[0089] like Figure 9 As shown, the target recognition unit 13c in this embodiment performs segmentation processing based on structure category on each ultrasonic image sequentially generated by the signal processing unit 12, generating a likelihood image representing the area where the target exists. Specifically, the target recognition unit 13c performs recognition processing on ultrasonic image B generated by ultrasonic scanning using an ultrasonic beam with a deflection angle of 0 degrees, generating a likelihood image B1; performs recognition processing on ultrasonic image A generated by ultrasonic scanning using an ultrasonic beam with a deflection angle of -θ degrees, generating a likelihood image A1; and performs recognition processing on ultrasonic image C generated by ultrasonic scanning using an ultrasonic beam with a deflection angle of +θ degrees, generating a likelihood image C1. Here, the recognition model D1 applied to ultrasonic images A, B, and C can be the same, or it can be a different recognition model optimized for each deflection angle.

[0090] According to the processing of the target recognition unit 13c, it is not affected by motion artifacts caused by spatial composite synthesis. Therefore, the target recognition unit 13c can generate likelihood images A1, B1 and C1 with high accuracy from ultrasonic images A, B and C respectively, which calculate the likelihood of the target region.

[0091] Furthermore, based on the processing of the target recognition unit 13c, it is also possible to identify structures such as the puncture needle QT, which have acoustic reflection anisotropy relative to the ultrasonic beam, with high accuracy in any one of the likelihood images A1, B1, or C1. Figure 9 In the ultrasound beam with a deflection angle of +θ degrees, the direction becomes orthogonal to the extension direction of the puncture needle QT, and the puncture needle QT becomes a state that is clearly depicted in the ultrasound image C and the likelihood image C1.

[0092] Furthermore, according to the processing of the target recognition unit 13c, structures existing at the edge of the image can also be recognized with high accuracy in any one of the likelihood images A1, B1, or C1. For example, in Figure 5 In the ultrasound image A, the neural tissue HT is clearly depicted, and a likelihood image A1 with high accuracy is generated to calculate the likelihood of the target region.

[0093] Moreover, such as Figure 11 As shown, the likelihood image synthesis unit 13e of this embodiment uses an image synthesis method set according to each structure category stored in the image synthesis method data table D2 to synthesize multiple likelihood images A1, B1, and C1 to generate a spatial composite likelihood image Sy1.

[0094] For example, when the target of the identified object is a structure with acoustic anisotropy relative to an ultrasonic beam (e.g., a puncture needle QT), the likelihood image synthesis unit 13e performs image synthesis on multiple likelihood images A1, B1, and C1 by selecting the maximum likelihood from the likelihoods of each of the multiple likelihood images A1, B1, and C1 of the synthesized object for each pixel region, or by selectively adding the likelihoods above a threshold from the likelihoods of each of the multiple likelihood images A1, B1, and C1 of the synthesized object. Alternatively, when the target of the identified object is a structure without acoustic anisotropy relative to an ultrasonic beam (e.g., nerve tissue HT), the likelihood image synthesis unit 13e performs image synthesis on multiple likelihood images A1, B1, and C1 by averaging the likelihoods of each of the multiple likelihood images A1, B1, and C1 of the synthesized object for each pixel region.

[0095] Furthermore, examples of structures that have acoustic anisotropy relative to an ultrasonic beam include puncture needles and fascia, while examples of structures that do not have acoustic anisotropy relative to an ultrasonic beam include nerve tissue and muscle tissue.

[0096] When there is only one target, the likelihood image synthesis unit 13e can also synthesize likelihood images A1, B1, and C1 for the entire region of the likelihood image using an image synthesis method corresponding to the construction category of the target. On the other hand, when there are multiple targets (i.e., when generating likelihood images representing the distribution of the likelihoods of multiple targets), the likelihood image synthesis unit 13e can also synthesize multiple likelihood images for each pixel region of the likelihood image using an image synthesis method corresponding to the type of target present in that pixel region.

[0097] also, Figure 9 In the diagram, two targets are shown: neural tissue (HT) and puncture needle (QT). Furthermore, the pixel values ​​in the neural tissue (HT) region of the spatial composite likelihood image Sy1 are calculated as the average of the likelihoods of likelihood images A1, B1, and C1. The maximum value among the likelihoods of likelihood images A1, B1, and C1 is selected to calculate the pixel values ​​in the puncture needle (QT) region of the spatial composite likelihood image Sy1.

[0098] According to the processing of the likelihood image synthesis unit 13e, the likelihood images A1, B1, and C1, which calculate the likelihood of the target region with high precision, can be synthesized to generate a spatial composite likelihood image Sy1. Therefore, the likelihood image synthesis unit 13e can construct a highly accurate likelihood image (i.e., likelihood distribution) that suppresses the influence of motion artifacts. This is because it avoids the difficulty of using a recognition model to identify the target from a spatial composite ultrasonic image that is blurred due to motion artifacts.

[0099] Furthermore, according to the processing of the likelihood image synthesis unit 13e, multiple likelihood images A1, B1, and C1 are synthesized using an image synthesis method set according to each structure category to generate a spatial composite likelihood image Sy1. Therefore, on the spatial composite likelihood image Sy1, for example, information of the likelihood image of the structure (here, the puncture needle QT) that has acoustic reflection anisotropy relative to the ultrasonic beam is clearly shown in likelihood images A1, B1, or C1. That is, the area where the target exists can be clearly identified on the spatial composite likelihood image Sy1.

[0100] Furthermore, according to the processing of the likelihood image synthesis unit 13e, after generating likelihood images A1, B1, and C1 from ultrasonic images A, B, and C respectively, they are synthesized. Therefore, targets existing at the edge of the image can also be identified with high precision. This is because at least one of ultrasonic images A, B, or C reflects the overall state of the structure existing at the edge of the image, and targets can be identified with high precision from at least one of the likelihood images A1, B1, and C1 generated from such ultrasonic images A, B, and C.

[0101] Furthermore, the likelihood image synthesis unit 13e preferably removes noise from each of the likelihood images A1, B1, and C1 based on changes in likelihood-related information (e.g., target likelihood, likelihood distribution) obtained from temporally continuous ultrasonic images before synthesizing the likelihood images A1, B1, and C1. In this case, the likelihood image synthesis unit 13e preferably performs noise processing, for example, by dividing the likelihood image B1 obtained from the temporal variation of an ultrasonic image B generated using an ultrasonic beam with a deflection angle of 0 degrees, the temporal variation of an likelihood image A1 obtained from an ultrasonic image A generated using an ultrasonic beam with a deflection angle of -θ degrees, and the temporal variation of an likelihood image C1 obtained from an ultrasonic image C generated using an ultrasonic beam with a deflection angle of +θ degrees.

[0102] In this case, the likelihood image synthesis unit 13e can remove noise from the likelihood image by applying moving average filtering or median filtering in the time axis direction, for example. In this case, regions where the change (steepness) of likelihood-related information exceeds a preset threshold can also be detected as noise regions, and noise removal processing can be performed only on these noise regions.

[0103] In addition, the likelihood image synthesis unit 13e may also perform normalization processing on the likelihood images A1, B1 and C1 respectively as preprocessing when synthesizing the likelihood images A1, B1 and C1.

[0104] The display image generation unit 13f applies the spatial composite likelihood image Sy1 generated in the target recognition unit 13c as an enhancement map of the target region in the spatial composite ultrasonic image Sy generated in the ultrasonic image synthesis unit 13b.

[0105] The display image generation unit 13f overlays, for example, a spatial composite ultrasonic image Sy generated in the ultrasonic image synthesis unit 13b onto a spatial composite ultrasonic image Sy, and outputs a spatial composite likelihood image Sy1 generated in the target recognition unit 13c to the display unit 14.

[0106] At this time, the image generation unit 13f can also be used, for example. Figure 1 The color map shown is used to synthesize a spatial composite ultrasound image Sy and a spatial composite likelihood image Sy1. The display image generation unit 13f, for example, based on the pixel values ​​of the spatial composite ultrasound image Sy and the spatial composite likelihood image Sy1, which are in corresponding positional relationships within the image, adds color information (hue, saturation, brightness) to each pixel of the spatial composite ultrasound image Sy, and changes at least one of the hue, saturation, and brightness of the spatial composite ultrasound image Sy to generate a display image provided to the user.

[0107] Furthermore, the display image generation unit 13f can replace the superposition of the spatial composite ultrasonic image Sy generated in the target recognition unit 13c onto the spatial composite ultrasonic image Sy generated in the ultrasonic image synthesis unit 13b, and instead arrange and display the spatial composite ultrasonic image Sy and the spatial composite likelihood image Sy1.

[0108] [Effect]

[0109] As described above, the ultrasonic diagnostic apparatus 1 according to this embodiment includes: a transmission and reception unit 11 that enables an ultrasonic probe 20 to transmit and receive ultrasonic beams; a signal processing unit 12 that generates an ultrasonic image based on the received signal obtained from the ultrasonic probe 20; a target recognition unit 13c that performs segmentation processing based on structure type on the ultrasonic image to generate a likelihood image representing the presence area of ​​a target in the ultrasonic image; and a likelihood image synthesis unit 13e that synthesizes the likelihood images of multiple ultrasonic images generated by ultrasonic scanning using ultrasonic beams with different deflection angles to generate a spatial composite likelihood image.

[0110] Therefore, according to the ultrasound diagnostic apparatus 1 of this embodiment, a high-accuracy likelihood image (i.e., likelihood distribution) can be constructed that suppresses the effects of motion artifacts. Furthermore, this allows for high-precision identification of targets at the edges of the image. In other words, the area where the target exists can be more clearly defined on the likelihood image (here, the spatial composite likelihood image Sy1).

[0111] Furthermore, in the ultrasound diagnostic apparatus 1 according to this embodiment, in particular, the likelihood image synthesis unit 13e uses an image synthesis method set according to each structure category to synthesize multiple likelihood images to generate a spatial composite likelihood image.

[0112] Therefore, according to the ultrasound diagnostic apparatus 1 of this embodiment, structures (puncture needles, fascia, etc.) that have acoustic anisotropy relative to the ultrasound beam can also be identified with high precision. As a result, the area where the target exists can be more clearly defined on the likelihood image (here, the spatial composite likelihood image Sy1).

[0113] (Modified Example)

[0114] In the above embodiment, the following structure is shown: the likelihood image synthesis unit 13e uses an image synthesis method pre-stored in the image synthesis method data table D2, set according to each structure category, to synthesize multiple likelihood images A1, B1, and C1 to generate a spatial composite likelihood image Sy1 (see reference). Figure 11 ).

[0115] However, the image synthesis method of the likelihood image synthesis unit 13e can also be set by the user. This allows for more flexible processing.

[0116] The likelihood image synthesis unit 13e can, for example, be configured so that the user can individually set the image synthesis method for each target projected into the ultrasound image or for each pixel region of the ultrasound image. Thus, for example, it can be configured to select the maximum value of the likelihoods of each of the likelihood images A1, B1, and C1 for targets at the ends of the image, and on the other hand, to calculate the average likelihood of each of the likelihood images A1, B1, and C1 for targets in the center of the image. Furthermore, in this case, the user interface image can be displayed on the display unit 14 in a manner that allows the user to selectively set the image synthesis method for a specified pixel region from the ultrasound image.

[0117] The specific examples of the present invention have been described in detail above, but these are merely illustrative and not intended to limit the scope of the claims. The technology described in the claims includes various modifications and alterations to the specific examples described above.

[0118] Industrial availability

[0119] According to the ultrasound diagnostic apparatus disclosed herein, it is possible to improve the accuracy of the likelihood image representing the target region in ultrasound image diagnosis using the spatial composite method.

[0120] Explanation of reference numerals in the attached figures

[0121] 1...Ultrasonic diagnostic device; 10...Main body of ultrasonic diagnostic device; 11...Transmitter and receiver unit; 11a...Transmitter unit; 11b...Receiver unit; 12...Signal processing unit; 13...Image processing unit; 13a...First DSC; 13b...Ultrasonic image synthesis unit; 13c...Target recognition unit; 13d...Second DSC; 13e...Likelihood image synthesis unit; 13f...Display image generation unit; 14...Display unit; 15...Operation input unit; 16...Control unit; 20...Ultrasonic probe; 30...Cable.

Claims

1. An ultrasonic diagnostic device, comprising: The transmitting and receiving unit enables the ultrasonic probe to transmit and receive ultrasonic beams. The signal processing unit generates an ultrasonic image based on the received signal acquired from the ultrasonic probe; The target recognition unit performs structure-based segmentation processing on the ultrasonic image to generate a likelihood image representing the region where the target exists in the ultrasonic image; and The likelihood image synthesis unit synthesizes the likelihood images of multiple ultrasonic images generated by ultrasonic scanning using ultrasonic beams with different deflection angles to generate a spatial composite likelihood image. The target comprises at least a first structure having acoustic reflection anisotropy relative to the ultrasonic beam and a second structure not having acoustic reflection anisotropy relative to the ultrasonic beam. The likelihood image synthesis unit synthesizes multiple likelihood images to generate the spatial composite likelihood image by using different image synthesis methods set according to each category of the target for each pixel region of the likelihood image.

2. The ultrasonic diagnostic device as described in claim 1, wherein, The spatial composite likelihood image is applied as an enhancement map of the target region in a spatial composite ultrasound image generated by synthesizing multiple ultrasound images, which are generated by scanning the ultrasound beams with mutually different deflection angles.

3. The ultrasonic diagnostic device as described in claim 1, wherein, When the target of the identified object is the first structure, the likelihood image synthesis unit selects the maximum likelihood from the likelihoods of multiple likelihood images of the synthesized object for each pixel region, or selectively adds the likelihoods above a threshold from the likelihoods of multiple likelihood images of the synthesized object, thereby performing image synthesis on multiple likelihood images. When the target of the identified object is the second structure, the likelihood image synthesis unit averages the likelihood of each of the multiple likelihood images of the synthesized object according to each pixel region, thereby performing image synthesis on the multiple likelihood images.

4. The ultrasonic diagnostic device as described in claim 3, wherein, The first structure includes a puncture needle. The second structure includes neural tissue.

5. The ultrasonic diagnostic device as described in claim 1, wherein, The likelihood image synthesis unit uses a user-defined image synthesis method to synthesize multiple likelihood images to generate the spatial composite likelihood image.

6. The ultrasonic diagnostic device as described in claim 1, wherein, The target recognition unit uses a recognition model learned through machine learning to perform segmentation processing on the multiple ultrasonic images based on the structure category.

7. The ultrasonic diagnostic device as described in claim 6, wherein, The recognition model is a neural network.

8. A control method for an ultrasonic diagnostic device, comprising the following processing: To enable the ultrasonic probe to transmit and receive ultrasonic beams; An ultrasonic image is generated based on the received signal obtained from the ultrasonic probe; The ultrasonic image is subjected to structure category-based segmentation to generate a likelihood image representing the presence region of the target in the ultrasonic image; and A spatial composite likelihood image is generated by combining the likelihood images of multiple ultrasonic images generated by ultrasonic scanning using ultrasonic beams with different deflection angles. The target comprises at least a first structure having acoustic reflection anisotropy relative to the ultrasonic beam and a second structure not having acoustic reflection anisotropy relative to the ultrasonic beam. In the process of generating the spatial composite likelihood image, the multiple likelihood images are synthesized to generate the spatial composite likelihood image by using different image synthesis methods set according to each category of the target for each pixel region of the likelihood image.

9. The control method for the ultrasonic diagnostic device as described in claim 8, wherein, The spatial composite likelihood image is applied as an enhancement map of the target region in a spatial composite ultrasound image generated by synthesizing multiple ultrasound images, which are generated by scanning the ultrasound beams with mutually different deflection angles.

10. The control method for the ultrasonic diagnostic device as described in claim 8, wherein, In the process of generating the spatial composite likelihood image, When the target of the identified object is the first structure, for each pixel region, the maximum likelihood of the likelihoods of the multiple likelihood images of the synthesized object is selected, or the likelihoods above a threshold of the multiple likelihood images of the synthesized object are selectively added together, thereby performing image synthesis on the multiple likelihood images. When the target of the identified object is the second structure, the likelihood of each of the multiple likelihood images of the synthesized object is averaged according to each pixel region, thereby performing image synthesis on the multiple likelihood images.

11. The control method for the ultrasonic diagnostic device as described in claim 10, wherein, The first structure includes a puncture needle. The second structure includes neural tissue.

12. The control method for the ultrasonic diagnostic device as described in claim 8, wherein, In the process of generating the spatial composite likelihood image, a user-defined image synthesis method is used to synthesize multiple likelihood images to generate the spatial composite likelihood image.

13. The control method for the ultrasonic diagnostic device as described in claim 8, wherein, In the process of generating a likelihood image representing the region where the target exists, a recognition model learned through machine learning is used to perform segmentation processing based on the structure category for each of the multiple ultrasonic images.

14. The control method for the ultrasonic diagnostic device as described in claim 13, wherein, The recognition model is a neural network.

15. A computer-readable recording medium storing a control program, the control program being a control program for an ultrasonic diagnostic device, causing a computer to perform the following processes: To enable the ultrasonic probe to transmit and receive ultrasonic beams; An ultrasonic image is generated based on the received signal obtained from the ultrasonic probe; The ultrasonic image is subjected to structure category-based segmentation to generate a likelihood image representing the presence region of the target in the ultrasonic image; and A spatial composite likelihood image is generated by combining the likelihood images of multiple ultrasonic images generated by ultrasonic scanning using ultrasonic beams with different deflection angles. The target comprises at least a first structure having acoustic reflection anisotropy relative to the ultrasonic beam and a second structure not having acoustic reflection anisotropy relative to the ultrasonic beam. In the process of generating the spatial composite likelihood image, the multiple likelihood images are synthesized to generate the spatial composite likelihood image by using different image synthesis methods set according to each category of the target for each pixel region of the likelihood image.

16. The computer-readable recording medium storing a control program as described in claim 15, wherein, In the process of generating the spatial composite likelihood image, an enhancement map of the target region in the spatial composite ultrasound image generated by synthesizing multiple ultrasound images is applied. These multiple ultrasound images are generated by scanning the ultrasound beams with mutually different deflection angles.

17. The computer-readable recording medium storing a control program as described in claim 15, wherein, In the process of generating the spatial composite likelihood image, When the target of the identified object is the first structure, for each pixel region, the maximum likelihood of the likelihoods of the multiple likelihood images of the synthesized object is selected, or the likelihoods above a threshold of the multiple likelihood images of the synthesized object are selectively added together, thereby performing image synthesis on the multiple likelihood images. When the target of the identified object is the second structure, the likelihood of each of the multiple likelihood images of the synthesized object is averaged according to each pixel region, thereby performing image synthesis on the multiple likelihood images.

18. The computer-readable recording medium storing a control program as described in claim 17, wherein, The first structure includes a puncture needle. The second structure includes neural tissue.

19. The computer-readable recording medium storing a control program as described in claim 15, wherein, In the process of generating the spatial composite likelihood image, a user-defined image synthesis method is used to synthesize multiple likelihood images to generate the spatial composite likelihood image.

20. The computer-readable recording medium storing a control program as described in claim 15, wherein, In the process of generating a likelihood image representing the region where the target exists, a recognition model learned through machine learning is used to perform segmentation processing based on the structure category for each of the multiple ultrasonic images.

21. The computer-readable recording medium storing a control program as described in claim 20, wherein, The recognition model is a neural network.

Images