Ultrasound imaging diagnostic device, classifier change method, and classifier change program

Abstract

To provide an ultrasonic image diagnostic apparatus, a discriminator change method and a discriminator change program which can reduce the time and effort for switching a discriminator when discriminating a discrimination object reflected on an ultrasonic image by using a discriminator.SOLUTION: An ultrasonic image diagnostic apparatus comprises: an ultrasonic image acquisition unit which acquires an ultrasonic image generated on the basis of a reception signal obtained by an ultrasonic probe that transmits / receives an ultrasonic wave to / from an object; a discrimination result acquisition unit which acquires a discrimination result output from a discriminator by inputting the acquired ultrasonic image to any of discriminators that discriminates the discrimination object reflected on the input ultrasonic image; and a discriminator change unit which changes the discriminator to which the acquired ultrasonic image is input.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an ultrasound imaging diagnostic apparatus, a classifier changing method, and a classifier changing program. [Background technology]

[0002] Conventionally, there has been known an ultrasound imaging diagnostic device that transmits and receives ultrasound waves to and from a subject such as a living organism using an ultrasound probe, generates ultrasound image data based on signals obtained from the received ultrasound, and displays an ultrasound image based on this on an image display device. Ultrasound imaging diagnosis using an ultrasound imaging diagnostic device can obtain real-time images of the heartbeat, fetal movement, etc., with a simple operation of simply placing the ultrasound probe on the surface of the subject's body, and is non-invasive and highly safe, so it can be performed repeatedly.

[0003] Incidentally, techniques that use machine learning such as deep learning are known as image recognition techniques for identifying subjects and the like that appear in images.

[0004] In this type of image recognition technology, a classifier (e.g., a Convolutional Neural Network; hereinafter also referred to as "CNN") is trained to understand the characteristics of the probability distribution latent in image data by performing machine learning using training images. Typically, a trained classifier can identify image patterns simply by inputting pixel value information from the image.

[0005] Patent Document 1, which relates to the field of analysis of digital images obtained by ultrasound scans, describes a technology for automatically detecting nerves in a series of echo images to assist anesthesiologists in their work. The technology described in Patent Document 1 generates regions that appear to be nerves in the echo images using a probability distribution, and classifies the regions determined by the generated probability distribution as to which nerves they represent. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Special Publication No. 2020-501289 Summary of the Invention [Problem to be solved by the invention]

[0007] Incidentally, there are multiple nerves (for example, the median nerve, ulnar nerve, radial nerve, brachial plexus, etc.) whose image features are different when viewed in an ultrasound image. Furthermore, even the same nerve can look completely different in an image depending on where the ultrasound probe is placed. Furthermore, depending on the person, there are multiple patterns of nerves themselves and other components (blood vessels, fat, muscles) running through the nerves. Even for the same person, the running / arrangement patterns of the nerves themselves and other components (blood vessels, fat, muscles) differ between the left and right sides. As can be seen from the above, there are a large number of nerve image patterns that must be identified.

[0008] When using a classifier (CNN) as an image recognition technology to identify objects in ultrasound images with a large number of image patterns, such as the nerves mentioned above, in order to improve the accuracy of identifying objects, there are a number of possible ways to switch between multiple classifiers optimized for each object. However, this requires complex UI (User Interface) operations to switch between multiple classifiers optimized for individual regions, which makes it difficult for users (e.g., doctors) to perform the procedure they originally intended to perform (e.g., puncture).

[0009] The present invention aims to provide an ultrasound imaging diagnostic device, a classifier changing method, and a classifier changing program that can reduce the effort required to switch classifiers when using a classifier to identify an object to be identified in an ultrasound image. [Means for solving the problem]

[0010] The ultrasound diagnostic imaging device according to the present invention comprises: an ultrasound image acquisition unit that acquires an ultrasound image generated based on a reception signal obtained by an ultrasound probe that transmits and receives ultrasound to and from a subject; a classification result acquisition unit that inputs the acquired ultrasound image into one of a plurality of classifiers that classify objects to be classified that appear in the input ultrasound image, and acquires a classification result output from the classifier; changing the classifier to which the acquired ultrasound image is input according to the inclination or angle of the ultrasound probe; At the timing when the ultrasonic probe is switched from an air radiation state to a state where the ultrasonic probe is in contact with the body of the subject, too a classifier change unit that changes the classifier to which the ultrasound image is input according to the type of object to be classified appearing in the acquired ultrasound image; Equipped with 。

[0011] The classifier change method according to the present invention includes the steps of: Acquiring an ultrasound image generated based on a reception signal obtained by an ultrasound probe that transmits and receives ultrasound to and from the subject; inputting the acquired ultrasound image into one of a plurality of classifiers that classify objects to be identified that appear in the input ultrasound image, thereby obtaining a classification result output from the classifier; changing the classifier to which the acquired ultrasound image is input according to the inclination or angle of the ultrasound probe; At the timing when the ultrasonic probe is switched from an air radiation state to a state where the ultrasonic probe is in contact with the body of the subject, too The classifier to which the ultrasound image is input is changed according to the type of object to be identified that appears in the acquired ultrasound image. 。

[0012] The classifier change program according to the present invention comprises: On the computer, A process of acquiring an ultrasound image generated based on a reception signal obtained by an ultrasound probe that transmits and receives ultrasound to and from the subject; a process of inputting the acquired ultrasound image into one of a plurality of classifiers that classify objects to be identified appearing in the input ultrasound image, thereby obtaining a classification result output from the classifier; changing the classifier to which the acquired ultrasound image is input according to the inclination or angle of the ultrasound probe;At the timing when the ultrasonic probe is switched from an air radiation state to a state where the ultrasonic probe is in contact with the body of the subject, too a process of changing the classifier to which the ultrasound image is input according to the type of object to be classified appearing in the acquired ultrasound image; Run 。 [Effects of the Invention]

[0013] According to the present invention, when a classifier is used to identify an object to be identified that appears in an ultrasound image, the effort required to switch between classifiers can be reduced. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is an external view of an ultrasound diagnostic imaging apparatus. [Figure 2] FIG. 2 is a block diagram showing the functional configuration of the ultrasound diagnostic imaging apparatus. [Figure 3] FIG. 4 is a diagram showing an example of a display image displayed by a display unit. DETAILED DESCRIPTION OF THE INVENTION

[0015] The ultrasound diagnostic imaging apparatus 100 according to this embodiment will be described in detail below with reference to the drawings. FIG.

[0016] 1, an ultrasound diagnostic imaging device 100 includes an ultrasound diagnostic imaging device main body 1 and an ultrasound probe 2. Note that a mobile terminal such as a tablet terminal or a smartphone may be used as the ultrasound diagnostic imaging device main body 1.

[0017] The ultrasonic probe 2 transmits ultrasonic waves (transmitted ultrasonic waves) into a subject such as a living body (not shown), and receives reflected waves of the ultrasonic waves (reflected ultrasonic waves: echoes) reflected within the subject.

[0018] The ultrasound diagnostic imaging device main body 1 is connected to an ultrasound probe 2 via a cable 3, and transmits an electrical drive signal to the ultrasound probe 2, thereby causing the ultrasound probe 2 to transmit ultrasound waves to the subject.

[0019] The ultrasound diagnostic imaging device main body 1 visualizes the internal state of the subject as an ultrasound image based on a received signal, which is an electrical signal generated by the ultrasound probe 2 in response to ultrasound reflected from the subject and received by the ultrasound probe 2. The ultrasound diagnostic imaging device main body 1 also includes an operation input unit 11 and a display unit 18, which will be described later.

[0020] The ultrasonic probe 2 includes transducers 2a (see FIG. 2) each made of a piezoelectric element. A plurality of the transducers 2a are arranged in a one-dimensional array in the azimuth direction (scanning direction), for example. In this embodiment, an ultrasonic probe 2 including, for example, 192 transducers 2a is used.

[0021] The transducers 2a may be arranged in a two-dimensional array. The number of transducers 2a can be set arbitrarily. In this embodiment, a linear electronic scan probe is used as the ultrasound probe 2 to perform ultrasound scanning by a linear scan method, but any of a sector scan method, a convex scan method, a radial scan method, and a circular scan method can also be adopted. Communication between the ultrasound diagnostic imaging device main body 1 and the ultrasound probe 2 may be performed by wireless communication such as UWB (Ultra Wide Band) instead of wired communication via a cable 3.

[0022] Next, the functional configuration of the ultrasound diagnostic imaging apparatus 100 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the functional configuration of the ultrasound diagnostic imaging apparatus 100.

[0023] 2, the ultrasound diagnostic imaging device main body 1 includes an operation input unit 11, a transmission unit 12, a reception unit 13, an image generation unit 14, a discrimination unit 15, a storage unit 16, a confidence factor acquisition unit 17, a display unit 18, and a control unit 19. The confidence factor acquisition unit 17 functions as the "ultrasound image acquisition unit" and the "classification result acquisition unit" of the present invention. The control unit 19 functions as the "classifier change unit" of the present invention.

[0024] The operation input unit 11 includes various switches, buttons, a trackpad, a trackball, a mouse, a keyboard, a touch panel that is integrally provided on the display screen of the display unit 18 and detects touch operations on the display screen, and the like. The operation input unit 11 inputs, for example, commands for attaching / detaching (connecting / disconnecting) the ultrasound probe 2, selecting a diagnostic region (preset), starting / ending diagnosis, selecting a measurement item, starting / ending a device or application, starting / ending freezing, commands for selecting a processor that performs a process of identifying an object to be identified using a classifier, data such as personal information of the subject, and various parameters for displaying an ultrasound image on the display unit 18. Examples of processors that perform a process of identifying an object to be identified using a classifier include a central processing unit (CPU), a graphics processing unit (GPU), and a field programmable gate array (FPGA). The operation input unit 11 outputs an operation signal corresponding to the input operation to the control unit 19.

[0025] The measurement items that can be selected via the operation input unit 11 include morphological measurements using ultrasound images (e.g., length, area, angle, velocity, volume), measurements using brightness values ​​(e.g., histograms), cardiac measurements, gynecological measurements, and obstetric measurements.

[0026] The transmitting unit 12 is a circuit that supplies a driving signal, which is an electric signal, to the ultrasonic probe 2 via the cable 3 under the control of the control unit 19, causing the ultrasonic probe 2 to generate an ultrasonic wave to be transmitted.

[0027] The transmitter 12 also includes, for example, a clock generation circuit, a delay circuit, and a pulse generation circuit. The clock generation circuit is a circuit that generates a clock signal that determines the transmission timing and transmission frequency of the drive signal. The delay circuit is a circuit that sets a delay time for each individual path corresponding to each transducer 2a, and delays the transmission of the drive signal by the set delay time to perform focusing of the transmission beam formed by the transmitted ultrasound (transmission beam forming), etc. The pulse generation circuit is a circuit that generates a pulse signal as a drive signal at a set voltage and time interval.

[0028] The transmitting unit 12 configured as described above sequentially switches among the multiple transducers 2a to which the drive signals are supplied, shifting them by a predetermined number for each transmission and reception of ultrasound waves, in accordance with the control of the control unit 19, and performs scanning by supplying drive signals to the multiple transducers 2a whose outputs are selected.

[0029] The receiving unit 13 is a circuit that receives a reception signal, which is an electrical signal, from the ultrasonic probe 2 via the cable 3 under the control of the control unit 19. The receiving unit 13 includes, for example, an amplifier, an A / D conversion circuit, and a phasing and summing circuit.

[0030] The amplifier is a circuit for amplifying the received signal by a preset amplification factor for each individual path corresponding to each transducer 2a. The A / D conversion circuit is a circuit for analog-to-digital conversion (A / D conversion) of the amplified received signal. The phasing addition circuit is a circuit for adjusting the time phase by applying a delay time to the A / D converted received signal for each individual path corresponding to each transducer 2a, and adding these (phasing addition) to generate sound ray data. In other words, the phasing addition circuit performs receive beamforming on the received signal for each transducer 2a to generate sound ray data.

[0031] The image generator 14, under the control of the controller 19, performs envelope detection processing, logarithmic compression, and the like on the sound ray data from the receiver 13, adjusts the dynamic range and gain, and performs brightness conversion to generate B-mode image data (hereinafter referred to as ultrasound image data) as two-dimensional tomographic image data. That is, the ultrasound image data represents the strength of the received signal by brightness. The image generator 14 outputs the generated ultrasound image data to the discriminator 15, the certainty factor acquirer 17, and the controller 19. The image generator 14 may also generate A-mode image data, M-mode image data (two-dimensional tomographic image data), Doppler image data, color mode image data, or three-dimensional image data.

[0032] The image generating unit 14 also includes an image memory unit (not shown) configured with a semiconductor memory such as a DRAM (Dynamic Random Access Memory). The image generating unit 14 stores the generated ultrasound image data in the image memory unit on a frame-by-frame basis.

[0033] The image generating unit 14 also performs image processing such as image filtering and time smoothing on the ultrasound image data read out from the image memory unit, and scan-converts the data into a display image pattern for display on the display unit 18 .

[0034] The discrimination unit 15 analyzes the ultrasound image data output from the image generation unit 14 and discriminates the image pattern of the ultrasound image corresponding to the ultrasound image data. Specifically, the discrimination unit 15 discriminates the body parts shown in the ultrasound image and determines a state in which the ultrasound probe 2 is not in contact with the body (air radiation state). The discrimination unit 15 then outputs discrimination result information indicating the discrimination result of the ultrasound image to the control unit 19. The discrimination result information includes body part information indicating the limbs, neck, waist, etc., anatomical information indicating nerves (median nerve, ulnar nerve, radial nerve, brachial plexus), muscles, blood vessels, etc., and information indicating whether or not the air radiation state is in effect.

[0035] The storage unit 16 is a storage unit capable of writing and reading information, such as a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0036] In this embodiment, the memory unit 16 stores a plurality of classifiers (convolutional neural networks: CNNs) used by the confidence acquisition unit 17 to identify objects to be identified that appear in the ultrasound images generated by the image generation unit 14.

[0037] Here, the object to be identified is a nerve, fascia, muscle, blood vessel, needle, heart, placenta, lymph node, brain, prostate, carotid artery, or breast, and may be not only a detailed structure but also an organ itself, a body structure such as a limb, face, neck, or waist, a lesion indicating some kind of disease, or an abnormal brightness area in an ultrasound image. The storage unit 16 stores a plurality of classifiers optimized for each object to be identified so as to increase the accuracy of identifying the object to be identified.

[0038] A convolutional neural network is a type of forward propagation neural network based on knowledge of the structure of the brain's visual cortex. Its basic structure consists of a convolutional layer, which extracts local image features, and a pooling layer (subsampling layer), which aggregates local features. Each layer of a convolutional neural network contains multiple neurons, arranged in a way that corresponds to the visual cortex. The basic function of each neuron is to input and output signals. However, when transmitting signals between neurons in each layer, rather than simply outputting the input signal, a connection weight is assigned to each input. When the sum of these weighted inputs exceeds a threshold value set for each neuron, a signal is output to the neuron in the next layer. The connection weights between these neurons are calculated from training data. This enables output values ​​to be estimated by inputting real-time data. Well-known convolutional neural network models include GoogleNet, ResNet, SENet, U-Net, and MobileNet. However, the algorithms that constitute the convolutional neural network are not particularly limited as long as it is suitable for this purpose.

[0039] The classifier is trained in advance to input ultrasound image data generated by the image generation unit 14 and output a confidence level representing the likelihood that an object to be identified (specifically, the position, boundary, or area of ​​the object to be identified) appears in the ultrasound image corresponding to the ultrasound image data, for each predetermined portion constituting the ultrasound image. In this embodiment, the confidence level output from the classifier is expressed as a value greater than 0 and equal to or less than 1. A higher confidence level means a higher likelihood that an object to be identified appears in the ultrasound image.

[0040] The learning process for the classifier is performed by well-known supervised learning, specifically, by adjusting network parameters including weight coefficients, biases, etc. using backpropagation (error backpropagation) based on previously prepared training data. The training data is a pair of previously generated ultrasound image data and corresponding ground truth data. Examples of the ground truth data include a labeled image in which a desired area in an ultrasound image corresponding to the ultrasound image data is labeled with an arbitrary value, coordinate data indicating the desired area with coordinates, or an equation of a line or curve indicating the boundary of the desired area. In this embodiment, a labeled image is used as the ground truth data, and in the ultrasound image serving as training data, parts in which the object to be identified is shown are labeled with "1" and parts in which the object to be identified is not shown are labeled with "0".

[0041] The classifier is not limited to a convolutional neural network, and a mathematical model including a calculation algorithm and coefficients may be used. Furthermore, the storage unit 16 may include a classifier (a general-purpose model, corresponding to the "first classifier" of the present invention) trained to output a confidence level indicating the likelihood that multiple identification objects (corresponding to multiple general-purpose parts (classes)) appear in the input ultrasound image, and a classifier (a dedicated model, corresponding to the "second classifier" of the present invention) trained to output a confidence level indicating the likelihood that a specific identification object (corresponding to a specific part) appears in the input ultrasound image. Furthermore, the storage unit 16 may store, as classifiers used by the confidence level acquisition unit 17 to identify the same identification object, a responsiveness-oriented model classifier that prioritizes responsiveness (classification processing time, real-time performance) over classification accuracy, and a classification accuracy-oriented model classifier that prioritizes classification accuracy over responsiveness.

[0042] The certainty factor acquiring unit 17 reads out from the storage unit 16 the classifier specified by the classifier specifying information output from the control unit 19, and inputs the ultrasound image data generated by the image generating unit 14 to the classifier. Then, the certainty factor acquiring unit 17 acquires the certainty factor output from the classifier for each predetermined region constituting the ultrasound image corresponding to the ultrasound image data, and outputs certainty factor information indicating the acquired certainty factor to the display unit 18.

[0043] Display unit 18 is a display device such as an LCD (Liquid Crystal Display), a CRT (Cathode-Ray Tube) display, an organic EL (Electronic Luminescence) display, an inorganic EL display, a plasma display, etc. Display unit 18 displays, under the control of control unit 19, an ultrasound image corresponding to the ultrasound image data generated by image generation unit 14 and a display image based on the certainty factor information output from certainty factor acquisition unit 17 on a display screen.

[0044] Fig. 3 is a diagram showing an example of a display image displayed by the display unit 18. Fig. 3A is a diagram showing an example of an ultrasound image corresponding to ultrasound image data as a display image. Fig. 3B is a diagram showing an example of a superimposed image in which boundary images 20, 21 showing the boundary of an object to be identified are superimposed on an ultrasound image corresponding to the ultrasound image data as a display image. The boundary of the object to be identified represented by boundary images 20, 21 is composed of parts for which the certainty indicated by the certainty information output from the certainty acquisition unit 17 is equal to or greater than a predetermined value, i.e., parts which are highly likely to depict the boundary of the object to be identified.

[0045] 3C is a diagram showing an example of a superimposed image in which region images 22 and 23 indicating the region of the object to be identified are superimposed on an ultrasound image corresponding to the ultrasound image data as a display image. The region of the object to be identified represented by region images 22 and 23 is composed of parts for which the certainty indicated by the certainty information output from certainty acquiring unit 17 is equal to or greater than a predetermined value, i.e., parts that are highly likely to depict the region of the object to be identified.

[0046] 3D is a diagram showing an example of a superimposed image in which position images 24 and 25 indicating the position of the identification target with an X mark are superimposed on an ultrasound image corresponding to ultrasound image data. The position of the identification target represented by position images 24 and 25 is composed of a portion for which the certainty indicated by the certainty information output from the certainty factor acquisition unit 17 is equal to or greater than a predetermined value, i.e., a portion that is highly likely to represent the position of the identification target. As shown in FIG. 3E, a superimposed image in which position images 26 and 27 indicating the position of the identification target with an arrow on an ultrasound image corresponding to ultrasound image data may be superimposed on a display screen as a display image. Furthermore, although not shown, a superimposed image in which a position image indicating the position of the identification target with a dot is superimposed on an ultrasound image corresponding to ultrasound image data may be displayed as a display image as a display image.

[0047] By checking the superimposed image in which the identification object is highlighted on the display screen, the user can intuitively grasp the location, boundary, or area of ​​the identification object in the ultrasound image. For example, by highlighting nerve structures such as peripheral nerves on the display screen, the user can easily grasp nerve areas that are difficult for even specialists to grasp in normal ultrasound images, thereby facilitating procedures such as puncture. Alternatively, by highlighting organs such as the fetal brain, the user can easily grasp the developmental state of organs that are difficult to grasp, thereby easily determining whether or not there are any congenital or acquired problems.

[0048] The control unit 19 includes, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory), and reads out various processing programs such as system programs stored in the ROM, expands them into the RAM, and centrally controls the operations of each part of the ultrasound imaging diagnostic device main body 1 in accordance with the expanded programs.

[0049] The ROM is configured with nonvolatile memory such as a semiconductor, and stores a system program corresponding to the ultrasound diagnostic imaging apparatus 100, various processing programs executable on the system program, and various data such as a gamma table. These programs are stored in the form of computer-readable program code, and the CPU sequentially executes operations in accordance with the program code. The RAM forms a work area for temporarily storing various programs executed by the CPU and data related to these programs. In this embodiment, a "classifier change program" is stored in the ROM of the control unit 19.

[0050] The control unit 19 selects, in the confidence level acquisition unit 17, a classifier to which the ultrasound image data generated by the image generation unit 14 is input. The control unit 19 then outputs classifier identification information that identifies the selected classifier to the confidence level acquisition unit 17. Here, selecting a classifier to which the ultrasound image data is input includes changing only a part of the classifier or changing the entire classifier. When changing only a part of the classifier, it is possible to change only the coefficients (parameters) used in the calculation algorithm of the classifier without changing the calculation algorithm of the classifier. In this way, when changing only a part of the classifier, the speed of switching the classifier increases, and the delay occurring before switching to a new classifier can be reduced.

[0051] For example, the control unit 19 changes the classifier to which ultrasound image data is input in the certainty factor acquisition unit 17, in accordance with an operation input to the operation input unit 11 (an operation input by the user to the ultrasound diagnostic imaging apparatus 100). For example, if the user does not directly specify a classifier via the operation input unit 11 (for example, when only a diagnostic region is specified, or when an instruction is given to start / end an application, start / end of freezing, etc.), the control unit 19 determines a classifier to which ultrasound image data is input in the certainty factor acquisition unit 17, in accordance with the operation input to the operation input unit 11. For example, when the lower back is specified as the diagnostic region, the control unit 19 determines a classifier that identifies the sciatic nerve (object to be identified) as the classifier to which ultrasound image data is input in the certainty factor acquisition unit 17. This allows the classifier to be changed (switched) as appropriate in a natural diagnostic workflow that the user normally performs (for example, selection of a diagnostic region, selection of preset examination conditions, etc.), thereby eliminating the user's need to switch classifiers one by one.

[0052] Furthermore, when the user directly specifies a classifier via the operation input unit 11, the control unit 19 determines the specified classifier as the classifier to which the ultrasound image data will be input in the confidence factor acquisition unit 17. In this case, a superimposed image in which the object to be identified identified by the specified classifier is superimposed on the ultrasound image is displayed on the display screen, allowing the user to intuitively grasp the portion of the ultrasound image in which the position, boundary, or area of ​​the object to be identified is captured.

[0053] The control unit 19 may also determine a classifier other than the directly specified classifier as the classifier to which ultrasound image data is input in the certainty factor acquisition unit 17. In this case, a superimposed image in which the object to be identified by the other classifier is superimposed on the ultrasound image is displayed on the display screen, that is, the supplementary object to be identified by the other classifier is also highlighted, allowing the user to more intuitively grasp the portion of the ultrasound image in which the position, boundary, or area of ​​the object to be identified is captured. The control unit 19 may also determine a classifier that was previously specified (for example, a classifier that was specified when the diagnosis was completed or frozen) as the classifier to which ultrasound image data is input in the certainty factor acquisition unit 17.

[0054] Furthermore, the control unit 19 changes the classifier to which the ultrasound image data is input in the certainty factor acquisition unit 17, depending on the type of identification object appearing in the ultrasound image corresponding to the ultrasound image data generated by the image generation unit 14. Specifically, the control unit 19 references the discrimination result information output from the discrimination unit 15 and changes the classifier to which the ultrasound image data is input in the certainty factor acquisition unit 17, depending on the type of identification object (e.g., body part) appearing in the ultrasound image. This allows the classifier to be switched appropriately even if the user unexpectedly changes the diagnostic part, thereby eliminating the user's need to switch classifiers one by one. Furthermore, there is no risk of the user being shown an inappropriate identification result on the display screen, preventing the user from making a misdiagnosis. The control unit 19 may also change the classifier to which the ultrasound image data is input in the certainty factor acquisition unit 17, depending on the type of identification object appearing in the ultrasound image and the operation input to the operation input unit 11.

[0055] The control unit 19 refers to the discrimination result information output from the discrimination unit 15, and changes the classifier to which the ultrasound image data is input in the confidence level acquisition unit 17, depending on the type of object to be identified (e.g., a body part) shown in the ultrasound image, when the state switches from an air-emitting state to a state in which the ultrasound probe 2 is in contact with the body. The user may move the ultrasound probe 2 away from the body when changing the diagnostic part or when it feels difficult to see the desired body part. In such cases, by appropriately changing the classifier when the state switches from an air-emitting state to a state in which the ultrasound probe 2 is in contact with the body, the user is saved from the trouble of switching the classifier each time, and can naturally intuitively grasp the part of the ultrasound image displayed on the display screen that shows the position, boundary, or area of ​​the object to be identified.

[0056] The control unit 19 may refer to the discrimination result information output from the discrimination unit 15 and start / end the discrimination process of the classifier in the confidence level acquisition unit 17 or start / end freezing. This makes it possible to prevent the discrimination process of the classifier and the display of ultrasound images, etc. from being performed if the user accidentally changes the diagnostic region or moves the ultrasound probe 2 away from the body, thereby eliminating the risk of the user being shown an inappropriate discrimination result on the display screen and preventing the user from making a misdiagnosis. Furthermore, by preventing the discrimination process of the classifier and the display of ultrasound images, etc. from being performed unnecessarily, power consumption of the ultrasound diagnostic imaging device 100 can be reduced.

[0057] Furthermore, the control unit 19 may refer to the discrimination result information output from the discrimination unit 15, and if the airborne radiation state continues for a certain period of time, reset the discrimination result of the classifier up to that point and return to the default (preset) classifier. If the ultrasound probe 2 is removed from the body and then placed on the body again, the area diagnosed may be different from the area previously diagnosed. If the location where the ultrasound probe 2 is placed changes, the characteristics of the ultrasound image change significantly. Therefore, by selecting a classifier (general-purpose model) that corresponds to various areas, for example, when the airborne radiation state has continued for a certain period of time, the user can quickly grasp the anatomical structure in the ultrasound image displayed on the display screen without having to perform an operation to select an unnecessary classifier.

[0058] Furthermore, the control unit 19 changes the classifier to which the ultrasound image data is input in the certainty factor acquisition unit 17, depending on the type of ultrasound probe 2 (linear scanning, sector scanning, convex scanning, radial scanning, or circular scanning). Since the image size, frame rate, and image quality that can be rendered differ depending on the type of ultrasound probe 2, by optimizing the classifier depending on the type of ultrasound probe 2, the certainty factor acquisition unit 17 can acquire stable classification results even if the ultrasound probe 2 used by the user is changed each time.

[0059] Furthermore, the control unit 19 changes the classifier to which the ultrasound image data is input in the confidence factor acquisition unit 17, depending on the classification result of the classifier (e.g., maximum confidence factor, average confidence factor, mode, standard deviation, etc.). For example, the control unit 19 changes to a classifier with a higher confidence factor (higher classification accuracy) depending on the classification result of the classifier. This allows the confidence factor acquisition unit 17 to acquire a highly accurate classification result. Furthermore, the control unit 19 changes to a classifier with the most stable confidence factor over time depending on the classification result of the classifier. A stable confidence factor over time indicates high classification accuracy, so a highly accurate classification result (superimposed image) can be displayed on the display screen. Furthermore, classification results with unstable confidence factors over time will not be unnecessarily shown to the user, preventing a degradation in the user's diagnostic workflow.

[0060] Furthermore, the control unit 19 changes the classifier to which the ultrasound image data is input in the confidence level acquisition unit 17, depending on the processing time for the classifier's classification. Here, the processing time for the classifier's classification refers to the processing time from the start to the end of the classifier's classification process, or the time from the start to the end of the classifier's classification process until the classification result is displayed superimposed on the ultrasound image (including communication delay time). For example, the control unit 19 changes to a classifier with higher responsiveness (real-time performance) depending on the processing time for the classifier's classification. This improves the classifier's classification responsiveness without compromising the classifier's classification accuracy, thereby improving the user's diagnostic workflow and enabling the user to take prompt action.

[0061] As described above in detail, in this embodiment, the ultrasound diagnostic imaging device 100 includes an ultrasound image acquisition unit (certainty factor acquisition unit 17) that acquires an ultrasound image generated based on a reception signal obtained by the ultrasound probe 2 that transmits and receives ultrasound to a subject, a classification result acquisition unit (certainty factor acquisition unit 17) that inputs the acquired ultrasound image to one of a plurality of classifiers that identify an object to be classified appearing in the input ultrasound image, thereby acquiring a classification result output from the classifier, and a classifier change unit (control unit 19) that changes the classifier to which the acquired ultrasound image is input.

[0062] According to the present embodiment configured as described above, the classifier to which the ultrasound image is input is changed and optimized depending on, for example, the body part of the user's interest, the measurement target, the processor performing the classification processing, the type of ultrasound probe 2, etc., thereby achieving both high classification accuracy for the object to be classified in the ultrasound image and real-time display (displaying the results of image processing as quickly as possible). Furthermore, since the classifier can be switched appropriately within the user's natural diagnostic workflow (e.g., selection of the diagnostic target, selection of preset examination conditions, start / release of freeze, etc.), the user is freed from the hassle of switching classifiers one by one and can concentrate on the procedure they are actually performing. Furthermore, the user does not need to search for the optimal classifier, improving real-time display and reducing unnecessary processing within the ultrasound diagnostic imaging device 100. This also improves the responsiveness and accuracy of other processing performed in parallel (e.g., color Doppler processing, needle tip enhancement processing, etc.).

[0063] In the above embodiment, if the classifier to which the ultrasound image data is input in the certainty factor acquisition unit 17 is not appropriate, or if the classification process of the classifier is not completed by the required timing (i.e., not in time), an image showing the object to be classified may not be superimposed on the ultrasound image corresponding to the ultrasound image data, but a superimposed image in which the image showing the object to be classified is superimposed on the ultrasound image at an appropriate timing in the next or subsequent frame may be displayed. This prevents unnecessary information from being shown to the user, thereby preventing a degradation in the user's diagnostic workflow.

[0064] Furthermore, in the above embodiment, when a superimposed image in which an image showing the boundary, area, and position of the object to be identified is superimposed on an ultrasound image corresponding to ultrasound image data is displayed on the display screen as a display image, the type of object to be identified (e.g., nerve) identified by the discrimination result of discrimination unit 15 may also be displayed together.

[0065] Furthermore, in the above embodiment, the certainty factor acquisition unit 17 may be provided on a separate server (a separate computer) rather than on the ultrasound imaging diagnostic device 100. In this case, the ultrasound imaging diagnostic device 100 and the separate server correspond to the "ultrasound imaging diagnostic device" of the present invention. The processor on the ultrasound imaging diagnostic device 100 (especially an older model) may not have sufficiently high processing performance, and performing classification processing using a certain classifier may not meet user requirements in terms of processing speed or classification accuracy. Therefore, the certainty factor acquisition unit 17 provided on the separate server (a computer with a higher-spec processor) receives ultrasound image data and the classifier selected by the control unit 19, performs classification processing using the selected classifier, and transmits the classification results to the ultrasound imaging diagnostic device 100, where they are displayed on the display unit 18. This allows for more accurate classification results to be displayed more quickly.

[0066] Furthermore, in the above embodiment, the control unit 19 may change the classifier to which the ultrasound image data is input in the certainty factor acquisition unit 17, depending on the communication method (wired communication or wireless communication) between the ultrasound diagnostic imaging device main body 1 and the ultrasound probe 2. Since the communication speed differs depending on the communication method between the ultrasound diagnostic imaging device main body 1 and the ultrasound probe 2, by changing the classifier depending on the communication method, it is possible to achieve both high accuracy in identifying the object to be identified in the ultrasound image and real-time display.

[0067] Furthermore, in the above embodiment, the control unit 19 may change the classifier to which the ultrasound image data is input in the confidence factor acquisition unit 17, depending on the tilt / angle of the ultrasound probe 2. Since the ultrasound image changes depending on the angle at which the ultrasound probe 2 contacts the body, by optimizing the classifier used in accordance with the angle of the ultrasound probe 2, it is possible to obtain stable classification results from the classifier for various ways in which the ultrasound probe 2 is contacted.

[0068] Furthermore, in the above embodiment, the control unit 19 may change the classifier to which the ultrasound image data is input in the confidence factor acquisition unit 17, depending on information about the subject (for example, the patient's name, age, sex, BMI, medical history of injuries or illnesses, etc.). The physical characteristics of the subject patient vary depending on their age, sex, BMI, and medical history, which results in different characteristics of the ultrasound image. Therefore, by changing the classifier depending on the subject information, it is possible to achieve both high accuracy in identifying objects appearing in ultrasound images and real-time display, even for subjects with different physical characteristics.

[0069] Furthermore, in the above embodiment, the control unit 19 may change the classifier to which the ultrasound image data is input in the certainty factor acquisition unit 17, depending on information about the user using the ultrasound diagnostic imaging device 100 (for example, which doctor or technician is currently using the device). Since the way the ultrasound probe 2 is applied varies from user to user, the ultrasound image changes depending on the user. Therefore, by changing the classifier depending on information about the user using the ultrasound diagnostic imaging device 100, the accuracy of identifying objects shown in the ultrasound image and the real-time display performance can be optimized for each user.

[0070] Furthermore, in the above embodiment, the control unit 19 may first select a classifier that identifies thick nerves (objects to be identified) to grasp the outline of the object to be identified, and then change to a classifier that identifies finer nerves (objects to be identified) once the cross section to be viewed has been determined to some extent. This allows the user to check on the display screen a superimposed image in which an image showing the object to be identified is superimposed on the ultrasound image, and to more reliably identify the finer nerves by tracing from thicker nerves to thinner regions rather than immediately searching for finer nerves.

[0071] Furthermore, in the above embodiment, the control unit 19 may initially select a classifier (general-purpose model) compatible with multiple general-purpose regions (classes), and after performing a scan for a while, select a classifier (dedicated model) specialized for a specific region (class). In this case, the display unit 18 may display candidate classifiers (dedicated models) on the display screen so that the user can select one. If the ultrasound probe 2 is not placed properly, the desired region may not be imaged in the first place, and even a dedicated model (classifier) ​​specialized for a certain region may not be able to accurately identify the region. Therefore, using a dedicated model from the beginning may not be appropriate. This is because, particularly for beginners, it may be more appropriate to display more information, such as the classification results for multiple regions, even if the accuracy is somewhat lower, than to display information on limited regions using a dedicated model, because it facilitates anatomical understanding and allows the user to quickly grasp the approximate location of the desired region, i.e., the position where the ultrasound probe 2 should be placed. Therefore, when the ultrasonic probe 2 is first applied, the identification results are displayed for multiple areas, and then when the cross section has been fixed, that is, when the identification target has been narrowed down, the system switches to a dedicated model, allowing the user to smoothly grasp the desired area (identification target).

[0072] Furthermore, in the above-described embodiment, with respect to the transmitter 12, receiver 13, image generator 14, discriminator 15, confidence factor acquirer 17, and controller 19 included in the ultrasound diagnostic imaging apparatus 100, some or all of the functions of each functional block can be implemented as a hardware circuit such as an integrated circuit. An example of an integrated circuit is an LSI (Large Scale Integration), which is also referred to as an IC, system LSI, super LSI, or ultra LSI depending on the level of integration. Furthermore, the integrated circuit implementation is not limited to LSI; it can also be implemented using a dedicated circuit or a general-purpose processor, or an FPGA or a reconfigurable processor that allows the connections and settings of circuit cells within an LSI to be reconfigured. Furthermore, some or all of the functions of each functional block can also be implemented by software. In this case, the software is stored in one or more storage media such as a ROM, an optical disk, or a hard disk, and is executed by a processor.

[0073] Furthermore, the above-described embodiments are merely examples of specific embodiments for carrying out the present invention, and the technical scope of the present invention should not be construed as being limited by these embodiments. In other words, the present invention can be carried out in various forms without departing from the gist or main features thereof. [Explanation of symbols]

[0074] 1. Ultrasound diagnostic imaging device 2 Ultrasonic probe 2a Oscillator 3 Cable 11 Operation input section 12 Transmitter 13 Receiving unit 14 Image generation unit 15 Discrimination part 16 Memory section 17 Confidence acquisition part 18 Display 19 Control Unit 100 Ultrasound imaging diagnostic device

Claims

1. an ultrasound image acquisition unit that acquires an ultrasound image generated based on a reception signal obtained by an ultrasound probe that transmits and receives ultrasound to and from a subject; a classification result acquisition unit that inputs the acquired ultrasound image into one of a plurality of classifiers that classify objects to be classified that appear in the input ultrasound image, and acquires a classification result output from the classifier; a classifier change unit that changes the classifier to which the acquired ultrasound image is input in accordance with the inclination or angle of the ultrasound probe, and changes the classifier to which the ultrasound image is input in accordance with the type of object to be identified appearing in the acquired ultrasound image even when the ultrasound probe is switched from an air radiation state to a state in which the ultrasound probe is in contact with the body of the subject; An ultrasound diagnostic imaging device comprising:

2. the classifier change unit changes the classifier to which the acquired ultrasound image is input in response to an operation input by a user to the ultrasound diagnostic imaging apparatus. The ultrasound diagnostic imaging apparatus according to claim 1 .

3. the classifier change unit changes the classifier to which the acquired ultrasound image is input, depending on the type of the ultrasound probe. The ultrasound diagnostic imaging apparatus according to claim 1 .

4. the classifier change unit changes the classifier to which the acquired ultrasound image is input, depending on the classification result of the classifier. The ultrasound diagnostic imaging apparatus according to claim 1 .

5. the classifier change unit changes the classifier to which the acquired ultrasound image is input, depending on a processing time for classification of the classifier. The ultrasound diagnostic imaging apparatus according to claim 1 .

6. The classifier is trained to output a confidence level indicating the likelihood that an object to be identified is captured on an input ultrasound image. The ultrasound diagnostic imaging apparatus according to any one of claims 1 to 5.

7. The plurality of classifiers a first classifier that is trained to output a degree of certainty that indicates the likelihood that a plurality of objects to be identified are captured on an input ultrasound image; a second classifier that is trained to output a confidence level that indicates the likelihood that a specific object to be identified is captured on the input ultrasound image; The ultrasound diagnostic imaging apparatus according to claim 6.

8. a display unit that displays a display image based on the acquired ultrasound image and the identification result; 7. The ultrasound diagnostic imaging apparatus according to claim 1.

9. Acquiring an ultrasound image generated based on a reception signal obtained by an ultrasound probe that transmits and receives ultrasound to and from the subject; inputting the acquired ultrasound image into one of a plurality of classifiers that classify objects to be identified in the input ultrasound image, thereby obtaining a classification result output from the classifier; The classifier to which the acquired ultrasound image is input is changed according to the inclination or angle of the ultrasound probe, and the classifier to which the ultrasound image is input is changed according to the type of object to be identified appearing in the acquired ultrasound image even when the state is switched from an air radiation state to a state in which the ultrasound probe is in contact with the body of the subject. How to change the classifier.

10. On the computer, A process of acquiring an ultrasound image generated based on a reception signal obtained by an ultrasound probe that transmits and receives ultrasound to and from the subject; a process of inputting the acquired ultrasound image into one of a plurality of classifiers that classify objects to be identified appearing in the input ultrasound image, thereby obtaining a classification result output from the classifier; a process of changing the classifier to which the acquired ultrasound image is input in accordance with the inclination or angle of the ultrasound probe, and also changing the classifier to which the ultrasound image is input in accordance with the type of object to be identified appearing in the acquired ultrasound image even when the state is switched from an air radiation state to a state in which the ultrasound probe is in contact with the body of the subject; A classifier change program that executes the above.

Images