Ultrasonic diagnostic apparatus, data management system, data estimation method, and program

Abstract

To enable even a user who is not used to performing diagnosis to appropriately perform diagnosis from images of a variety of image quality.SOLUTION: An ultrasonic diagnostic apparatus includes: a first ultrasonic probe for transmitting / receiving an ultrasonic wave to / from a subject; and an estimation unit. The estimation unit uses a learned model that has performed machine learning using leaning data. The learning data includes: first data based on a reception signal for image generation which has been received by a second ultrasonic probe of the same kind as the first ultrasonic probe; and second data imaged on a predetermined condition. A control part 18 converts third data based on the reception signal for image generation received by the first ultrasonic probe to fourth data similar to the second data and outputs the converted data. The third data is original image data of an original image 81 using the first ultrasonic probe. The fourth data is conversion image data of a conversion image 82 similar to the second data.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to an ultrasonic diagnostic apparatus, a data management system, a data estimation method, and a program. [Background technology]

[0002] Conventionally, there has been known an ultrasonic diagnostic apparatus that uses an ultrasonic probe to irradiate ultrasonic waves into the inside of a subject, receives and analyzes the reflected waves, and displays an ultrasonic image of the inside of the subject. The subject is, for example, a living body of a patient.

[0003] In recent years, ultrasound diagnostic devices have been used for various purposes using AI (Artificial Intelligence) technology. Specifically, ultrasound diagnostic devices automatically identify (recognize) target organs / tissues from ultrasound image data, and apply the results of the identification to automatic measurements.

[0004] For example, an ultrasound imaging diagnostic device is known that obtains a classification result output from a classifier that identifies a classification object captured in an ultrasound image (see Patent Document 1). The classification objects are organs, body structures, diseased areas, and abnormal brightness areas. The ultrasound imaging diagnostic device changes the classifier to be used from among multiple classifiers depending on the body part of the user's interest, the measurement target, the processor that performs the classification process, and the type of ultrasound probe. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2022-172565 Summary of the Invention [Problem to be solved by the invention]

[0006] Conventional ultrasound diagnostic devices display ultrasound images with various image quality depending on the ultrasound probe, device, manufacturer, and transmission / reception settings (frequency). However, inexperienced users such as new doctors and paramedicals may not know how to view the image when it changes due to the reasons mentioned above, which may make it difficult to make a diagnosis. For the reasons mentioned above, ultrasound diagnostic devices are said to be a modality that is highly dependent on the examination skills of the user. Variations in examination skills carry a high risk of leading to variations in diagnosis.

[0007] The ultrasound imaging diagnostic device of Patent Document 1 displays ultrasound images in various image quality depending on the ultrasound probe, etc., and notifies the AI's identification result of the ultrasound image. However, the user must understand the image content according to the one displayed ultrasound image, judge whether the AI's identification result is appropriate, and make a final diagnosis.

[0008] An object of the present invention is to enable even a user who is inexperienced in diagnosis to make an appropriate diagnosis from images of various image qualities. [Means for solving the problem]

[0009] In order to solve the above problem, the ultrasonic diagnostic apparatus according to the invention described in claim 1 comprises: A first ultrasonic probe for transmitting and receiving ultrasonic waves to and from a subject; a first estimation unit that converts third data based on a reception signal for image generation received by the first ultrasonic probe into fourth data similar to the second data and outputs the fourth data by using a trained model that is machine-learned using training data including first data based on a reception signal for image generation received by a second ultrasonic probe of the same type as the first ultrasonic probe and second data captured under a predetermined condition; Equipped with.

[0010] The invention described in claim 2 is the ultrasonic diagnostic apparatus described in claim 1, The first data and the third data are a predetermined received signal or B-mode image data based on the received signal.

[0011] The invention described in claim 3 is the ultrasonic diagnostic apparatus described in claim 1 or 2, The second data includes at least one of ultrasound data, MRI data, and CT data.

[0012] The invention described in claim 4 is the ultrasonic diagnostic apparatus described in claim 1 or 2, The fourth data is ultrasound data, MRI-like image data, or CT-like image data.

[0013] The invention described in claim 5 is the ultrasonic diagnostic apparatus described in claim 1 or 2, the first data is data based on a reception signal for generating an image received by the second ultrasonic probe, The second data is data based on a reception signal for generating an image received by a third ultrasonic probe that is a different type from the second ultrasonic probe.

[0014] The invention described in claim 6 is the ultrasonic diagnostic apparatus according to claim 1 or 2, The image processing device further includes an output unit that outputs information indicating that the fourth data is a processed image.

[0015] The invention described in claim 7 is the ultrasonic diagnostic apparatus described in claim 1 or 2, The apparatus further includes an output unit that outputs at least one of information on the type of ultrasonic probe, the type of modality, and information on frequency corresponding to the fourth data.

[0016] The invention described in claim 8 is the ultrasonic diagnostic apparatus described in claim 1 or 2, The image processing device further includes an output unit that outputs information indicating that the third data is an original image.

[0017] The invention described in claim 9 is the ultrasonic diagnostic apparatus described in claim 1 or 2, The device further includes an output section that outputs the third data or outputs the third data and the fourth data in association with each other.

[0018] The invention described in claim 10 is the ultrasonic diagnostic apparatus described in claim 1 or 2, the third data is third image data and the fourth data is fourth image data; The display unit displays the third image data and the fourth image data on the same screen or switches between them.

[0019] The invention described in claim 11 is the ultrasonic diagnostic apparatus described in claim 1 or 2, The trained model is A first trained model trained by machine learning using training data including the first data and the second data; a second trained model that is machine-learned using training data that includes fifth data based on a reception signal for image generation received by a second ultrasonic probe of the same type as the first ultrasonic probe and sixth data indicating an attribute of the fifth data; A switching means for switching to the output of one of the plurality of trained models by a user operation, The first estimation unit converts the third data into fourth data similar to the second data using a first trained model selected according to the attributes of the third data identified by the second trained model and the switching means, and outputs the fourth data.

[0020] The invention described in claim 12 is the ultrasonic diagnostic apparatus described in claim 11, The switching means includes a plurality of switching units, The device further includes a selection unit that identifies third data and selects one of the plurality of switching units.

[0021] The invention described in claim 13 is the ultrasonic diagnostic apparatus described in claim 1 or 2, The apparatus further includes a second estimation unit that outputs fifth data from the fourth data by using a classifier trained by machine learning using training data including a data set of fourth data obtained by converting the first data using the trained model and a predetermined correct answer label.

[0022] The data management system of the invention described in claim 14 comprises: The ultrasonic diagnostic apparatus according to claim 1 or 2, and a management device that receives and stores the third data, or the third data and the fourth data, from the ultrasonic diagnostic device.

[0023] The data estimation method of the invention described in claim 15 comprises: an estimation step of converting third data based on a reception signal for generating an image received by a first ultrasonic probe into fourth data similar to the second data and outputting the fourth data by using a trained model trained by machine learning using training data including first data based on a reception signal for generating an image received by a second ultrasonic probe of the same type as a first ultrasonic probe that transmits and receives ultrasonic waves to a subject, and second data captured under predetermined conditions; Includes.

[0024] The program of the invention described in claim 16 comprises: Computer, an estimation unit that converts third data based on a reception signal for generating an image received by a first ultrasonic probe into fourth data similar to the second data and outputs the fourth data, using a trained model that is machine-learned using training data including first data based on a reception signal for generating an image received by a second ultrasonic probe of the same type as a first ultrasonic probe that transmits and receives ultrasonic waves to a subject, and second data captured under predetermined conditions; Function as. Effect of the Invention

[0025] According to the present invention, even a user who is inexperienced in diagnosis can make an appropriate diagnosis from images of various image qualities. [Brief description of the drawings]

[0026] [Figure 1] 1 is a block diagram showing a schematic configuration of an ultrasound diagnostic system according to an embodiment of the present invention; [Diagram 2]FIG. 2 is a block diagram showing the functional configuration of the ultrasound diagnostic apparatus. [Diagram 3] FIG. 1 is a diagram illustrating the configuration of a GAN. [Figure 4] A diagram showing the configuration of Cycle-GAN. [Diagram 5] 13 is a flowchart showing a first ultrasound image output process. [Figure 6] FIG. 11 is a diagram showing data and processing of a first ultrasound image output processing. [Figure 7] FIG. [Figure 8] (a) is a diagram showing an ultrasound image, and (b) is a diagram showing an MRI-like image. [Figure 9] 13 is a flowchart showing a second ultrasound image output process. [Figure 10] 13 is a flowchart showing a third ultrasound image output process. [Figure 11] FIG. 13 is a diagram showing data and processing of the third ultrasound image output processing. [Figure 12] 13 is a flowchart showing a fourth ultrasound image output process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] Hereinafter, a first embodiment, a modified example thereof, a second embodiment, and a third embodiment of the present invention will be described in detail in this order with reference to the drawings. However, the scope of the present invention is not limited to the illustrated examples.

[0028] (First embodiment) A first embodiment of the present invention will be described with reference to Figs. 1 to 8(b). First, the device configuration of this embodiment will be described with reference to Figs. 1 to 3. Fig. 1 is a block diagram showing a schematic configuration of an image management system 100 of this embodiment. Fig. 2 is a block diagram showing the functional configuration of an ultrasound diagnostic device 10.

[0029] 1, an image management system 100 serving as a data management device system is installed in a medical facility such as a hospital, etc. The image management system 100 is a system for managing medical image data such as ultrasound image data.

[0030] The image management system 100 includes an ultrasound diagnostic device 10 and an image management server 40 as a management device. The devices in the image management system 100 are connected to each other so as to be able to communicate data with each other via a communication network N. The communication network N is a LAN (Local Area Network) or the like.

[0031] The image management server 40 is a PACS (Picture Archiving and Communication System) device. The image management server 40 receives, stores, and manages ultrasound image data generated by the ultrasound diagnostic device 10.

[0032] The image management system 100 may be configured to be connected to an examination device of a modality other than the ultrasound diagnostic device 10. The examination device may be an MRI (magnetic resonance imaging device), a CT (computed tomography device), a DR (digital X-ray device), etc. In this configuration, the image management server 40 manages medical image data generated by each of the examination devices.

[0033] As shown in FIG. 2, the ultrasonic diagnostic device 10 includes an ultrasonic diagnostic device main body 1 and ultrasonic probes 2A and 2B. One of the ultrasonic probes 2A and 2B is connected to the ultrasonic diagnostic device main body 1. The ultrasonic probes 2A and 2B are different types of ultrasonic probes. The type of ultrasonic probe refers to a model number, a product name, a version, etc. However, ultrasonic probes of the same type include ultrasonic probes having the same model number, product name, version, etc., and ultrasonic probes that are physically the same (having the same serial number). Here, the ultrasonic probe 2A has, for example, a relatively high ultrasonic frequency and a function excellent in the resolution of an ultrasonic image. The ultrasonic probe 2B has, for example, a relatively low ultrasonic frequency and a function excellent in the contrast of an ultrasonic image. The ultrasonic probes connected to the ultrasonic diagnostic device main body 1 may be one type or three or more types.

[0034] The ultrasonic probes 2A and 2B transmit ultrasonic waves (transmitted ultrasonic waves) into the subject and receive reflected waves of the ultrasonic waves (reflected ultrasonic waves: echoes) reflected in the subject. The ultrasonic probe 2A has an ultrasonic probe body 21A, a cable 22, and a connector 23. The ultrasonic probe 2B has an ultrasonic probe body 21B, a cable 22, and a connector 23. The ultrasonic probe body 21A is a header part of the ultrasonic probe 2A and transmits and receives ultrasonic waves. The ultrasonic probe body 21B is a header part of the ultrasonic probe 2B and transmits and receives ultrasonic waves. The cable 22 is connected to the ultrasonic probe body 21A or 21B and the connector 23. The cable 22 is a cable through which drive signals for the ultrasonic probe bodies 21A and 21B and received signals of ultrasonic waves flow. The connector 23 is a plug connector for connecting to a receptacle connector (not shown) of the ultrasonic diagnostic device body 1.

[0035] The connector 23 also has a storage unit (not shown). This storage unit stores identification information including the type of the ultrasound probe of the device itself. When the ultrasound probe is connected to the ultrasound diagnostic device main body 1, the storage unit of the connector 23 can be accessed from the ultrasound diagnostic device main body 1.

[0036] The ultrasound diagnostic device body 1 is connected to the ultrasound probe body 21A or 21B via a connector 23 and a cable 22. The ultrasound diagnostic device body 1 transmits an electric drive signal to the ultrasound probe body 21A or 21B. The ultrasound diagnostic device body 1 causes the ultrasound probe body 21A or 21B to transmit an ultrasonic wave to the subject by the drive signal. The ultrasound probes 2A and 2B generate reception signals, which are electric signals, in response to the reflected ultrasonic waves from within the subject received by the ultrasound probe bodies 21A and 21B. The ultrasound diagnostic device body 1 images the internal state of the subject as ultrasound image data based on the reception signals generated by the ultrasound probes 2A and 2B.

[0037] The ultrasonic probe body 21A has a transducer 2a at the tip side. The transducers 2a are arranged in a one-dimensional array in the azimuth direction (scanning direction), for example. The transducers 2a may be arranged in a two-dimensional array. The number of transducers 2a can be set arbitrarily. In the present embodiment, an electronic scanning probe of a linear scanning type is adopted as the ultrasonic probes 2A and 2B. However, the ultrasonic probes 2A and 2B may be either an electronic scanning type or a mechanical scanning type. The ultrasonic probes 2A and 2B may be either a linear scanning type, a sector scanning type, or a convex scanning type. The ultrasonic diagnostic device body 1 and the ultrasonic probes 2A and 2B may be configured to perform wireless communication instead of wired communication via the cable 22. The wireless communication is, for example, UWB (Ultra Wide Band).

[0038] The ultrasound diagnostic device main body 1 includes an operation input unit 11, a transmission unit 12, a reception unit 13, an image generation unit 14, an image processing unit 15, a display control unit 16, a display unit 17, a control unit 18, a storage unit 19, and a communication unit 31. The control unit 18 functions as a first estimation unit, a second estimation unit, an estimation unit, an output unit, a switching means, and a selection unit.

[0039] The operation input unit 11 has operation elements such as a push button, an encoder, a lever switch, a joystick, a trackball, a keyboard, a touch pad, a multifunction switch, etc. The operation input unit 11 accepts operation input to each operation element from a user such as a doctor or a technician, and outputs the operation information to the control unit 18.

[0040] The transmitting unit 12 supplies a driving signal, which is an electric signal, to the ultrasonic probe 2A or 2B under the control of the control unit 18 to cause the ultrasonic probe 2A or 2B to generate a transmission ultrasonic wave. The transmitting unit 12 includes, for example, a clock generating circuit, a delay circuit, and a pulse generating circuit. The clock generating circuit generates a clock signal that determines the transmission timing and transmission frequency of the driving signal. The delay circuit sets a delay time for each individual path corresponding to each transducer 2a, and delays the transmission of the driving signal by the set delay time. The delay circuit focuses a transmission beam formed by the transmission ultrasonic wave by the delay. The pulse generating circuit generates a pulse signal as a driving signal at a predetermined period. The transmitting unit 12 generates a transmission ultrasonic wave by driving, for example, a continuous part (for example, 64 pieces) of a plurality of transducers 2a (for example, 192 pieces) arranged in the ultrasonic probes 2A and 2B. Then, the transmitting unit 12 scans by shifting the driven transducer 2a in the azimuth direction (scanning direction) every time a transmission ultrasonic wave is generated.

[0041] The receiving unit 13 receives a received signal, which is an electrical signal, from the ultrasonic probe 2A or 2B under the control of the control unit 18. The receiving unit 13 includes, for example, an amplifier, an A / D conversion circuit, and a phasing and summing circuit. The amplifier amplifies the received signal by a preset amplification factor for each individual path corresponding to each transducer 2a. The A / D conversion circuit performs analog-to-digital conversion (A / D conversion) of the amplified received signal. The phasing and summing circuit adjusts the time phase by providing a delay time for the A / D converted received signal for each individual path corresponding to each transducer 2a, and adds them together (phasing and summing) to generate sound ray data.

[0042] The image generating unit 14 performs envelope detection processing, logarithmic compression, and the like on the sound ray data from the receiving unit 13 under the control of the control unit 18, and adjusts the dynamic range and gain to convert the luminance. The image generating unit 14 generates B (Brightness) mode image data consisting of pixels having a luminance value as the received energy through the luminance conversion. That is, the B mode image data represents the strength of the received signal by the luminance. The image generating unit 14 may be configured to generate tomographic image data in other image modes such as a color Doppler mode and an elasticity image mode in addition to the B mode. Furthermore, the image generating unit 14 may be configured to have other image modes such as an M (Motion) mode that generate image data other than tomographic image data.

[0043] The image processing unit 15 has an image memory unit 15a. The image memory unit 15a is composed of a semiconductor memory such as a dynamic random access memory (DRAM). The image processing unit 15 performs appropriate image processing on the B-mode image data input from the image generating unit 14 under the control of the control unit 18. The image processing unit 15 also stores the input or image-processed B-mode image data in units of frames in the image memory unit 15a under the control of the control unit 18. The B-mode image data in units of frames is sometimes called ultrasonic image data. The image processing unit 15 transmits the ultrasonic image data stored in the image memory unit 15a to the display control unit 16 one frame at a time at a predetermined time under the control of the control unit 18. The image memory unit 15a is composed of a large-capacity memory capable of holding frame image data for about 10 seconds, for example. The image memory unit 15a holds ultrasonic image data for the most recent 10 seconds by a first-in first-out (FIFO) method.

[0044] The display control unit 16 performs processing such as coordinate conversion on the B-mode image data input from the image processing unit 15 under the control of the control unit 18 to convert the image data into an image signal for display. The display control unit 16 outputs the image signal to the display unit 17.

[0045] The display unit 17 has a display panel such as an LCD (Liquid Crystal Display), an organic EL (Electro-Luminescence) display, an inorganic EL display, etc. The display unit 17 displays an ultrasound image on the display panel according to an image signal output from the display control unit 16 under the control of the control unit 18. The display unit 17 also displays various display information input from the control unit 18 on the display panel.

[0046] The control unit 18 includes, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). The control unit 18 reads out various processing programs stored in the ROM, expands them in the RAM, and controls each part of the ultrasound diagnostic device 10 in cooperation with the expanded programs and the CPU. The ROM is composed of a non-volatile memory such as a semiconductor. The ROM stores a system program corresponding to the ultrasound diagnostic device 10, various processing programs executable on the system program, and various data such as a gamma table. These programs are stored in the RAM in the form of computer-readable program codes. The CPU sequentially executes operations in accordance with the program codes on the RAM. The RAM forms a work area for temporarily storing various programs executed by the CPU and data related to these programs.

[0047] In particular, it is assumed that the ROM of the control unit 18 stores a first ultrasonic image output program for executing a first ultrasonic image output process, which will be described later.

[0048] The storage unit 19 is a storage unit such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) that stores information such as ultrasound image data in a writable and readable manner. In particular, the storage unit 19 stores a converter and a classifier as a trained model of an AI, which will be described later.

[0049] The communication unit 31 is connected to the communication network N. The communication unit 31 includes a network card and the like. The control unit 18 transmits and receives information to and from external devices such as an image management server 40 connected to the communication network N via the communication unit 31.

[0050] For each unit of the ultrasound diagnostic device 10, a part or all of the functions of each functional block can be realized as a hardware circuit such as an integrated circuit. The integrated circuit is, for example, an LSI (Large Scale Integration). Depending on the degree of integration, the LSI may be called an IC (Integrated Circuit), a system LSI, a super LSI, or an ultra LSI. The method of integration is not limited to LSI. The integration may be realized by a dedicated circuit or a general-purpose processor, or may use a field programmable gate array (FPGA) or a reconfigurable processor that can reconfigure the connections and settings of circuit cells inside the LSI. Each unit may execute a part or all of the functions of the functional blocks by software. In this case, the software is stored in one or more storage media such as a ROM, an optical disk, or a hard disk. The arithmetic processor of the ultrasound diagnostic device 10 reads the software from the storage medium and executes it.

[0051] Next, an AI algorithm used for image data conversion in the present embodiment will be described with reference to Fig. 3 and Fig. 4. Here, as the AI ​​algorithm, a GAN (Generative Adversarial Network) and Cycle-GAN will be described. Fig. 3 is a diagram showing the configuration of a GAN. Fig. 4 is a diagram showing the configuration of a Cycle-GAN.

[0052] GAN is a type of AI algorithm used in unsupervised learning that does not require data labeling. GAN can generate non-existent data or convert existing data according to its characteristics by learning features from data (machine learning).

[0053] GAN is implemented by a system of two neural networks competing with each other in a zero-sum game framework. The two neural networks are a generation network (Generator) and a discrimination network (Discriminator). If the purpose is to generate image data as in this embodiment, the Generator outputs image data. The Discriminator discriminates (determines) whether the image data output from the Generator is correct or not. The Generator learns to deceive the Discriminator. The Discriminator learns to discriminate more accurately. This is why the way in which two networks learn with opposing purposes is called adversarial.

[0054] 3, the GAN includes a Generator 51 and a Discriminator 52. The Generator 51 converts input image data 61 and outputs output image data 62. The Generator 51 performs conversion so that the output image data 62 can be discriminated as genuine by the Discriminator 52.

[0055] Meanwhile, the Discriminator 52 uses the output image data 62 as input image data 63, and receives the input image data 63 and real image data 64. The Discriminator 52 discriminates whether the input image data 63 and the real image data 64 are real or fake, and outputs the discrimination result. At this time, the Discriminator 52 learns to discriminate the input image data 63 as fake and the real image data 64 as real. The Generator 51 uses the discrimination result of the Discriminator 52 to learn to discriminate the output image data 62 as real by the Discriminator 52. These learning processes are repeated.

[0056] For example, suppose that the initial input image data 61 is image data with random pixel values. The real image data 64 is B-mode image data. The output image data 62 at the beginning of learning is image data of a chaotic image. With each repetition of learning, the output image data 62 becomes an image that looks more real. The output image data 62 at the end of learning becomes an image that looks more real, like the real image data 64.

[0057] In this embodiment, Cycle-GAN, which is a type of GAN, is used. Cycle-GAN is a GAN that extracts a predetermined feature of an image in a certain image data group X and converts the image data of the image data group X into image data with a predetermined feature of the other image data group Y. The features of the image data groups X and Y are pairs such as (horse, zebra) and (summer landscape, winter landscape). In addition, Cycle-GAN is capable of mutual conversion of image data between the image data group X and the image data group Y.

[0058] As shown in Fig. 4, Cycle-GAN has Generators 71 and 72 and Discriminators 73 and 74. The Generator 71 converts real image data X1 having characteristics of an image data group X, and outputs it as fake image data Y1 having characteristics of an image data group Y. The Generator 71 performs conversion so that the fake image data Y1 is discriminated by the Discriminator 73 as being real from the image data group Y.

[0059] On the other hand, the Discriminator 73 receives input of fake image data Y1 and real image data Y2 having characteristics of the image data group Y. The Discriminator 73 discriminates whether the fake image data Y1 and the real image data Y2 are real or fake in the image data group Y, and outputs the discrimination result. At this time, the Discriminator 73 learns to discriminate the fake image data Y1 as a fake in the image data group Y, and to discriminate the real image data Y2 as real. The Generator 71 uses the discrimination result of the Discriminator 73 to learn to discriminate the fake image data Y1 as real by the Discriminator 73.

[0060] The Generator 72 converts genuine image data Y2 of the image data group Y and outputs it as fake image data Y1 having the characteristics of the image data group X. The Generator 72 performs conversion so that the fake image data X2 can be identified by the Discriminator 74 as genuine image data of the image data group X.

[0061] On the other hand, the Discriminator 74 receives the fake image data X2 and the real image data X1. The Discriminator 74 discriminates whether the fake image data X2 and the real image data X1 are real or fake in the image data group X, and outputs the discrimination result. At this time, the Discriminator 74 learns to discriminate the fake image data X2 as a fake in the image data group X, and to discriminate the real image data X1 as real. The Generator 72 uses the discrimination result of the Discriminator 74 to learn to discriminate the fake image data X2 as real by the Discriminator 74.

[0062] In this embodiment, the storage unit 19 stores a Generator 71. Also, pairs of features (attributes) of the image data group X, Y are (ultrasonic probe 2A, ultrasonic probe 2B), (ultrasonic probe 2A, MRI), (ultrasonic probe 2B, MRI). Specifically, the storage unit 19 stores Generators 71A and 71B (FIG. 6) as the Generator 71. The Generator 71A is a converter for image data generated using the ultrasonic probe 2A. The Generator 71B is a converter for image data generated using the ultrasonic probe 2B. The Generator 71A has Generators 71a and 71b. The Generator 71B has Generators 71c and 71d.

[0063] "(Ultrasound) image data generated using an ultrasonic probe" means "(Ultrasound) image data based on a reception signal for image generation received by an ultrasonic probe." Hereinafter, "(Ultrasound) image data generated using an ultrasonic probe" will be simply referred to as "(Ultrasound) image data using an ultrasonic probe."

[0064] Generator 71a image-converts ultrasound image data obtained using ultrasound probe 2A to an image quality similar to that of an ultrasound image obtained using ultrasound probe 2B (ultrasound probe 2B-like). Generator 71b image-converts ultrasound image data obtained using ultrasound probe 2A to an image quality similar to that of an MRI image (MRI-like). Generator 71c image-converts ultrasound image data obtained using ultrasound probe 2B to an image quality similar to that of an ultrasound image obtained using ultrasound probe 2A (ultrasound probe 2A-like). Generator 71d image-converts image data obtained using ultrasound using ultrasound probe 2B to an image quality similar to that of an MRI image.

[0065] The generators 71a and 71c are trained using ultrasonic image data obtained by using the ultrasonic probe 2A and ultrasonic image data obtained by using the ultrasonic probe 2B. The discriminators 73 and 74 corresponding to the generators 71a and 71c are trained in the same manner. For example, the real image data X1 is ultrasonic image data obtained by using the ultrasonic probe 2A. The real image data Y2 is ultrasonic image data obtained by using the ultrasonic probe 2B.

[0066] Generator 71b and corresponding discriminators 73 and 74 are trained using ultrasonic image data obtained by ultrasonic probe 2A and MRI image data. Generator 71d and corresponding discriminators 73 and 74 are trained using ultrasonic image data obtained by ultrasonic probe 2B and MRI image data.

[0067] Cycle-GAN can proceed with learning without strict paired learning data (training data). Specifically, the real image data of the image data group X as training data and the real image data of the image data group Y may have different cross sections and positioning of the subject.

[0068] When using the Generator 71 for X→Y image conversion, essentially, the real image data of the image data group X as the learning data may be any image data. However, in order to increase the conversion accuracy, it is preferable to narrow down the real image data of the image data group X as the training data to one feature before conversion (such as the ultrasonic probe 2A). In this way, with Cycle-GAN, it is relatively easy to collect learning data. Also, with Cycle-GAN, image conversion can be performed to the features of the target to be converted.

[0069] As learning progresses, the converted image data converted by the Generator 71 becomes image data similar to the real image data Y2. For example, the fake image data Y1 becomes image data similar to the real image data Y2. Image data being similar means that the image features in Cycle-GAN are the same, and the image contrast, gradation, graininess, and sharpness are similar. In addition, image data similar to the real image data Y2 is, for example, image data (e.g., tomographic image data) captured in the same direction as the real image data Y2.

[0070] The real image data Y2 is image data captured under a predetermined condition, for example, image data captured using an ultrasound probe of a different type from that used for the real image data X1.

[0071] Hereinafter, the Generators 71, 71A, 71B, and 71a to 71d are referred to as the converters 71, 71A, 71B, and 71a to 71d. The Discriminators 73 and 74 are referred to as the discriminators 73 and 74.

[0072] Next, the operation of the image management system 100 will be described with reference to Fig. 5 to Fig. 8(b). Fig. 5 is a flowchart showing the first ultrasound image output process. Fig. 6 is a diagram showing data and processing of the first ultrasound image output process. Fig. 7 is a diagram showing a display screen 80. Fig. 8(a) is a diagram showing an ultrasound image 91. Fig. 8(b) is a diagram showing an MRI-like image 92.

[0073] 5, a first ultrasonic image output process executed by the ultrasonic diagnostic device 10 will be described. It is assumed that the ultrasonic probe 2A or 2B is connected in advance to the ultrasonic diagnostic device main body 1. It is also assumed that the converters 71a to 71d are stored in the storage unit 19.

[0074] In the ultrasonic diagnostic device 10, for example, a user inputs an instruction to execute a first ultrasonic image output process via the operation input unit 11. In response to the input of the instruction, the control unit 18 executes the first ultrasonic image output process in accordance with the first ultrasonic image output program stored in the ROM.

[0075] First, the control unit 18 acquires the type of the ultrasound probe connected to the ultrasound diagnostic device main body 1 (step S11). The control unit 18 reads out the identification information of the ultrasound probe stored in the storage unit of the connector 23 of the connected ultrasound probe. The control unit 18 acquires the type of the ultrasound probe corresponding to the read out identification information. As shown in FIG. 6, the type of the ultrasound probe is, for example, "ultrasound probe 2A" or "ultrasound probe 2B".

[0076] The control unit 18 controls the transmitting unit 12 and the receiving unit 13 to transmit and receive ultrasonic waves via the ultrasonic probe (step S12). In step S12, the user places the tip of the ultrasonic probe against the subject and performs a scan.

[0077] The control unit 18 controls the receiving unit 13, the image generating unit 14, the image processing unit 15, and the display control unit 16 to generate ultrasound image data based on the ultrasound of step S12 (step S13). The ultrasound image data of step S13 is B-mode image data, which is regarded as original image data. As shown in FIG. 6, for example, original image data using the ultrasound probe 2A or original image data using the ultrasound probe 2B is generated.

[0078] The control unit 18 receives input of setting information indicating whether or not to convert the original image data using an AI classifier from the user via the operation input unit 11 (step S14). In step S14, the control unit 18 determines whether or not to convert the original image data based on the input setting information.

[0079] If the original image data is not to be converted (step S14; NO), the control unit 18 causes the display unit 17 to display the original image data generated in step S13 (step S15). The control unit 18 receives an input from the user via the operation input unit 11 as to whether or not to transmit the image data to the image management server 40 (step S16). In step S16, the control unit 18 determines whether or not to transmit the image data to the image management server 40 in response to the input.

[0080] If the image data is not to be transmitted (step S16; NO), the first ultrasound image output process ends. If the image data is to be transmitted (step S16; YES), the control unit 18 transmits the original image data to the image management server 40 via the communication unit 31 (step S17). In response to step S17, the image management server 40 receives the original image data from the ultrasound diagnostic device 10 and stores it in the memory unit of its own device. After step S17, the first ultrasound image output process ends.

[0081] When converting the original image data (step S14; YES), the control unit 18 accepts input of the type of AI converter from the user via the operation input unit 11 (step S18). As shown in Fig. 6, the types of AI converters are, for example, converters 71a, 71b, 71c, and 71d. In step S18, at least the converted characteristics of the converter (ultrasound probe 2A style, ultrasound probe 2B style, MRI style) need only be input.

[0082] The control unit 18 sets the transducer to be used according to the type of ultrasound probe acquired in step S11 and the type of transducer input in step S18 (step S19). As shown in Fig. 6, the transducer to be used is set to, for example, one of the transducers 71a, 71b, 71c, and 71d. Here, the transducer to be used is set to be the same as the type of ultrasound probe in step S11 and the type of ultrasound probe in the real image data X1 in Fig. 4, and corresponds to the type of transducer in step S18.

[0083] The control unit 18 reads out the converter set in step S19 from the storage unit 19, and converts the original image data in step S13 using the converter (step S20). The converted original image data is defined as converted image data. As shown in FIG. 8, for example, when the converter 71a is used, the converted image data becomes processed image data in the style of the ultrasonic probe 2B with image quality characteristic of the ultrasonic image obtained using the ultrasonic probe 2B. When the converter 71b or 71d is used, the converted image data becomes processed image data in the style of the MRI with image quality characteristic of the MRI image. When the converter 71c is used, for example, the converted image data becomes processed image data in the style of the ultrasonic probe 2A with image quality characteristic of the ultrasonic image obtained using the ultrasonic probe 2A.

[0084] The control unit 18 receives an input from the user as to whether or not to display image data side by side via the operation input unit 11 (step S21). In step S21, the control unit 18 determines whether or not to display image data side by side according to the input information.

[0085] When displaying side by side (step S21; YES), the control unit 18 displays the original image data of step S13 and the converted image data of step S20 side by side on the display unit 17 (step S22). In step S22, for example, a display screen 80 shown in Fig. 7 is displayed. The display screen 80 corresponds to a case where the original image data is generated using the ultrasonic probe 2A and image converted using the converter 71a.

[0086] The display screen 80 has an original image 81, a converted image 82, and supplementary information 83, 84, 85, and 86. The original image 81 is an image of original image data obtained using the ultrasonic probe 2A. The converted image 82 is an image of converted image data obtained by image-converting the original image data of the original image 81 using the converter 71a. The original image 81 and the converted image 82 are arranged in parallel in the left-right direction. However, the original image 81 and the converted image 82 may also be arranged in parallel in the up-down direction.

[0087] The incidental information 83 is information indicating that the original image 81 is an original image. The incidental information 83 is, for example, text information of "Original" arranged near the original image 81. However, the incidental information 83 may be other text information, a mark, the frame of the original image 81, etc. The incidental information 84 is incidental information related to the original image 81. The incidental information 83 is, for example, text information of "Freq. Mid" arranged near the original image 81. "Freq. Mid" indicates that the frequency of the ultrasonic transmission and reception settings of the original image 81 is at a mid level. The incidental information 83, 84 is, for example, white.

[0088] The incidental information 85 is information indicating that the converted image 82 is a processed image of the original image and its contents. The incidental information 85 is, for example, text information of "Probe 2B Style" arranged near the original image 81. "Probe 2B Style" indicates that the converted image 82 is a converted image with image quality characteristic of "ultrasonic probe 2B style". However, the incidental information 83 may be other text information, a mark, a frame of the converted image 82, or the like. The incidental information 86 is incidental information related to the converted image 82. The incidental information 86 is, for example, text information of "Freq.High(AI)" arranged near the original image 81. "Freq.High(AI)" indicates that the frequency of the ultrasonic transmission and reception setting of the converted image 82 is at a high level and that it is an AI processed image. The incidental information 85, 86 is, for example, black. In this manner, the auxiliary information 83, 84 and the auxiliary information 85, 86 are set in different colors to distinguish between the original image and the converted image (processed image).

[0089] Returning to FIG. 5, the control unit 18 receives an input from the user via the operation input unit 11 to switch the positions of the two images being displayed (step S23). In step S23, the control unit 18 determines whether or not there is an input to switch the positions of the two images being displayed in response to the input. If there is a switching input (step S23; YES), the control unit 18 switches the positions of the original image and the converted image being displayed side by side (step S24). In step S24, for example, when the display screen 80 is being displayed, it is switched to a right-side side-by-side display in which the converted image 82 is arranged on the left and the original image 81 is arranged on the right.

[0090] The control unit 18 receives an input from the user to end the image display via the operation input unit 11 (step S25). In step S25, the control unit 18 determines whether or not to end the image display in response to the input. If there is no switching input (step S23; NO), the process proceeds to step S25. If the display is not to be ended (step S25; NO), the process proceeds to step S23.

[0091] If the converted image data is not displayed in parallel (step S21; NO), the control unit 18 displays the converted image data in step S20 alone on the display unit 17 (step S26). In step S26, for example, on the display screen 80, only the converted image 82 and the additional information 85, 86 are displayed.

[0092] The control unit 18 receives an input from the user via the operation input unit 11 to switch the display of one of the images being displayed (step S27). In step S27, the control unit 18 determines whether or not there is an input to switch the image being displayed in response to the input. If there is an input to switch (step S27; YES), the control unit 18 switches the display of the image being displayed to the other image corresponding thereto (step S28). In step S28, for example, when only the converted image 82 and the incidental information 85, 86 are being displayed, the display is switched to only the original image 81 and the incidental information 83, 84.

[0093] The control unit 18 receives an input of ending the image display from the user via the operation input unit 11 (step S29). In step S29, the control unit 18 determines whether or not to end the image display in response to the input. If the display is not to be ended (step S29; NO), the process proceeds to step S27.

[0094] If the display is to be ended (step S25; YES), the control unit 18 receives an input from the user via the operation input unit 11 as to whether or not to send image data (step S30). If the display is to be ended (step S29; YES), the process proceeds to step S30. In step S30, the control unit 18 determines whether or not to send image data to the image management server 40 in response to the input.

[0095] If image data is not to be transmitted (step S30; NO), the first ultrasound image output process ends. If image data is to be transmitted (step S30; YES), the control unit 18 transmits the original image data and the converted image data to the image management server 40 via the communication unit 31 in association with each other (step S31). In response to step S31, the image management server 40 receives the original image data and the converted image data from the ultrasound diagnostic device 10. The image management server 40 stores the original image data and the converted image data in association with each other in the memory unit of its own device. After step S31, the first ultrasound image output process ends.

[0096] The display screen 80 is an example in which the original image data of an original image 81 obtained using the ultrasonic probe 2A is converted into converted image data of a converted image 82 in the style of the ultrasonic probe 2B using the converter 71a. As shown in Fig. 8(a), for example, consider the original image data of an ultrasonic image 91 obtained using the ultrasonic probe 2A. As shown in Fig. 8(b), the original image data of the ultrasonic image 91 can be converted into converted image data of an MRI-like image 92 with image quality characteristic of MRI using the converter 71b.

[0097] As described above, according to the present embodiment, the ultrasound diagnostic device 10 includes a first ultrasound probe as the ultrasound probe 2A or 2B that transmits and receives ultrasound to and from a subject, and a control unit 18 as an estimation unit. The control unit 18 uses a converter 71 as a trained model that is machine-learned using training data. The training data includes first data based on a reception signal for image generation received by a second ultrasound probe of the same type as the first ultrasound probe, and second data captured under a predetermined condition. The first data is real image data X1 in FIG. 4. The second data is real image data Y2 in FIG. 4. The control unit 18 converts third data based on a reception signal for image generation received by the first ultrasound probe into fourth data similar to the second data and outputs the fourth data. The third data is original image data using the first ultrasound probe. The fourth data is converted image data similar to the real image data Y2.

[0098] Therefore, even a user who is not familiar with diagnosis can make an appropriate diagnosis from images of various image qualities (original images, converted images) using the original image data and converted image data.

[0099] The first data (real image data X1) and the third data (original image data) are B-mode image data based on a predetermined reception signal. Therefore, even a user who is not familiar with diagnosis can make an appropriate diagnosis from images of various image quality (original image, converted image) by using the original image data and the converted image data.

[0100] The second data (genuine image data Y2) includes ultrasound image data and MRI image data, so that appropriate diagnosis can be made from images with various image qualities (ultrasound images, MRI-like images).

[0101] The fourth data (converted image data) is ultrasound image-like image data or MRI image data, which allows appropriate diagnosis to be made from images with various image qualities (ultrasound images, MRI image-like images).

[0102] The first data (real image data X1) is data based on a reception signal for image generation received by the second ultrasonic probe. The second data (real image data Y2) is data based on a reception signal for image generation received by a third ultrasonic probe as the ultrasonic probe 2A or 2B different from the second ultrasonic probe. Therefore, the third ultrasonic probe-like image data allows appropriate diagnosis from images of various image quality (ultrasonic image of the first ultrasonic probe, third ultrasonic probe-like image).

[0103] The control unit 18 displays on the display unit 17 supplementary information 85, 86 indicating that the fourth data (converted image data) is a processed image. Therefore, since the fourth data is a processed image, it is not suitable for definitive diagnosis / final confirmation, and by notifying the user that it is a processed image, it is possible to draw attention to the fact. In addition, it is possible to prevent confusion between the displayed original image and the converted image.

[0104] The control unit 18 displays on the display unit 17 supplementary information 85, 86 indicating the type and frequency of the ultrasound probe corresponding to the fourth data (converted image data). When the fourth data (converted image data) is MRI-like image data, the supplementary information includes the type of modality (MRI) instead of the type of ultrasound probe. This makes it possible to notify the user of the type, frequency, and modality of the ultrasound probe corresponding to the fourth data, and to notify the user that inference processing is running on the fourth data.

[0105] The control unit 18 displays, on the display unit 17, supplementary information 83, 84 indicating that the third data (original image data) is an original image. This makes it possible to notify (indicate) to the user that the third data is suitable for definitive diagnosis / final confirmation and is an original image. In addition, it is possible to prevent confusion between the displayed original image and the converted image (processed image).

[0106] The control unit 18 displays the third data (original image data) or displays the third data and the fourth data (converted image data) in association with each other. This makes it possible to prevent a definitive diagnosis / final confirmation from being made on the processed image and to encourage a definitive diagnosis / final confirmation on the original image.

[0107] The image management system 100 also includes an ultrasound diagnostic device 10 and an image management server 40. The image management server 40 receives and stores the third data (original image data) or the third data and the fourth data (converted image data) from the ultrasound diagnostic device 10 in association with each other. The control unit 18 transmits the third data (original image data) or the third data and the fourth data (converted image data) in association with each other to the image management server 40. Therefore, the image management server 40 can receive only the third data or the third data and the fourth data, and can store and manage them in association with each other. The control unit 18 may be configured to output the third data (original image data) or the third data and the fourth data (converted image data) in association with each other on an image interpretation report.

[0108] The third data is the original image data. And the fourth data is the converted image data. The ultrasound diagnostic device 10 includes a display unit 17 that displays the original image data and the converted image data on the same screen or switches between them. Therefore, the converted image can be displayed side by side with the original image or switched between them, and a display method that is preferable for the user can be adopted, enabling an appropriate diagnosis to be made.

[0109] (Modification) A modified example of the first embodiment will be described with reference to Fig. 9. Fig. 9 is a flow chart showing the second ultrasonic image output process.

[0110] The first embodiment is configured to convert original image data into converted image data. This modified example is configured to convert pre-imaging sound ray data of the original image into pre-imaging sound ray data of the converted image. The pre-imaging sound ray data of the original image is referred to as original imaging data. The pre-imaging sound ray data of the converted image is referred to as converted imaging data.

[0111] The device configuration of this modification uses the image management system 100, similarly to the first embodiment. However, instead of the first ultrasonic image output program, a second ultrasonic image output program is stored in the ROM of the control unit 18 of the ultrasonic diagnostic device 10. The second ultrasonic image output program is a program for executing a second ultrasonic image output process, which will be described later.

[0112] Next, the operation of the image management system 100 will be described with reference to Fig. 9. Specifically, the second ultrasound image output process executed by the ultrasound diagnostic device 10 will be described with reference to Fig. 9. It is assumed that the ultrasound probe 2A or 2B is connected to the ultrasound diagnostic device main body 1 in advance in the ultrasound diagnostic device 10. It is also assumed that the converters 71a to 71d are stored in the storage unit 19.

[0113] In the ultrasonic diagnostic device 10, for example, a user inputs an instruction to execute a second ultrasonic image output process via the operation input unit 11. In response to the input of the instruction, the control unit 18 executes the second ultrasonic image output process in accordance with the second ultrasonic image output program stored in the ROM.

[0114] Steps S41 and S42 shown in Fig. 9 are similar to steps S11 and S12 of the first ultrasonic image output process in Fig. 5. The control unit 18 controls the receiving unit 13 to generate sound ray data of an ultrasonic image based on the ultrasonic waves of step S12 (step S43). The sound ray data of step S43 is imaging data for generating a B-mode image, and is regarded as original imaging data. The control unit 18 controls the image generating unit 14, the image processing unit 15, and the display control unit 16 to generate ultrasonic image data based on the original imaging data of step S43 (step S44). The ultrasonic image data of step S44 is B-mode image data, and is regarded as original image data.

[0115] The control unit 18 receives input of setting information as to whether or not to convert the original visualization data using an AI classifier from the user via the operation input unit 11 (step S45). In step S45, the control unit 18 determines whether or not to convert the original visualization data based on the input setting information.

[0116] If the original imaging data is not to be converted (step S45; NO), the control unit 18 causes the display unit 17 to display the original image data generated in step S44 (step S46). Steps S47 to S50 are similar to steps S16 to S19 in FIG. 5. The control unit 18 reads out the converter set in step S50 from the storage unit 19, and converts the original imaging data of step S43 using the converter (step S51). The converted original imaging data becomes converted imaging data.

[0117] The control unit 18 controls the image generation unit 14, the image processing unit 15, and the display control unit 16 to generate converted image data from the converted imaging data of step S51 (step S52). Steps S53 to S63 are similar to steps S21 to S31 in FIG.

[0118] As described above, according to this modification, the first data (sound ray data corresponding to the real image data X1) and the third data (sound ray data corresponding to the original image data) are sound ray data as a predetermined reception signal. Therefore, as in the first embodiment, even a user who is not familiar with diagnosis can make an appropriate diagnosis from images of various image quality (original image, converted image) by using the original image data and the converted image data.

[0119] The second data (genuine image data Y2) includes ultrasound data (sound ray data) and MRI image data, so that appropriate diagnosis can be made from images with various image qualities (ultrasound images, MRI-like images).

[0120] The fourth data (converted image data) is ultrasound data (sound ray data) and MRI-like image data. This allows appropriate diagnosis to be made from images with various image qualities (ultrasound images, MRI-like images).

[0121] As described in the first embodiment or this modified example, the first data and the third data are of the same data type (image data or sound ray data). For example, when the first data is B-mode image data, it is preferable that the third data is also B-mode image data. When the first data is sound ray data, it is preferable that the third data is also sound ray data. In this configuration, the conversion accuracy of the converter can be improved.

[0122] In this modification, the acoustic ray data is used as the reception signals of the first data and the third data, but the present invention is not limited to this. An RF (Radio Frequency) signal of the receiving unit 13 may be used as the reception signal.

[0123] (Second embodiment) A modified example of the second embodiment of the present invention will be described with reference to Fig. 10 and Fig. 11. Fig. 10 is a flowchart showing the third ultrasonic image output processing. Fig. 11 is a diagram showing data and processing of the third ultrasonic image output processing.

[0124] The first embodiment described above is configured to read the type of ultrasound probe connected to the ultrasound diagnostic device main body 1 as an attribute of the original image data from the storage unit and use it to set the converter. The present embodiment is configured to use an AI classifier to identify the type of ultrasound probe as an attribute of the original image data and use it to set the converter.

[0125] The device configuration of this embodiment is the same as that of the first embodiment, and uses the image management system 100. However, instead of the first ultrasonic image output program, a third ultrasonic image output program is stored in the ROM of the control unit 18 of the ultrasonic diagnostic device 10. The third ultrasonic image output program is a program for executing a third ultrasonic image output process described later.

[0126] Next, the operation of the image management system 100 will be described with reference to Fig. 10 and Fig. 11. Specifically, the third ultrasound image output process executed by the ultrasound diagnostic device 10 will be described with reference to Fig. 10. It is assumed that the ultrasound probe 2A or 2B is connected to the ultrasound diagnostic device main body 1 in advance in the ultrasound diagnostic device 10. It is also assumed that the converters 71a to 71d and the classifiers 73a and 74b as the classifiers 73 and 74 are stored in the storage unit 19 as shown in Fig. 11.

[0127] The classifier 73a is a classifier that classifies ultrasound image data obtained using the ultrasound probe 2B as genuine. The classifier 74a is a classifier that classifies ultrasound image data obtained using the ultrasound probe 2A as genuine. The classifiers 73a and 74a are trained using, for example, ultrasound image data obtained using the ultrasound probe 2A and ultrasound image data obtained using the ultrasound probe 2B in FIG. 4. For example, the genuine image data X1 is ultrasound image data obtained using the ultrasound probe 2A. The genuine image data Y2 is ultrasound image data obtained using the ultrasound probe 2B.

[0128] In the ultrasonic diagnostic device 10, for example, a user inputs an instruction to execute a third ultrasonic image output process via the operation input unit 11. In response to the input of the instruction, the control unit 18 executes the third ultrasonic image output process in accordance with the third ultrasonic image output program stored in the ROM.

[0129] Steps S71 to S76 shown in Fig. 9 are similar to steps S12 to S17 of the first ultrasound image output process in Fig. 5. When converting the original image data (step S73; YES), the control unit 18 reads out the classifiers 73a and 73b from the storage unit 19 (step S77). In step S77, the control unit 18 uses the classifiers 73a and 73b to classify the image attributes of the original image data.

[0130] For example, when the original image data is input to the classifier 73a and a classification result indicating that the image is genuine is obtained, "ultrasound probe 2B" is obtained as the attribute of the original image data. When the original image data is input to the classifier 74a and a classification result indicating that the image is genuine is obtained, "ultrasound probe 2B" is obtained as the attribute of the original image data. In other words, the type of ultrasound probe is obtained as the attribute of the original image data.

[0131] Step S78 corresponds to step S18 in Fig. 5. The control unit 18 sets the converter to be used according to the attribute of the original image data acquired in step S77 and the type of converter input in step S78 (step S79). Steps S80 to S91 are similar to steps S20 to S31 in Fig. 5.

[0132] As described above, according to the present embodiment, the trained models stored in the storage unit 19 include a plurality of trained models including at least the first trained model and the second trained model. The first trained model (converter 71) is machine-trained using trained data including the first data and the second data. The second trained model (classifiers 73, 74) is machine-trained using trained data including the fifth data and the sixth data. The fifth data is based on a received signal for image generation received by a second ultrasonic probe of the same type as the first ultrasonic probe. The sixth data indicates an attribute of the fifth data. The control unit 18 switches to output of one of the trained models according to a user operation. The control unit 18 selects the first trained model according to the attribute of the third data (original image data) identified by the second trained model and the user operation. The control unit 18 converts the third data into fourth data similar to the second data using the selected first trained model and outputs the fourth data. Therefore, the second trained model makes it possible to easily obtain attributes of the third data from the third data.

[0133] The control unit 18 includes a plurality of switching units ("converter switching" in FIG. 11) for each attribute of the third data. The control unit 18 identifies the attribute of the third data and selects one of the plurality of switching units. Therefore, the attribute of the third data can be easily acquired from the third data by the second trained model.

[0134] (Third embodiment) A modified example of the third embodiment of the present invention will be described with reference to Fig. 12. Fig. 12 is a flowchart showing a fourth ultrasonic image output process.

[0135] The first embodiment is configured to convert the original image data to generate converted image data, which is then displayed and transmitted. The present embodiment is configured to identify an object to be identified, such as a specific organ / tissue, in an image from the converted image data.

[0136] The device configuration of this embodiment is the same as that of the first embodiment, and uses the image management system 100. However, instead of the first ultrasonic image output program, a fourth ultrasonic image output program is stored in the ROM of the control unit 18 of the ultrasonic diagnostic device 10. The fourth ultrasonic image output program is a program for executing a fourth ultrasonic image output process described later.

[0137] Next, the operation of the image management system 100 will be described with reference to Fig. 12. Specifically, a fourth ultrasound image output process executed by the ultrasound diagnostic device 10 will be described with reference to Fig. 12. It is assumed that an ultrasound probe 2A or 2B is connected to the ultrasound diagnostic device main body 1 in advance in the ultrasound diagnostic device 10. It is also assumed that a classifier D1 (not shown) for identifying an identification object of a subject is stored in a storage unit 19.

[0138] The classifier D1 is an AI classifier that identifies a predetermined object to be identified from an image of converted image data converted from the original image data by the converter 71. The classifier D1 is, for example, a trained model configured with a convolutional neural network. The object to be identified is at least one of an organ and a tissue. The organ is the heart, the liver, etc. The tissue is an organ, a nerve, a fascia, a muscle, a blood vessel, a placenta, a lymph node, a brain, a prostate, a carotid artery, a breast, etc. The object to be identified may also be a lesion indicating some kind of disease, an abnormal brightness area in an ultrasound image, etc.

[0139] The classifier D1 is trained using converted image data including an object to be identified in the image and a predetermined correct answer label of the converted image data. The predetermined correct answer label includes the photographed area, photographed part, position, measurement, image classification, etc. of the object to be identified. When the converted image data converted from the original image data by the converter 71 is input, the classifier D1 identifies the object to be identified in the image of the converted image data and outputs the classification result. For example, the classifier D1 is prepared for each type of object to be identified.

[0140] Conventionally, the classifier D1 was trained using image data and correct answer labels for each ultrasound probe. In this case, for example, when lesions are to be classified, it is difficult to make a classifier with sufficient performance unless training data is prepared for each corresponding probe, and it takes time to create a classifier because new training data must be prepared to add a new ultrasound probe. In addition, supporting multiple ultrasound probes requires the trained model to be compatible with images of various image qualities, and in such a case, the size of the trained model becomes large, which impairs the real-time performance required for an ultrasound diagnostic device. On the other hand, when converted image data is input, it is possible to reduce the diversity of image quality that the classifier can handle. This means that the trained model can be made smaller. Furthermore, when adding new compatible ultrasound probes, it is only necessary to create a converter 71, and it is possible to significantly reduce the training data for the classifier. Here, since the training data used to create the converter 71 does not necessarily include images that include lesions, sufficient training data can be easily prepared and created.

[0141] In the ultrasonic diagnostic device 10, for example, a user inputs an instruction to execute a fourth ultrasonic image output process via the operation input unit 11. In response to the input of the instruction, the control unit 18 executes the fourth ultrasonic image output process in accordance with the fourth ultrasonic image output program stored in the ROM.

[0142] As shown in Fig. 12, steps S101 to S110 are similar to steps S71 to S80 of the third ultrasound image output process in Fig. 10. The control unit 18 receives an input of the type of identification object to be identified from the user via the operation input unit 11 (step S111). In step S111, the control unit 18 reads out a classifier D1 corresponding to the input type of identification object from the storage unit 19 and sets it for identification.

[0143] The control unit 18 uses the classifier set in step S111 to classify the object to be classified in the converted image data of step S110, and obtains the classification result (step S112). Step S113 is the same as step S81 in FIG. 10. In the case of side-by-side display (step S113; YES), the control unit 18 displays the original image data of step S13 and the converted image data of step S20 side-by-side on the display unit 17 (step S114). In step S114, the control unit 18 displays the classification result obtained in step S112 together with the original image data and the converted image data. The classification result is a classification result of the object to be classified in the converted image, and is displayed, for example, near the converted image.

[0144] Steps S115 to S117 are the same as steps S83 to S85 in Fig. 10. If parallel display is not performed (step S113; NO), the control unit 18 displays the converted image data of step S110 alone on the display unit 17 (step S118). In step S118, the control unit 18 displays the classification result acquired in step S112 together with the converted image data. Steps S119 to S123 are the same as steps S87 to S91 in Fig. 10.

[0145] As described above, according to this embodiment, the control unit 18 uses the classifier D1 to output the fifth data (classification result) from the fourth data (converted image data). The classifier D1 is machine-trained using learning data in which the fourth data obtained by converting the first data using the trained model (converter 71) and a predetermined correct answer label are used as a data set. For this reason, it is possible to easily identify objects to be identified, such as organs / tissues, from the fourth data.

[0146] In the above description, an example in which a ROM is used as a computer-readable medium for the program according to the present invention has been disclosed, but the present invention is not limited to this example. As other computer-readable media, non-volatile memories such as flash memories and portable recording media such as CD-ROMs can be applied. In addition, carrier waves can also be applied to the present invention as a medium for providing data of the program according to the present invention via a communication line.

[0147] The description of the above embodiment is merely an example of the ultrasound diagnostic device, image management system, data estimation method, and program according to the present invention, and is not limited thereto. For example, at least two of the above embodiment and modified examples may be appropriately combined.

[0148] In the above embodiment, the second data (real image data Y2) is characterized by two types of ultrasonic probes 2A and 2B different from the ultrasonic probes used for the first data (real image data X1). However, this is not limited to this. The second data and the corresponding fourth data may be characterized by one type or three or more types of ultrasonic probes.

[0149] In the above embodiment, the feature of the second data (real image data Y2) is MRI, which is a modality different from that of the ultrasound diagnostic device used for the third data (original image data). However, this is not limited to this. The feature of the second data may be another modality such as CT.

[0150] Furthermore, the characteristic of the second data (genuine image data Y2) may be a different type of ultrasonic diagnostic device from that used for the first data (genuine image data X1). The ultrasonic diagnostic device used for the third data (original image data) will be the same type as that used for the first data. The characteristic of the fourth data (converted image data) corresponds to the ultrasonic diagnostic device of the characteristic of the second data. In this configuration, the supplementary information 83 to 86 in FIG. 11 includes information such as the model name of the ultrasonic diagnostic device corresponding to the original image and the converted image.

[0151] Additionally, the detailed configuration and operation of the image management system 100 in the above embodiment may be modified as appropriate without departing from the spirit of the present invention. [Explanation of symbols]

[0152] 100 Image Management System 10. Ultrasound diagnostic equipment 1. Ultrasound diagnostic device body 11 Operation input section 12 Transmitter 13 Receiving section 14 Image Generation Unit 15 Image processing section 15a Image memory section 16 Display control section 17 Display 18 Control Unit 19 Memory section 31 Communications Department 2A,2B Ultrasonic probe 21A,21B Ultrasonic probe body 22 Cable 23 Connector 40 Image Management Server N Communication Network

Claims

1. a first ultrasonic probe for transmitting and receiving ultrasonic waves to and from a subject; a first estimation unit that converts third data based on a reception signal for image generation received by the first ultrasonic probe into fourth data similar to the second data and outputs the fourth data by using a trained model that is machine-learned using training data including first data based on a reception signal for image generation received by a second ultrasonic probe of the same type as the first ultrasonic probe and second data captured under a predetermined condition; An ultrasound diagnostic device comprising:

2. The ultrasonic diagnostic apparatus according to claim 1 , wherein the first data and the third data are predetermined received signals or B-mode image data based on the received signals.

3. The ultrasonic diagnostic apparatus according to claim 1 , wherein the second data includes at least one of ultrasonic data, MRI data, and CT data.

4. 3. The ultrasonic diagnostic apparatus according to claim 1, wherein the fourth data is ultrasonic data, MRI-like image data, or CT-like image data.

5. the first data is data based on a reception signal for generating an image received by the second ultrasonic probe, 3. The ultrasonic diagnostic apparatus according to claim 1, wherein the second data is data based on a reception signal for image generation received by a third ultrasonic probe of a type different from that of the second ultrasonic probe.

6. The ultrasonic diagnostic apparatus according to claim 1 , further comprising an output unit that outputs information indicating that the fourth data is a processed image.

7. 3. The ultrasonic diagnostic apparatus according to claim 1, further comprising an output unit configured to output at least one of information on a type of ultrasonic probe, a type of modality, and a frequency corresponding to the fourth data.

8. 3. The ultrasonic diagnostic apparatus according to claim 1, further comprising an output unit that outputs information indicating that the third data is an original image.

9. 3. The ultrasonic diagnostic apparatus according to claim 1, further comprising an output section that outputs the third data or outputs the third data and the fourth data in association with each other.

10. the third data is third image data and the fourth data is fourth image data; The ultrasonic diagnostic apparatus according to claim 1 , further comprising a display unit which displays the third image data and the fourth image data on the same screen or which switches between the displays.

11. The trained model is A first trained model trained by machine learning using training data including the first data and the second data; a second trained model that is machine-learned using training data including fifth data based on a reception signal for image generation received by a second ultrasonic probe of the same type as the first ultrasonic probe and sixth data indicating an attribute of the fifth data; A switching means for switching to the output of one of the plurality of trained models by a user operation, 3. The ultrasound diagnostic device according to claim 1, wherein the first estimation unit converts the third data into fourth data similar to the second data using a first trained model selected according to an attribute of the third data identified by the second trained model and the switching means, and outputs the fourth data.

12. The switching means includes a plurality of switching units, The ultrasonic diagnostic apparatus according to claim 11 , further comprising a selection unit that identifies third data and selects one of the plurality of switching units.

13. 3. The ultrasound diagnostic device according to claim 1, further comprising: a second estimation unit that outputs fifth data from the fourth data by using a classifier that is machine-learned using training data including a data set of fourth data obtained by converting the first data using the trained model and a predetermined correct answer label.

14. The ultrasonic diagnostic apparatus according to claim 1 or 2, a management device that receives and stores the third data, or the third data and the fourth data, from the ultrasonic diagnostic apparatus.

15. an estimation step of converting third data based on a reception signal for generating an image received by a first ultrasonic probe into fourth data similar to the second data and outputting the fourth data by using a trained model trained by machine learning using training data including first data based on a reception signal for generating an image received by a second ultrasonic probe of the same type as a first ultrasonic probe that transmits and receives ultrasonic waves to and from a subject, and second data captured under predetermined conditions; A data estimation method including:

16. Computer, an estimation unit that converts third data based on a reception signal for generating an image received by a first ultrasonic probe into fourth data similar to the second data and outputs the fourth data, using a trained model that is machine-learned using training data including first data based on a reception signal for generating an image received by a second ultrasonic probe of the same type as a first ultrasonic probe that transmits and receives ultrasonic waves to a subject, and second data captured under predetermined conditions; A program to function as a

Images