Machine learning device, training data creation device, training data creation method, and training data creation program

Abstract

To provide a learning model, a diagnosis program, an ultrasonic diagnostic device, an ultrasonic diagnostic system, an image diagnostic device, a machine learning device, a learning data generation device, a learning data generation method, and a learning data generation program that can provide higher accuracy.SOLUTION: Machine learning of a learning model is carried out using learning data consisting of a pair of first ultrasonic image data based on a reception signal for image generation received with an ultrasonic probe, and second correct answer data obtained by subjecting first correct answer data for second ultrasonic image data obtained by executing processing including coordinate conversion for the first ultrasonic image data, to inverse conversion of the coordinate conversion.SELECTED DRAWING: Figure 5

Description

Technical Field

[0005] , , ,

[0003] , , , , , , , ,

[0004] , , , , , , , , , machine ,

[0006]

[0001] This invention relates to , machine a machine learning device, a learning data creation device, a learning data creation method, and a learning data creation program.

Background Art

[0002] Conventionally, medical diagnoses have been made based on captured medical images. The medical images are converted into an appropriate coordinate system as appropriate for easy viewing by doctors and the like, and then displayed and output.

[0003] Regarding the diagnosis of such medical images, in order to prevent oversights due to variations in diagnostic ability and the like, technologies that use a learning model related to image recognition using a neural network or the like to perform automatic determination of images by learning through machine learning have been attracting attention. Patent Document 1 discloses a technique that uses probability information obtained from an image using a machine learning algorithm for the diagnosis of an image using ultrasonic echoes.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In machine learning, an expert generates correct data (teacher data) and inputs it together with learning data into a learning model to train the learning model. At this time, in learning related to image recognition, when using image data after processing such as the above coordinate transformation, a part of the information contained in the original image is removed and the amount of information decreases, so there is a problem that the learning accuracy decreases.

[0006] An object of this invention is to obtain a learning model with higher accuracy and be able to use it for diagnosis machineThe objective is to provide a machine learning device, a learning data creation device, a learning data creation method, and a learning data creation program. [Means for solving the problem]

[0007] To achieve the above objective, the invention described in claim 1 is: super First ultrasonic data based on the received signal for image generation received by the sound wave probe, and obtained by processing the first ultrasonic data including detection and coordinate transformation. Multiple ultrasound image data were combined. The second ultrasound image data was obtained by performing the inverse transformation of the coordinate transformation on the first ground truth data. multiple Second set of correct data The second correct data selected from among them and, Using training data consisting of pairs A machine learning device that performs machine learning on learning models. That is the case. [Effects of the Invention]

[0008] According to the present invention, it is possible to obtain a more efficient and accurate trained model and use it for diagnosis. [Brief explanation of the drawing]

[0009] [Figure 1] This diagram illustrates the configuration of the ultrasound diagnostic apparatus according to this embodiment. [Figure 2] This is a block diagram showing the functional configuration of an ultrasound diagnostic device. [Figure 3] This is a block diagram showing the functional configuration of an electronic computer. [Figure 4] This diagram explains how to create training data. [Figure 5] This flowchart shows the control procedure for the training data creation process. [Figure 6] This flowchart shows the control procedure for the learning control process. [Figure 7] This diagram illustrates the processing performed by the image processing unit. [Figure 8] This figure shows an example of target detection using a learning model. [Figure 9] It is a flowchart showing the control procedure of ultrasonic diagnosis control processing. [Figure 10] It is a diagram for explaining an example of spatial compounding and setting of teaching data for a spatial compounding image. [Figure 11] It is a diagram for explaining an example of setting teaching data from a spatial compounding image.

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present invention will be described based on the drawings. FIG. 1 is a diagram for explaining the configuration of an ultrasonic diagnostic apparatus 1 (ultrasonic diagnostic system) according to the present embodiment. The ultrasonic diagnostic apparatus 1 includes a main body unit 10 and an ultrasonic probe 20.

[0011] The ultrasonic probe 20 is a probe that transmits ultrasonic waves to a subject and receives the reflected waves. The ultrasonic probe 20 has a plurality of vibrators as piezoelectric members, and when a voltage is applied to each vibrator at an appropriate frequency, the vibrator is deformed to generate ultrasonic waves. Also, the ultrasonic waves are received by converting the deformation of the vibrator caused by the input ultrasonic waves into an electrical signal, and the obtained electrical signal is output as a received signal for image generation. The ultrasonic probe 20 has a signal cable 22, and a connection terminal (not shown) located at one end of the signal cable 22 is connected to the main body unit 10, so that an electrical signal for transmitting the ultrasonic waves to be transmitted is sent from the main body unit 10 to the ultrasonic probe 20, and the received signal is sent from the ultrasonic probe 20 to the main body unit 10.

[0012] The main body unit 10 performs control related to the transmission and reception of the above ultrasonic waves, and also includes an operation reception unit 18 and a display unit 19. The display unit 19 displays the status and menu of the ultrasonic diagnostic apparatus 1, photographed images, diagnostic results, and the like. The display unit 19 has, for example, a liquid crystal display (LCD) as a display screen, but is not limited thereto. It may have other things, for example, an organic EL display.

[0013] The operation reception unit 18 receives an external input operation by a user or the like, and outputs the content of the received input operation as an input signal to the control unit 11 (see FIG. 2). The operation reception unit 18 may have some or all of a keyboard, a keypad, a push button switch, a slide switch, a toggle switch, a rocker switch, and the like.

[0014] FIG. 2 is a block diagram showing the functional configuration of the ultrasonic diagnostic apparatus 1. The main body unit 10 of the ultrasonic diagnostic apparatus 1 includes a control unit 11, a transmission drive unit 12, a reception drive unit 13, a transmission / reception switching unit 14, an image processing unit 15, a communication unit 17, an operation reception unit 18, a display unit 19, and the like.

[0015] The transmission drive unit 12 outputs a pulse signal to be supplied to the ultrasonic probe 20 in accordance with a control signal input from the control unit 11, and causes the ultrasonic probe 20 to generate ultrasonic waves. The transmission drive unit 12 includes, for example, a clock generation circuit, a pulse generation circuit, a pulse width setting unit, and a delay circuit. The clock generation circuit is a circuit that generates a clock signal for determining the transmission timing and transmission frequency of the pulse signal. The pulse width setting unit sets the waveform (shape), voltage amplitude, and pulse width of the transmission pulse output from the pulse generation circuit. The pulse generation circuit generates a transmission pulse based on the setting of the pulse width setting unit and outputs it to each individual vibrator of the ultrasonic probe 20 via different wiring paths. The delay circuit counts the clock signal output from the clock generation circuit, and when the set delay time has elapsed, causes the pulse generation circuit to generate a transmission pulse and outputs it to each wiring path.

[0016] The receiving drive unit 13 is a circuit that acquires the received signal input from the ultrasonic transducer 20 according to the control of the control unit 11. The receiving drive unit 13 includes, for example, an amplifier, an A / D conversion circuit, and a phase-correcting summing circuit. The amplifier is a circuit that amplifies the received signal corresponding to the ultrasonic waves received by each transducer of the ultrasonic transducer 20 at a predetermined amplification factor. The A / D conversion circuit is a circuit that converts the amplified received signal into digital data at a predetermined sampling frequency. The phase-correcting summing circuit is a circuit that adjusts the time phase of the A / D converted received signal by giving a delay time to the wiring path corresponding to each transducer, and then adds these together (phase-correcting summing) to generate sound line data.

[0017] The transmit / receive switching unit 14, based on the control unit 11, performs a switching operation to cause the transmit drive unit 12 to send a drive signal to the transducer when ultrasonic waves are emitted (transmitted) from each transducer, and to output a receive signal to the receive drive unit 13 when the transducer acquires a signal related to the ultrasonic waves emitted.

[0018] The image processing unit 15 generates a diagnostic image (second ultrasound image) based on the received ultrasound data (received signal). The image processing unit 15 includes a storage unit 151, a processing unit 152 (output unit), a coordinate transformation unit 153, a synthesis unit 154, and the like.

[0019] The processing unit 152 detects (envelope detection) the sound line data (RF data) input from the receiving drive unit 13 to acquire a signal, and performs intermediate processing as needed, such as logarithmic amplification (logarithmic compression), STC (Sensitivity Time Control), filtering (e.g., low-pass filter, smoothing, dynamic filter, etc.), and enhancement processing. The processing unit 152 may also be capable of frequency analysis processing such as FFT Doppler (power Doppler) processing and color Doppler processing. The processing unit 152 outputs the generated image (intermediate processed image).

[0020] The processing unit 152 can detect the structure (including external shape, etc.) of the object of detection (object of interest) from the generated intermediate processing image and generate data that displays the structure and characteristics in a way that the user can identify and understand. For structure detection of the object of detection in the image processing unit 15, a machine learning model 1521 (trained model) that has been trained to detect the object of detection from the input image is used. That is, when an image in which the object of detection is to be detected is input to the machine learning model 1521, the machine learning model 1521 detects the characteristic structure of the object of detection and outputs it as a distribution of the probability (also called confidence) that each pixel position is included in the structure. From the output of the machine learning model 1521, the data can be further converted into data that can be overlaid and displayed on a diagnostic image, such as contour lines when the probabilities are binarized at a certain threshold, or characteristic values ​​(physical quantities) of the structure of the object of detection, such as length (width), width (height), diameter (diameter, radius, major axis, minor axis) of circular or elliptical structures, area, centroid (center) position, perimeter, distance between specific positions of the structure, etc. may be obtained. Furthermore, if the three-dimensional shape can be identified and estimated, volume, surface area, height, and depth may also be determined. The generation and use of the learning model 1521 will be described in detail later.

[0021] The areas that can be detected and diagnosed by the ultrasound diagnostic device 1 are not particularly limited, but examples include the lungs, heart (heart wall, valve annulus, etc.), blood vessels (region, location) such as the inferior vena cava, nerves, and muscles. Furthermore, fetuses may also be detected from the examination images. In addition to the human body itself, medical devices used for examinations and treatments, such as catheters and puncture needles, may also be detected. Moreover, these are not limited to specific states (static states), but may also be identified as changes such as contraction / expansion associated with respiration or pulse (heartbeat). The learning model 1521 may be learned and generated separately for each detection target. Multiple states corresponding to changes in a certain area may be detectable within the same learning model 1521.

[0022] The coordinate transformation unit 153 performs a process (such as digital scan conversion; DSC) to transform the intermediate processed image generated by the processing unit 152 to match the coordinates of the display screen (each pixel position). For example, when outputting frame image data related to B-mode display as one of the diagnostic images, in which the two-dimensional structure in the plane (structure inside the subject) including the signal transmission direction (incident direction, depth direction of the subject) and the direction of the ultrasound transmitted by the ultrasound probe 20 (one scanning cycle) is represented in a Cartesian coordinate system, the coordinate transformation unit 153 performs a transformation from the coordinate system of the original received signal to a Cartesian coordinate system. Coordinate transformation will be described later. The coordinate transformation unit 153 may also perform image adjustment processing such as gamma correction, but such processing may be performed by the processing unit 152.

[0023] The image processing unit 15 outputs the diagnostic image, which has been coordinate-transformed by the coordinate transformation unit 153, to the display unit 19 or the like. The diagnostic image may be output as is, or it may be returned to the processing unit 152 and output directly or after fine-tuning to the display unit 19 or the like from the processing unit 152. For example, if characteristic values ​​(physical quantities) as described above are to be measured and calculated based on the diagnostic image after coordinate transformation, the processing unit 152 may perform these measurements and calculations after the coordinate transformation processing by the coordinate transformation unit 153. If the output is made directly from the coordinate transformation unit 153, the output processing from the coordinate transformation unit 153 to the display unit 19 may be included in the configuration of the output unit of the present invention.

[0024] The image processing unit 15 has a storage unit 151. A program related to medical diagnosis (diagnostic program 1511) using diagnostic images and the output results of the learning model 1521 is stored in the storage unit 151. The learning model 1521 may also be stored in the storage unit 151 and used by the processing unit 152. The storage unit 151 includes, for example, non-volatile memory such as flash memory or an HDD (Hard Disk Drive).

[0025] The synthesis unit 154 synthesizes multiple images (images after intermediate processing including coordinate transformation) by performing spatial compounding, frequency compounding, time averaging, smoothing, etc., and outputs the combined images by performing processes such as alignment and weighting. In addition, the synthesis unit 154 may also perform synthesis and decomposition of probability distribution images output by the learning model 1521, as shown in the modified example described later.

[0026] The image processing unit 15 may, as a control unit, be equipped with a dedicated CPU and RAM used for generating (image processing) diagnostic images and output images of the learning model 1521, and may also be equipped with a GPU (Graphics Processing Unit) for image processing. Alternatively, the image processing unit 15 may be equipped with a dedicated hardware configuration for image generation formed on a board (such as an ASIC (Application-Specific Integrated Circuit)). Or, the image processing unit 15 may be configured so that image generation processing is performed by the CPU and RAM of the control unit 11. The processing of the processing unit 152, the coordinate transformation unit 153, and the synthesis unit 154 may be performed by a common CPU (processor), or each may be assigned a separate processor.

[0027] The communication unit 17 controls communication with the outside world in accordance with a predetermined communication standard. Examples of communication standards include those related to LANs (such as TCP / IP), and it can perform tasks such as transmitting diagnostic images and receiving trained machine learning models with the computer 40 described later.

[0028] The operation reception unit 18 is equipped with a push-button switch, a keyboard, a mouse, a trackball, or a touch panel positioned over the display screen, or a combination thereof, and generates an operation signal corresponding to the user's input and inputs the operation signal to the control unit 11.

[0029] The display unit 19 comprises a display screen and a drive unit, which are based on one of various display methods, such as an LCD (Liquid Crystal Display), an organic EL (Electro-Luminescent) display, an inorganic EL display, a plasma display, or a CRT (Cathode Ray Tube) display. The display unit 19 drives the display screen (each display pixel) according to control signals output from the control unit 11 and image data generated by the image processing unit 15, and displays menus and status related to ultrasound diagnosis, as well as captured images and diagnostic results based on received ultrasound, on the display screen. The display unit 19 may also be configured to include LED lamps or the like to display information related to power supply status, operational abnormalities, etc.

[0030] These operation reception units 18 and display units 19 may be integrated into the housing of the main unit 10, or they may be attached to the main unit 10 via RGB cables, USB cables, HDMI cables (registered trademark: HDMI), etc. Furthermore, if the main unit 10 is provided with operation input terminals and display output terminals, peripheral devices for operation reception and display may be connected to these terminals for use.

[0031] The ultrasonic transducer 20 emits ultrasound waves (in this case, approximately 1 to 30 MHz) and directs them towards a subject such as a living organism. It also functions as an acoustic sensor that receives the reflected waves (echoes) of the emitted ultrasound waves reflected by the subject and converts them into electrical signals. The ultrasonic transducer 20 is equipped with an array of multiple transducers for transmitting and receiving ultrasound waves, and a signal cable 22, etc.

[0032] The signal cable 22 has a connector (not shown) at one end for connection to the main unit 10. The ultrasonic probe 20 is detachable from the main unit 10 via this signal cable 22. The user operates the ultrasonic diagnostic device 1 by bringing the ultrasonic transmitting and receiving surface of the ultrasonic probe 20 into contact with the subject with appropriate pressure to perform an ultrasonic diagnosis.

[0033] Some ultrasonic probes 20 are capable of emitting ultrasonic waves using one or more of the following methods (sequence and direction): linear scanning (straight), sector scanning (radial), convex scanning (fan-shaped), and arc scanning (bow-shaped). Furthermore, the ultrasonic probes 20 themselves come in various structural types depending on the scanning method, such as having transducers arranged linearly on a plane or arranged convexly on a curved surface. An appropriate ultrasonic probe 20 can be selected according to the application and connected to a single main unit 10, and ultrasonic waves can be transmitted and received using the appropriate scanning method. Furthermore, the connection between the main unit 10 and the ultrasonic probe 20 may be made using wireless communication means such as infrared rays or radio waves, rather than a wired signal cable 22.

[0034] On the other hand, in this embodiment, the learning model 1521 is generated (learned) separately outside the ultrasound diagnostic device 1, and the completed model is copied to the ultrasound diagnostic device 1. Furthermore, the learning data related to the learning process is also created externally.

[0035] Figure 3 is a block diagram showing the functional configuration of the computer 40, which is a machine learning device and training data creation device in this embodiment. The electronic computer 40 may be a general PC (computer) and includes a control unit 41, a storage unit 45, a communication unit 47, a display unit 48, an operation reception unit 49, and the like.

[0036] The control unit 41 has a hardware processor that performs arithmetic processing and provides overall control over the operation of the electronic computer 40. The hardware processor may include a CPU (Central Processing Unit) and RAM (Random Access Memory), as well as logic circuits configured to perform specific processing, such as an ASIC (Application Specific Integrated Circuit).

[0037] The storage unit 45 has non-volatile memory. Non-volatile memory may include flash memory and HDD (Hard Disk Drive). The storage unit 45 may also have volatile memory (such as DRAM) for temporarily storing large amounts of image data and intermediate processing data. The storage unit 45 stores a machine learning model 451 and its learning parameters for detecting and estimating (inferring) the presence, location, and structure of a target object (object of interest) in a subject from the ultrasound measured by the ultrasound diagnostic device 1, training data 452 for training the machine learning model 451, and a training data creation program 453 that controls the process for creating the training data 452. The training of the machine learning model 451 is supervised learning, meaning that the training data 452 includes image data that serves as input data and training data (ground truth data) associated with each image data.

[0038] The communication unit 47 controls communication with the outside world using a predetermined communication standard. The communication standard includes network communication standards such as LANs, and the communication unit 47 has a network card or the like that corresponds to the said communication standard.

[0039] The display unit 48 has a display screen and displays various contents on the display screen based on the control of the control unit 41. The display screen is, for example, a liquid crystal display (LCD), but is not limited to this.

[0040] The operation reception unit 49 receives input operations from an external source and outputs an operation signal to the control unit 41 according to the content of the received input operation. The operation reception unit 49 includes a pointing device such as a mouse, and may also include a keyboard or push-button switches. Alternatively, or in addition to these, the operation reception unit 49 may include a touch panel or the like that is positioned on top of the display screen of the display unit 48.

[0041] The display unit 48 and the operation reception unit 49 may be peripheral devices connected to the connection terminals by a cable of any of the various standards, or capable of wirelessly exchanging data by Bluetooth® or 2.4GHz wireless communication.

[0042] Next, we will explain inference using the learning model 1521 and the creation of training data. As described above, the ultrasound diagnostic device 1 generates a measurement image based on the signal received by the ultrasound probe 20 and displays the measurement image. In addition, the ultrasound diagnostic device 1 can detect and add identifiable objects or structures to be detected during the display process, or calculate and add parameters related to the position, size, and shape of the detected object to the display. For detecting the object (estimating the position range), a learning model 1521 is used. The learning model 1521 is based on a known algorithm related to image recognition, and may include, for example, a convolutional neural network (CNN).

[0043] In the computer 40 that trains (machine learns) the machine learning model 451 to generate the learning model 1521, training data 452 is created in advance before training. The training data 452 is image data that will be used as input to the machine learning model 451 during training, with training data (ground truth data) attached to it. In this case, the training data defines, for example, the range (mask) of objects or structures to be detected within the image data, and this range is set by experts (for example, doctors or clinical laboratory technicians) who are proficient in interpreting results from ultrasound medical images.

[0044] Here, as described above, various processes, including coordinate transformation, are performed on the measurement data in the image processing unit 15 to obtain a diagnostic image, so some of the information contained in the original measurement data is lost or altered. As a result, when learning with diagnostic images, the accuracy and efficiency of learning, especially related to the detection of clinically meaningful structures, may decrease. The machine learning model 451 is more likely to improve the accuracy of its judgment if it is trained using intermediate processed images (first ultrasound image data) that are earlier than the final diagnostic image, especially before, during, or after intermediate processing other than coordinate transformation, rather than using the final diagnostic image as input data.

[0045] On the other hand, experts who set up training data usually only view and use the final diagnostic images, so directly setting up training data for intermediate processing images is at least time-consuming and often very difficult. Therefore, in the computer 40 of this embodiment, intermediate processing including coordinate transformation is performed on an intermediate processing image (first ultrasound image data) at a certain stage to obtain a diagnostic image (second ultrasound image data), the correct position range (second correct data) in the intermediate processing image is identified by performing the inverse transformation of the above coordinate transformation from the correct position range (first correct data) set for the diagnostic image, and the identified position range is included as training data (training data for machine learning) in association with the original intermediate processing image (as a pair) as training data.

[0046] Figure 4 illustrates the creation of training data. As is typically done with ultrasound diagnostic equipment 1, the intermediate processing image P1, represented in the coordinate system related to measurement, is transformed into the coordinate system related to display, i.e., the Cartesian coordinate system, to become the diagnostic image P2. For example, in the case of B-mode diagnostic images, if sector scanning or convex scanning is performed by the ultrasound probe 20 during measurement, data is obtained in polar coordinates, and a coordinate transformation is performed from polar coordinates to Cartesian coordinates. In addition, in polar coordinates, the data acquisition density changes depending on the radial value (distance from the origin), so interpolation between pixels is performed to obtain data points (pixel values) at uniform intervals in the Cartesian coordinate system. When linear scanning is performed by the ultrasound probe 20, the measurement data is also obtained in Cartesian coordinates, but usually the aspect ratio on the data does not match the aspect ratio of the actual size, or the two axes on the data are not orthogonal because the direction of ultrasound transmission and reception is set diagonally. Therefore, in these cases, the measurement data is transformed into a diagnostic image represented in Cartesian coordinates corresponding to the actual aspect ratio by affine transformation or projection transformation. Furthermore, the process between the intermediate processing image P1 and the diagnostic image P2 may include not only coordinate transformation but also various image processing steps described above to appropriately generate and improve the clarity of the diagnostic image P2 (these steps together may include coordinate transformation).

[0047] In the computer 40, processing including the above coordinate transformation may be performed separately using the same procedure as in the ultrasound diagnostic device 1, or the computer 40 may acquire images before and after the processing including the above coordinate transformation by acquiring a set of intermediate processing images P1 and diagnostic images P2 (image set) generated by the ultrasound diagnostic device 1. The images acquired by the computer 40 may be from a single ultrasound diagnostic device 1 or from multiple ultrasound diagnostic devices 1.

[0048] A first correct answer data C1 is set for the diagnostic image P2 by an expert (a person proficient in the above result judgment). The computer 40 may set a provisional correct answer range by applying a simple algorithm for detecting a simple target to the diagnostic image P2. When the diagnostic image P2 and the above provisional correct answer range are set, the range is displayed by the display unit 48, and the person in charge can set or modify the correct answer range by operating the operation reception unit 49 while looking at the diagnostic image P2, thereby generating the first correct answer data C1 (acquisition process). Subsequently, the first correct answer data C1 is inversely transformed to obtain second correct answer data C2 that shows the correct answer range in the same coordinate system as the intermediate processing image P1 (inverse transformation process). At this time, the inverse transformation of processes other than the coordinate transformation among the processes including the above coordinate transformation does not need to be performed.

[0049] As described above, the coordinate transformation parameters (matrix) differ depending on the scanning method (sector scanning, convex scanning, linear scanning) of the ultrasonic transducer 20, and, if necessary, the type of ultrasonic transducer 20. The diagnostic image P2 is supplemented with additional information (transmission direction information) such as the type of probe used and scanning phase information via an alpha channel (metadata, header data, etc.). By referring to this additional information, it is possible to determine what transformation parameters should be used for the inverse transformation.

[0050] The training data 452 for the machine learning model 451 is not simply a matter of quantity; its selection is also important. For example, if there are typical patterns that can occur as the structure to be classified, a person capable of classifying the structure to be detected (who may be different from an expert and whose level of proficiency may be lower than that of an expert) can manually select the necessary number of image data in an appropriate proportion for each pattern from a large number of pre-acquired image data, and use this selected data to create the training data 452. Alternatively, classification based on setting a provisional correct range as described above may be used before manual selection. The intermediate processed image P1 and the second correct data C2 are associated (paired) and stored in the learning data 452 of the memory unit 45 (memory control process).

[0051] Figure 5 is a flowchart showing the control procedure by the control unit 41 for the learning data creation process executed on the computer 40. This learning data creation process, which is the learning data creation method of this embodiment, is started, for example, when the user of the computer 40 specifies a dataset of measurement data to be used as learning data, as described above, and the learning data creation program 453 is started in response to a predetermined start command.

[0052] When the training data creation process is started, the control unit 41 acquires one unselected image data from the specified dataset (step S401). The image data includes the intermediate processing image P1 and the diagnostic image P2.

[0053] The control unit 41 sets a provisional correct range using a simple detection algorithm (step S402). The control unit 41 displays the diagnostic image P2 and the provisional correct range using the display unit 48 (step S403). As described above, the process in step S402 is not required, and in this case, the control unit 41 does not display the provisional correct range in the process of step S403 either. The control unit 41 waits for an input operation from the operation reception unit 49 and acquires information on the correct range of the object that will become the first correct data C1 based on the content of the input operation (step S404). The processes in steps S401 and S404 constitute the acquisition step (acquisition function) of the learning data creation method (learning data creation program) of this embodiment.

[0054] The control unit 41 refers to the supplementary information of the diagnostic image P2 and determines the inverse coordinate transformation parameters (transformation matrix) according to the scanning method and phase (and, if necessary, the type of ultrasonic probe 20) (step S405). The control unit 41 inversely transforms the correct range of the acquired first correct data C1 into second correct data C2 that indicates the correct range in the image range of the intermediate processed image P1 using the determined inverse transformation parameters (step S406; inverse transformation step, inverse transformation function).

[0055] The control unit 41 associates the obtained second correct answer data C2 with the intermediate processed image P1 and adds it to the training data 452 (step S407; memory control step, memory control function). The control unit 41 adds the training data to be added to the training data 452 of the storage unit 45. The control unit 41 determines whether all image data has been selected from the input dataset (step S408). If it is determined that not all image data has been selected (that there is unselected image data) (NO in step S408), the control unit 41 returns the process to step S401. If it is determined that all image data has been selected (YES in step S408), the control unit 41 terminates the training data creation process.

[0056] Once the training data 452 is generated in this way, the machine learning model 451 is trained using the training data 452. As is well known, machine learning is performed, for example, by inputting a diagnostic image P2 from the training data 452 to estimate (infer) the structure of the object, comparing the result of the inference with the training data, and feeding back (backpropagating) the difference (loss function) to the parameters.

[0057] Figure 6 is a flowchart showing the control procedure by the control unit 41 of the learning control process executed by the computer 40. This process is started in response to an input operation by the user of the computer 40 to the operation reception unit 49, specifying the generated learning data 452 mentioned above as a start command.

[0058] The control unit 41 sets the machine learning model 451 to be trained (step S421). The control unit 41 acquires the specified training data 452 (step S422). The control unit 41 sequentially inputs the training data 452 to the machine learning model 451 and performs machine learning by improving the parameters based on a comparison of the output results from the machine learning model 451 for the intermediate processing image P1 with the training data (step S423). When all the training data 452 has been input and machine learning is complete, the control unit 41 terminates the learning control process.

[0059] The machine learning model 451 (trained model) that has undergone machine learning is sent to the ultrasound diagnostic device 1, where it is stored as the trained model 1521 and used for estimating (inference) the structure of the target to be detected from the measurement image based on the received signal. It is not necessary for the trained model to be sent directly from the computer 40 to the ultrasound diagnostic device 1. The trained model may be sent to a management server for managing versions of trained models held by multiple ultrasound diagnostic devices 1, and then the trained model may be transmitted and distributed from the management server to the ultrasound diagnostic devices 1.

[0060] Figure 7 illustrates the processing performed by the image processing unit 15 in the ultrasound diagnostic device 1. When a received signal obtained from a normal measurement using the ultrasound probe 20 is input to the image processing unit 15, the image processing unit 15 generates an intermediate processed image P1 based on the received signal.

[0061] The intermediate processing image P1 (third ultrasound image data) is input to the learning model 1521, and a probability distribution image A1 (first inference result) showing the probability (confidence level) that each pixel position is included in the above structure is output. The intermediate processing image P1 and the probability distribution image A1 are coordinate-transformed to obtain a diagnostic image P2 and a probability distribution image A2 (second inference result) expressed in the same coordinate system as the diagnostic image P2. The transformation parameters for the coordinate transformation can be determined based on the supplementary information attached to the intermediate processing image P1 (similar to the diagnostic image P2 above, including the scanning method, scanning phase, and, if necessary, transmission direction information such as the type of ultrasound probe 20).

[0062] The probability distribution image A2 may be binarized at a predetermined threshold depending on the content and settings to be displayed on the display unit 19, or its brightness distribution may be transformed (applied) by referring to a lookup table (LUT) that changes the brightness values ​​to make it easier to see on the display screen. In addition, characteristic values ​​(physical quantities) of the above-mentioned structure may be measured and calculated based on the range of the structure identified by binarization. Performing these processes after coordinate transformation can suppress the occurrence of counterproductive effects that would rather emphasize unwanted noise.

[0063] The display unit 19 may display the inference result (second inference result) overlaid on the diagnostic image P2, or part or all of the inference result may be displayed in a separate window from the diagnostic image P2.

[0064] Figure 8 shows an example of target detection using the learning model 1521. As shown in the diagnostic image in Figure 8(a), when blood vessels near the liver, including the inferior vena cava Ba and hepatic vein Bb, are imaged, the learning model 1521 obtains their probability frequency distributions from the intermediate processed image. As shown in Figure 8(b), a region Ra with a high probability distribution, which is the area of ​​the inferior vena cava Ba, and a region Rb with a high probability distribution, which is the area of ​​the hepatic vein Bb, are shown. As shown in Figure 8(c), the contour R2a of the inferior vena cava Ba is obtained by comparing its probability distribution with an appropriate threshold. In addition, the point showing the maximum value in the probability distribution of the hepatic vein Bb can be obtained as the position R2b of the hepatic vein.

[0065] Figure 9 is a flowchart showing the control procedure by the control unit of the image processing unit 15 of the ultrasound diagnostic control process performed in the ultrasound diagnostic device 1. This ultrasound diagnostic control process is started when the diagnostic program 1511 is activated each time a received signal is input from the ultrasound probe 20.

[0066] The image processing unit 15 acquires data from the incoming received signal (step S101). The image processing unit 15 (processing unit 152) generates an intermediate processed image P1 based on the received signal (step S102).

[0067] The image processing unit 15 (processing unit 152) inputs the data of the intermediate processing image P1 into the machine learning model (step S103). The image processing unit 15 obtains the inference results output from the learning model (including the probability distribution image A1 relating to the extent of the structure) (step S104; output function).

[0068] The image processing unit 15 (coordinate transformation unit 153) sets coordinate transformation parameters based on the supplementary information of the intermediate processing image P1 and the image display mode (step S105). The image processing unit 15 (coordinate transformation unit 153) performs image processing, including the process of transforming the coordinates of the intermediate processing image P1 and the probability distribution image A1 using the above coordinate transformation parameters (step S106). The image processing unit 15 performs processing related to the display adjustment of the diagnostic image P2 obtained by image processing, such as gamma correction and contrast adjustment (step S107). The image processing unit 15 calculates characteristic values ​​from the probability distribution image A2 after coordinate transformation as needed (step S108). The image processing unit 15 displays the obtained diagnostic image P2 and the inference results on the display unit 19 (step S109). Then, the image processing unit 15 terminates the ultrasound diagnostic control processing.

[0069] Although the above description explains that the received signal from the ultrasound probe 20 is processed in near real-time, this is not the only option. For example, steps S103, S104, S108, etc., may be omitted in real-time processing and displayed, while the intermediate processing image P1 is stored and retained. Later, when a clinical laboratory technician or physician makes a diagnosis, the processing from step S103 onward may be performed using this intermediate processing image P1.

[0070] [Differentiation] In the above embodiment, it was described that there is a one-to-one correspondence between the intermediate processing image and the diagnostic image. However, in medical diagnosis, multiple intermediate processing images are often combined after intermediate processing to obtain a single final diagnostic image. Examples of such cases include images taken from multiple directions at timings where temporal changes are negligible (spatial compounding), images taken from the same range (shooting direction) using ultrasound of multiple frequencies (frequency compounding), superposition of multiple images taken at the same frequency and range (shooting direction), and temporal smoothing. In such cases, the inference results obtained from individual intermediate processing images may also be combined after coordinate transformation. The combination may be, for example, a simple average, or a weighted average according to the shooting conditions of the intermediate processing images. Furthermore, even if the diagnostic image is not actually combined, the coordinate-transformed inference results obtained from multiple intermediate processing images (second inference result) may be combined and displayed as a common inference result for multiple diagnostic images corresponding to each intermediate processing image. Even when using the learning model 1521, if sufficient accuracy cannot be obtained from a single intermediate processing image, this synthesis process can improve accuracy by increasing the signal-to-noise ratio, making it easier for doctors and other medical professionals to make diagnoses. This synthesis of inference results is performed by the synthesis unit 154 of the ultrasound diagnostic device 1, along with the synthesis process for diagnostic images.

[0071] Figure 10 illustrates an example of spatial compounding and the generation of inference results for spatial compounding images. The inference results T1 to T3 of the target structure detected in the three captured images D1 to D3 shown in Figure 10(a) can be combined and output as a single inference result T0 shown in Figure 10(b).

[0072] Conversely, when generating training data 452 by creating training data from synthesized diagnostic image data, the training data may be inversely transformed to the coordinate systems of multiple intermediate processing images before synthesis, thereby obtaining training data represented in the coordinate systems of each of those multiple intermediate processing images. This process is performed by the control unit 41 of the computer 40. That is, in creating training data 452, there may be multiple intermediate processing images P1 corresponding to the diagnostic image P2, and in this case, there may be multiple second ground truth data C2 that are inversely transformed from the first ground truth data C1, and some or all of the inverse transformation parameters for obtaining these multiple second ground truth data C2 may be different from each other.

[0073] Figure 11 illustrates an example of setting training data from a spatial compound image. As shown in Figure 11(a), the inference result T0 set for a single spatial compound image is decomposed according to the shooting direction of multiple images synthesized during the generation of the spatial compound image, as shown in Figure 11(b), and an inverse coordinate transformation is performed to divide it into multiple (3) intermediate processing images corresponding to the correct range Ta~Tc.

[0074] In such cases, it is not necessary to include all of the divided intermediate processing images in the training data. For example, in the example in Figure 11, it may be set so that only one or two of the three intermediate processing images are included in the training data, and accordingly, training data obtained by inversely transforming only the coordinate system corresponding to that single intermediate processing image may be obtained. The selected intermediate processing images may be from a fixed shooting direction with good specific accuracy according to the direction, or one intermediate processing image may be selected at a predetermined number of times regardless of the shooting direction.

[0075] As described above, the learning model 1521 of this embodiment was trained using learning data consisting of pairs of training data: an intermediate processed image P1 based on a received signal for image generation received by the ultrasonic probe 20, and a second ground truth data C2 obtained by performing an inverse transformation of the coordinate transformation on a first ground truth data C1 for a diagnostic image P2 obtained by performing intermediate processing including coordinate transformation on the intermediate processed image P1. Thus, by using the intermediate processing image P1, which is an earlier stage than the final diagnostic image P2, as input to the learning model 1521, it becomes possible to make judgments including the loss of information due to processing to make the diagnostic image P2 easier for doctors to view. As a result, the learning model 1521 can output more accurate and precise inferences. Furthermore, by using this learning model 1521 in ultrasound diagnosis, it becomes possible to perform more accurate diagnoses. On the other hand, when creating the training data 452 used to train the machine learning model 451, it is difficult, if not time-consuming, for doctors to directly assign correct answers to the intermediate processing image P1, which they are not familiar with, in order to generate training data. Therefore, by inversely transforming the second correct answer data C2, which has correct answers assigned to the diagnostic image P2, to obtain the first correct answer data C1 corresponding to the intermediate processing image P1, it is possible to easily create training data 452 for training the machine learning model 451, which can produce more accurate output, and obtain the learning model 1521.

[0076] Furthermore, the diagnostic image P2 may also be a B-mode image. In B-mode images, which are generally measured time-series in polar coordinates, the intermediate processed image P1 in the original polar coordinates looks significantly different from the diagnostic image P2. Therefore, by adding the first ground truth data C1 to the diagnostic image P2 as described above and then performing the inverse transform, the training data 452 can be obtained particularly easily.

[0077] Furthermore, the coordinate transformation may include interpolation between pixels. In polar coordinate system measurements as described above, the interval per predetermined azimuthal angle changes according to the radial movement, so if each point is directly transformed into a Cartesian coordinate system diagnostic image P2, the pixel points will be non-uniform. In such cases, interpolation (especially linear interpolation) between pixels can be performed to obtain a uniform display image, making the diagnostic image P2 easier to view. Similarly, each point of the ground truth data assigned to the diagnostic image P2 can also be appropriately represented in polar coordinates.

[0078] Furthermore, the first ground truth data C1 is inversely transformed based on the ultrasound transmission direction information to obtain the second ground truth data C2. Since the ultrasound probe 20 is normally scanned periodically, the scanning information attached to each diagnostic image P2 (frame image) is acquired as the ultrasound transmission direction information, allowing the position of each pixel in each diagnostic image P2 to be easily identified. Therefore, the first ground truth data C1 can be easily converted to the second ground truth data C2 based on this transmission direction information.

[0079] In particular, since the ultrasound transmission direction information is included in the header of the intermediate processing image P1 (and the diagnostic image P2), it is possible to easily perform coordinate transformation processing without having to separately acquire information for coordinate transformation or inverse transformation.

[0080] Furthermore, the diagnostic program 1511 of this embodiment uses the learning model 1521 described above to enable the computer to implement an output function that outputs a first inference result (probability distribution image A1) from ultrasound image data (intermediate processed image P1) before intermediate processing (including coordinate transformation) based on the received signal for image generation received by the ultrasound probe 20. By executing the diagnostic program 1511, which utilizes the learning model 1521, and detecting the target of interest (object of interest) with greater accuracy using the ultrasound diagnostic device 1 or the computer of an external electronic device, it is possible to suppress physician oversights and reduce the degree of dependence on the physician's experience and ability, thereby enabling more stable and reliable diagnoses.

[0081] Furthermore, the ultrasound diagnostic apparatus 1 of this embodiment includes an ultrasound transducer 20 that transmits and receives ultrasound to and from a subject, and a processing unit 152 that uses the above-mentioned learning model 1521 to output a first inference result (probability distribution image A1) from ultrasound image data (intermediate processed image P1) before intermediate processing (including coordinate transformation) based on the received signal for image generation received by the ultrasound transducer 20. According to this ultrasound diagnostic device 1, more accurate detection results can be obtained quickly from measurement data acquired using the ultrasound probe 20.

[0082] Furthermore, the ultrasound diagnostic device 1 further includes a coordinate transformation unit 153 that performs a coordinate transformation on the first inference result (of which, such as the probability distribution image A1, requires or is possible to perform a coordinate transformation) to obtain a second inference result (probability distribution image A2). The processing unit 152 outputs the second inference result after the coordinate transformation (such as the probability distribution image A2 and characteristic values ​​based on the probability distribution image A2). As described above, by obtaining data based on the original coordinate system, such as the probability distribution image A1, based on an image during processing, and then performing the same coordinate transformation on this image (data) as on the intermediate processing image P1, it is possible to obtain an inference result in the same coordinate system as the diagnostic image P2 with higher accuracy than the inference result that can be obtained with a learning model that inputs the diagnostic image P2 itself. In this case, if the first inference result includes a result that does not require a coordinate transformation, the coordinate transformation processing by the coordinate transformation unit 153 does not need to be performed on that result.

[0083] Furthermore, the ultrasound diagnostic device 1 has a display unit 19 that displays a second inference result (such as a probability distribution image A2). By displaying the probability distribution image A2, which is represented in the same coordinate system as the diagnostic image P2, on the display unit 19, users of the ultrasound diagnostic device 1, such as doctors, can easily visually confirm the detection results of the target to be detected with greater accuracy and perform a diagnosis.

[0084] Furthermore, the ultrasound diagnostic device 1 has a synthesis unit 154 that performs synthesis processing of multiple probability distribution images A2 related to the second inference result. By further synthesizing multiple probability distribution images A2 that cannot be obtained with sufficient accuracy even with the learning model 1521 of this embodiment, more accurate detection results can be obtained. In addition, if the diagnostic image P2 is originally output by synthesizing multiple images such as a spatial compound image or a frequency compound image, the probability distribution image A2 can also be synthesized in accordance with these synthesis results, thereby appropriately corresponding the detection results by the learning model 1521 to the diagnostic image P2.

[0085] Furthermore, the processing unit 152 may binarize or classify the second inference result (such as the probability distribution image A2 or characteristic values), or apply a lookup table to convert the values ​​to the second inference result. The obtained second inference result can then be output as an image that is easier to diagnose or as useful parameters. This makes it possible for doctors and others to make diagnoses more easily and accurately.

[0086] Furthermore, the processing unit 152 may binarize the second inference result (probability distribution image A2) and, based on this binarized inference result, estimate at least one of the position, area, volume, length, height, width, depth, and diameter associated with the object of interest (detection target) of the subject. By binarizing the probability distribution image in this way and identifying the range of the detection target, it becomes easier to obtain characteristic values. In addition, as described above, by obtaining a probability distribution image with greater accuracy, the accuracy of the characteristic values ​​themselves can also be improved.

[0087] Furthermore, the ultrasound diagnostic system of this embodiment includes an ultrasound transducer 20 that transmits and receives ultrasound to and from a subject, and a processing unit 152 that uses the above-mentioned learning model 1521 to output a first inference result (probability distribution image A1) from ultrasound image data (intermediate processed image P1) based on the received signal for image generation received by the ultrasound transducer 20. An ultrasound diagnostic system may consist of a combination of multiple devices, rather than just a single ultrasound diagnostic device. This facilitates partial updates and replacements.

[0088] Alternatively, the image diagnostic device of this embodiment has a processing unit 152 that uses a learning model 1521 to output a first inference result (probability distribution image A1) from pre-processed ultrasonic image data (intermediate processed image P1) based on the received signal for image generation received by the ultrasonic probe 20. In other words, the main unit 10 may be treated as a separate unit from the ultrasonic probe 20. As described above, since multiple types of ultrasonic probes 20 are attached and detached depending on the application, if the main unit 10 is sold or rented separately, each user can select and acquire the ultrasonic probe 20 they need separately.

[0089] Furthermore, the computer 40, which serves as a machine learning device in this embodiment, performs machine learning of the machine learning model 451 using training data 452 consisting of pairs: an intermediate processed image P1 based on a received signal for image generation received by the ultrasonic probe 20; ground truth data (probability distribution image A2) obtained by performing the inverse transformation of the coordinate transformation on the ground truth data (probability distribution image A1) for a diagnostic image P2 obtained by performing intermediate processing including coordinate transformation on the intermediate processed image P1; and the intermediate processed image P1 before the coordinate transformation. With this computer 40, it is possible to obtain a training model 1521 that can produce more accurate output based on the intermediate processed image P1 which contains more information.

[0090] Furthermore, machine learning model 451 includes a convolutional neural network. By using a CNN, which is known to consistently produce appropriate results in image recognition processing, as the algorithm for machine learning model 451, it becomes possible to detect the structure of the target object with greater reliability and accuracy.

[0091] Furthermore, the computer 40, which serves as a learning data creation device in this embodiment, comprises a control unit 41 and a storage unit 45. The control unit 41 performs an acquisition process to acquire first ultrasonic image data (intermediate processing image P1) based on a received signal for image generation received by the ultrasonic probe 20, and first correct answer data (probability distribution image A1) for a second ultrasonic image data (diagnostic image P2) obtained by performing intermediate processing including coordinate transformation on the first ultrasonic image data; an inverse transformation process to obtain second correct answer data (probability distribution image A2) by performing the inverse transformation of the above coordinate transformation on the first correct answer data (probability distribution image A1); and a storage control process to store the pair of the first ultrasonic image data (intermediate processing image P1) before intermediate processing and the second correct answer data (probability distribution image A2) in the storage unit 45 as learning data for machine learning. This electronic computer 40 can appropriately create the above-mentioned training data 452.

[0092] Furthermore, the learning data creation method of this embodiment, executed by the control unit 41, includes an acquisition step of acquiring first ultrasonic image data (intermediate processing image P1) based on the received signal for image generation received by the ultrasonic probe 20, and first ground truth data (probability distribution image A1) for a second ultrasonic image data (diagnostic image P2) obtained by performing intermediate processing including coordinate transformation on the first ultrasonic image data; an inverse transformation step of obtaining second ground truth data (probability distribution image A2) by performing an inverse transformation of the coordinate transformation on the first ground truth data; and a storage control step of storing the pair of the first ultrasonic image data before intermediate processing and the second ground truth data in the storage unit 45 as learning data for machine learning. With this learning data creation method, learning data 452 for obtaining a learning model 1521 that can obtain more accurate output results without significantly increasing the effort required can be obtained.

[0093] Furthermore, the learning data creation program 453 of this embodiment provides the following functions to the computer (electronic computer 40): an acquisition function that acquires first ultrasonic image data (intermediate processing image P1) based on the received signal for image generation received by the ultrasonic probe 20, and first correct data (probability distribution image A1) for a second ultrasonic image data (diagnostic image P2) obtained by performing intermediate processing including coordinate transformation on the first ultrasonic image data; an inverse transformation function that performs the inverse transformation of the above coordinate transformation on the first correct data to obtain second correct data (probability distribution image A2); and a storage control function that stores the pair of the first ultrasonic image data before intermediate processing and the second correct data in the storage unit 45 as learning data 452 for machine learning. By having the computer 40 execute this learning data creation program 453, learning data 452 can be easily created from the imaging data of the ultrasound diagnostic device 1 without requiring any special configuration.

[0094] It should be noted that the present invention is not limited to the embodiments described above, and various modifications are possible. For example, in the above embodiment, B-mode images were used as an example of diagnostic images, but the invention is not limited to this. Other diagnostic images may also be used, such as M-mode images, color Doppler images, power Doppler images, elastography images, and other images related to analysis results.

[0095] Furthermore, although the above embodiment was described as performing interpolation between pixels in conjunction with coordinate transformation, interpolation is naturally not necessary when interpolation is not required, such as in images with appropriate focal lengths obtained through linear scanning. Also, the coordinate transformation may include transformations other than those related to DSC.

[0096] Furthermore, although the above embodiment was described as setting contour shapes and regions as correct data, it is not limited to this. It may also be specific coordinate data, or calculated physical quantities such as the length of the contour shape (perimeter, width of specific components, distance between specific positions, etc.), area, volume, etc.

[0097] Furthermore, although the above embodiment described the transformation parameters related to coordinate transformation as being determined based on ultrasonic transmission direction information included in header data such as the intermediate processing image P1, the embodiment is not limited to this. Scanning information of the ultrasonic probe 20 may be acquired separately from outside the image, and the transformation parameters may be determined based on said scanning information. In this case, the scanning information is not attached to each individual intermediate processing image P1, but may be a combination of identification information of some reference images and information for determining the amount of change in the ultrasonic output direction according to the time difference or frame number difference with the reference image.

[0098] Furthermore, in the above embodiment, the image input to the learning model 1521 before coordinate transformation was described as the intermediate processing image P1, that is, an image before, during, or after intermediate processing excluding coordinate transformation. However, going further back, the RF data input to the learning model 1521 may also be the RF data before the detection process is performed. In other words, the processing including coordinate transformation may include the detection process.

[0099] Furthermore, although the above embodiment was described using a CNN as the image recognition algorithm for machine learning model 451 (learning model 1521), it is not limited to this. Any other algorithm capable of learning and identifying the shape and structure of the object to be detected, such as a support vector machine, may also be used.

[0100] Furthermore, the ultrasound diagnostic device 1 is not limited to medical devices that emit ultrasound waves towards the human body. Non-human organisms such as pets may be used as subjects, or devices used for inspecting the internal structure of structures may be used.

[0101] Furthermore, each component of the main unit 10 of the ultrasound diagnostic device 1 may be a combination (system) of multiple devices. For example, the operation reception unit 18 and the display unit 19 may be attached as peripheral devices, or some of the processing of the main unit 10, such as the processing unit 152 and the coordinate transformation unit 153, may be sent to an external computer or the like for separate processing. Alternatively, the first half of the main unit 10, such as signal amplification and envelope detection (processing as a receiving device), and the second half, such as detection (processing as an image diagnostic device), may be completely separated, resulting in a combination of multiple different devices.

[0102] Furthermore, although the above embodiment describes the training of the machine learning model 451 and the creation of its training data 452 as being performed on a separate computer 40 from the ultrasound diagnostic device 1, the training and the creation of the training data 452 may also be performed on the ultrasound diagnostic device 1. In this case, the training data creation program 453 is stored in the memory unit 151. Alternatively, the training of the machine learning model 451 and the creation of the training data 452 may be performed on separate computers. Also, the training data 452 may be stored in a memory unit other than the memory unit 45 of the computer 40, such as an external network storage, an external storage device, or a cloud server (database device).

[0103] Furthermore, in the above description, storage units 45 and 151, consisting of non-volatile memory such as HDDs and flash memory, were used as examples of a computer-readable medium for storing the learning data creation program 453 related to the creation control of learning data of the present invention, and a computer-readable medium for storing the diagnostic program 1511 related to the diagnosis of ultrasound images, respectively, but the invention is not limited to these. Other computer-readable media can be other non-volatile memories such as MRAM, or portable recording media such as CD-ROMs and DVD discs. In addition, a carrier wave can also be used as a medium for providing program data according to the present invention via a communication line. Furthermore, the specific configurations, processing operations, and procedures shown in the above embodiments can be modified as appropriate without departing from the spirit of the present invention. The scope of the present invention includes the scope of the invention described in the claims and its equivalents. [Explanation of symbols]

[0104] 1. Ultrasound diagnostic equipment 10 Main body 11 Control Unit 12 Transmitter drive unit 13 Receiving drive unit 14 Transmit / receive switching unit 15 Image Processing Unit 151 Storage section 1511 Diagnostic Program 152 Processing Unit 1521 Learning Models 153 Coordinate Transformation Unit 154 Synthesis part 17 Communications Department 18 Operation reception section 19 Display section 20 Ultrasonic probe 22 signal cables 40 Electronic computer 41 Control Unit 45 Storage section 451 Machine Learning Models 452 training data 453 Training Data Creation Program 47 Communications Department 48 Display section 49 Operation reception unit A1, A2 Probability Distribution Images C1 First correct data C2 Second set of correct data D1~D3 Shooting Images P1 Intermediate processing image P2 Diagnostic Images

Claims

1. A first ultrasonic data based on a received signal for image generation received by an ultrasonic probe, and a second ground truth data selected from a plurality of second ground truth data obtained by performing an inverse transformation of the coordinate transformation on the first ground truth data for a second ultrasonic image data obtained by synthesizing a plurality of ultrasonic image data obtained by performing a process including detection processing and coordinate transformation on the first ultrasonic data, A machine learning device that performs machine learning on a learning model using training data consisting of pairs.

2. The machine learning apparatus according to claim 1, wherein the learning model includes a convolutional neural network.

3. It comprises a control unit and a memory unit, The control unit, An acquisition process to acquire first ultrasonic data based on a received signal for image generation received by an ultrasonic probe, and first ground truth data for a second ultrasonic image data obtained by synthesizing a plurality of ultrasonic image data obtained by performing a process including detection processing and coordinate transformation on the first ultrasonic data, An inverse transformation process is performed on the first correct data to obtain a plurality of second correct data by performing the inverse transformation of the coordinate transformation, A storage control process that causes the storage unit to store a pair of the first ultrasonic data before processing including the coordinate transformation and a second ground truth data selected from the plurality of second ground truth data as training data for machine learning, A device for creating training data.

4. A method for creating training data for machine learning, wherein the control unit performs the following steps to create training data for machine learning, An acquisition step to acquire first ultrasonic data based on a received signal for image generation received by an ultrasonic probe, and first ground truth data for a second ultrasonic image data obtained by synthesizing a plurality of ultrasonic image data obtained by processing the first ultrasonic data including detection processing and coordinate transformation, An inverse transformation step is performed on the first correct data to obtain a plurality of second correct data by performing the inverse transformation of the coordinate transformation, A storage control step in which the first ultrasonic data before processing including the coordinate transformation and the second selected data selected from the plurality of second ground truth data are stored in the storage unit as training data for machine learning, A method for creating training data that includes this.

5. An acquisition function that acquires first ultrasonic data based on a received signal for image generation received by an ultrasonic probe, and first ground truth data for a second ultrasonic image data obtained by synthesizing a plurality of ultrasonic image data obtained by processing the first ultrasonic data including detection processing and coordinate transformation, An inverse transformation function that performs the inverse transformation of the coordinate transformation on the first correct data to obtain a plurality of second correct data, A memory control function that stores in the memory unit, as training data for machine learning, a pair of the first ultrasonic data before processing including the coordinate transformation and the second ground truth data selected from the plurality of second ground truth data, A program for creating training data that enables computers to achieve this.

Images