Information processing device, ultrasound diagnostic device, information processing method, and information processing program

Abstract

The present invention provides an information processing device, an ultrasound diagnostic device, an information processing method, and an information processing program that can more accurately identify the position of a puncture device in an ultrasound image. [Solution] The information processing device includes an acquisition unit that acquires an ultrasound image of a subject into which a puncture device has been inserted, and an output unit that outputs an estimation result that is output by inputting the acquired ultrasound image into a machine learning model. The machine learning model has been trained on a dataset of training ultrasound images showing at least a part of the puncture device and region information corresponding to the two-dimensional region in which the puncture device exists in the training ultrasound images.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an ultrasonic diagnostic apparatus, an information processing method, and an information processing program.

Background Art

[0002] An ultrasonic diagnostic apparatus that generates ultrasonic images is used in the medical field and the like. The ultrasonic diagnostic apparatus has an ultrasonic probe. A doctor or the like places the ultrasonic probe on the surface of the subject and performs transmission and reception of ultrasonic waves with the ultrasonic probe. Thereby, the shape and the like of the tissue in the subject can be acquired as an ultrasonic image.

[0003] In a hospital or the like, a subject may be punctured. The ultrasonic diagnostic apparatus may be used during this puncture. In orthopedics, anesthesiology, pain clinics, dialysis, etc., a doctor or the like uses the ultrasonic diagnostic apparatus to perform puncture while confirming the positional relationship between each tissue in the subject's body and the puncture instrument. Patent Document 1 discloses a technique for clearly displaying the position of the puncture instrument in an ultrasonic image.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] It is desirable to be able to more accurately specify the position of the puncture instrument in such an ultrasonic image.

[0006] The present invention has been made in view of the above circumstances, and an object thereof is to provide an information processing apparatus, an ultrasonic diagnostic apparatus, an information processing method, and an information processing program capable of more accurately specifying the position of the puncture instrument in an ultrasonic image.

Means for Solving the Problems

[0007] The above objectives of the present invention are achieved by the following means.

[0008] (1) An information processing device comprising: an acquisition unit that acquires an ultrasound image of a subject into which a puncture device has been inserted; and an output unit that outputs an estimation result that is output by inputting the acquired ultrasound image into a machine learning model, wherein the machine learning model has been trained on a dataset of training ultrasound images showing at least a part of the puncture device and region information corresponding to the two-dimensional region in which the puncture device exists in the training ultrasound images.

[0009] (2) The information processing apparatus according to (1) above, further comprising an estimation unit that estimates the puncture device region in which the puncture device is located in the ultrasound image by inputting the acquired ultrasound image into the machine learning model, wherein the output unit outputs the estimation result including information about the estimated puncture device region.

[0010] (3) The information processing device according to (2) above, wherein the output estimation result further includes information regarding the confidence level of the estimated puncture device area.

[0011] (4) The information processing apparatus according to (3) above, further comprising a notification unit for notifying the user of the outputted estimation result, wherein the notification unit is configured to change the manner of notification to the user according to the degree of confidence.

[0012] (5) The information processing device according to (2) above, wherein the output estimation result further includes information regarding the position where the tip of the puncture device is located in the estimated puncture device area.

[0013] (6) The information processing device according to (1) above, wherein the dataset further includes information regarding the puncture direction of the puncture instrument in the learning ultrasound image.

[0014] (7) The information processing device described in (6) above, wherein the tip of the puncture device is located in the two-dimensional region.

[0015] (8) The information processing device according to (6) above, wherein the two-dimensional region has a rectangular shape, and the puncture device is arranged on the diagonal of the rectangular shape.

[0016] (9) The information processing apparatus according to (6) above, wherein the learning ultrasound image is a B-mode image and the two-dimensional region contains parts with different brightness levels.

[0017] (10) The information processing apparatus according to (1) above, further comprising a memory unit in which the machine learning model is stored.

[0018] (11) The information processing apparatus described in (1) above, wherein the two-dimensional region is provided continuously.

[0019] (12) An ultrasound diagnostic device comprising: a probe that irradiates ultrasound onto a subject who has been punctured by a lancing device and receives the ultrasound reflected by the subject; an image generation unit that generates an ultrasound image of the subject based on the ultrasound; an acquisition unit that acquires the generated ultrasound image; and an output unit that outputs an estimation result that is output by inputting the acquired ultrasound image into a machine learning model, wherein the machine learning model has been trained on a dataset of training ultrasound images in which at least a part of the lancing device is captured and region information corresponding to the two-dimensional region in which the lancing device exists in the training ultrasound image.

[0020] (13) An information processing method performed by an information processing device, comprising: acquiring an ultrasound image of a subject who has been punctured by a lancing device; and outputting an estimation result that is output by inputting the acquired ultrasound image into a machine learning model, wherein the machine learning model has learned a dataset of a training ultrasound image showing at least a part of the lancing device and region information corresponding to a two-dimensional region in which the lancing device exists in the training ultrasound image.

[0021] An information processing program for causing a computer to execute the information processing method described in (13) above.

Advantages of the Invention

[0022] In the information processing apparatus, ultrasonic diagnostic apparatus, information processing method, and information processing program according to the present invention, an ultrasonic image is input into a machine learning model. In this machine learning model, a data set of a learning ultrasonic image and region information corresponding to a two-dimensional region where a puncture instrument exists in the learning ultrasonic image is learned. As a result, an accurate estimation result can be output from the ultrasonic image input into the machine learning model. Therefore, the position of the puncture instrument in the ultrasonic image can be specified more accurately.

Brief Description of the Drawings

[0023] The advantages and features provided by one or more embodiments of the present invention will be more fully understood from the following detailed description and the accompanying drawings, which are for illustrative purposes only and are not intended to define the limitations of the present invention. [Figure 1] It is a diagram showing the overall configuration of an ultrasonic diagnostic apparatus. [Figure 2] It is a block diagram showing the schematic configuration of the image processing apparatus shown in FIG. 1. [Figure 3] It is a block diagram showing the functional configuration of the image processing apparatus shown in FIG. 1. [Figure 4] It is a diagram for explaining a learning ultrasonic image or the like input into the machine learning model used in the estimation unit shown in FIG. 3. [Figure 5] It is a diagram showing an example of the estimation result displayed on the display unit shown in FIG. 2. [Figure 6] It is a diagram showing another example of the estimation result displayed on the display unit shown in FIG. 2. [Figure 7] It is a diagram showing another example of the estimation result displayed on the display unit shown in FIG. 2. [Figure 8] It is a diagram showing another example of the estimation result displayed on the display unit shown in FIG. 2. [Figure 9] This flowchart shows the procedure for the estimation process performed in the image processing device shown in Figure 1. [Figure 10] Figure 9 is a flowchart showing the machine learning method of the trained model used in the estimation process. [Figure 11] This is a block diagram showing the functional configuration of an image processing apparatus according to Modification Example 1. [Figure 12] This figure shows an example of a second embodiment displayed on the display unit of the image processing apparatus shown in Figure 11. [Figure 13] This flowchart shows the procedure for the estimation process performed in the image processing device shown in Figure 11. [Figure 14] This is a block diagram showing the functional configuration of an image processing apparatus according to modified example 2. [Figure 15] This figure shows an example of a puncture route displayed on the display unit of the image processing device shown in Figure 14. [Figure 16] This flowchart shows the procedure for the estimation process performed in the image processing device shown in Figure 14. [Figure 17] This is a block diagram showing the functional configuration of an image processing apparatus according to modified example 3. [Figure 18] This flowchart shows the procedure for the estimation process performed in the image processing device shown in Figure 17. [Figure 19] This is a block diagram showing the functional configuration of the image processing apparatus according to Modification 4. [Figure 20] Figure 19 is a diagram illustrating the dataset used by the organizational prediction unit. [Figure 21] Figure 19 shows an example of a target tissue displayed on the display unit of the image processing device. [Figure 22] Figure 19 is a flowchart showing the steps of the estimation process performed in the image processing device. [Figure 23] This is a block diagram showing the functional configuration of the image processing apparatus according to Modification 5. [Figure 24]This figure shows an example of a Doppler image displayed on the display unit of the image processing device shown in Figure 23. [Figure 25] This flowchart shows the procedure for the estimation process performed in the image processing device shown in Figure 23. [Modes for carrying out the invention]

[0024] Embodiments of the present invention will be described below with reference to the attached drawings. However, the scope of the present invention is not limited to the disclosed embodiments. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant descriptions are omitted. Also, the dimensional ratios in the drawings are exaggerated for illustrative purposes and may differ from the actual ratios.

[0025] [Embodiment] <Configuration of Ultrasound Diagnostic Device 1> Figure 1 shows a schematic configuration of an ultrasound diagnostic device 1 according to one embodiment. The ultrasound diagnostic device 1 includes an ultrasound probe 10 and an image processing device 20 connected to the ultrasound probe 10. The ultrasound diagnostic device 1 is used in hospitals and the like when a doctor or clinical engineer punctures a subject. In other words, the user of the ultrasound diagnostic device 1 is a doctor or the like. Here, the ultrasound probe 10 corresponds to one specific example of the probe of the present invention, and the image processing device 20 corresponds to one specific example of the information processing device of the present invention.

[0026] The user places the ultrasound probe 10 in contact with the subject's body surface, causing the ultrasound diagnostic device 1 to generate an ultrasound image of the subject's body. The user moves the lancing device inside the subject's body while checking the ultrasound image displayed on the image processing device 20's display. The lancing device is a puncture needle, etc.

[0027] The ultrasonic transducer 10 irradiates ultrasound from the subject's body surface into the subject's body and receives ultrasound reflected from within the subject's body. The transmission and reception of ultrasound by the ultrasonic transducer 10 is controlled by the image processing device 20. The ultrasonic transducer 10 converts the received ultrasound into an electrical signal and transmits it to the image processing device 20. The ultrasound frequency is approximately 1 MHz to 30 MHz. The ultrasonic transducer 10 includes linear probes, convex probes, sector probes, or three-dimensional probes.

[0028] The image processing device 20 generates an ultrasound image of the subject's body based on the electrical signal received from the ultrasound probe 10. The image processing device 20 is a medical image processing device. The image processing device 20 includes a computer, such as a PC. PC is an abbreviation for Personal Computer. The image processing device 20 may also include a smartphone or tablet device.

[0029] Figure 2 is a block diagram showing the schematic configuration of the image processing device 20. The image processing device 20 includes a CPU 21, ROM 22, RAM 23, storage 24, communication interface 25, display unit 26, operation reception unit 27, and audio input / output unit 28. Each component is connected to each other so as to be able to communicate via a bus 29. CPU is an abbreviation for Central Processing Unit. ROM is an abbreviation for Read Only Memory. RAM is an abbreviation for Random Access Memory. Here, storage 24 corresponds to one specific example of the storage unit of the present invention. At least one of the display unit 26 and the audio input / output unit 28 corresponds to one specific example of the notification unit of the present invention.

[0030] The CPU 21 controls each of the above configurations and performs various calculations according to the program recorded in the ROM 22 or storage 24. The specific functions of the CPU 21 will be described later.

[0031] ROM22 stores various programs and data.

[0032] RAM23 is used as a working area to temporarily store programs and data.

[0033] Storage 24 stores various programs, including the operating system, and various data. Storage 24 has an application installed that uses a trained machine learning model to estimate the position of a puncture device within an ultrasound image from an ultrasound image of the subject's body. Storage 24 may also store multiple ultrasound images. Storage 24 may also store information about the subject, such as the subject's name. Furthermore, Storage 24 may store a trained machine learning model and datasets used for machine learning.

[0034] The communication interface 25 is an interface for communicating with other devices. Various wired or wireless communication interfaces can be used as the communication interface 25. The communication interface 25 is used for transmitting signals to and receiving signals from the ultrasonic probe 10. The communication interface 25 may also be used to connect the image processing device 20 with external devices such as an external server.

[0035] The display unit 26 includes a liquid crystal display or an organic EL display, etc. EL is an abbreviation for electroluminescence. Various information is displayed on the display unit 26. The display unit 26 may also consist of viewer software or a printer, etc.

[0036] The operation reception unit 27 includes a touch sensor, a pointing device, or a keyboard, etc. The pointing device is a mouse, etc. The operation reception unit 27 receives various operations from the user. The display unit 26 and the operation reception unit 27 may constitute a touch panel.

[0037] The audio input / output unit 28 includes a speaker and a microphone, etc.

[0038] <Image processing device 20 functions> Figure 3 is a block diagram showing the functional configuration of the image processing device 20. The image processing device 20 functions as a transmit / receive control unit 211, an image generation unit 212, an acquisition unit 213, an estimation unit 214, and an output unit 215, by having the CPU 21 read a program stored in the storage 24 and execute processing.

[0039] The transmit / receive control unit 211 controls the irradiation and reception of ultrasound by the ultrasonic transducer 10. The transmit / receive control unit 211 controls the irradiation of ultrasound from the ultrasonic transducer 10 to the subject by transmitting voltage pulses to the ultrasonic transducer 10. The transmit / receive control unit 211 adjusts the voltage amplitude, pulse width, and transmission timing of these voltage pulses and transmits voltage pulses for each channel of the ultrasonic transducer 10. The transmit / receive control unit 211 sets an appropriate delay time for each of the multiple channels. This adjusts the depth of the position where the ultrasound is irradiated and the pulse waveform.

[0040] The transmit / receive control unit 211 receives electrical signals corresponding to the ultrasonic waves received by the ultrasonic probe 10. The transmit / receive control unit 211 amplifies the electrical signals received for each channel and converts them into digital signals. The transmit / receive control unit 211 may also perform phase-correcting summation of the signals received for each channel.

[0041] The image generation unit 212 acquires signals from the transmission / reception control unit 211 and generates an ultrasound image of the inside of the subject's body. The ultrasound image of the inside of the subject's body is a tomographic image. The image generation unit 212 generates a cross-sectional ultrasound image that includes the transmission direction and scanning direction of the ultrasound waves from the ultrasound probe 10.

[0042] The acquisition unit 213 acquires the ultrasound image generated by the image generation unit 212. This ultrasound image is an ultrasound image of the subject into which the lancing device was inserted. The acquisition unit 213 may also acquire image information related to the ultrasound image.

[0043] The estimation unit 214 estimates the puncture device region where the puncture device is located in the ultrasound image acquired by the acquisition unit 213. The estimation unit 214 estimates the puncture device region by inputting the ultrasound image into a machine learning model. This machine learning model is trained on a dataset of training ultrasound images showing at least a part of the puncture device and region information corresponding to the two-dimensional region where the puncture device is located in this training ultrasound image. Multiple datasets are trained on the machine learning model.

[0044] Figure 4 shows an example of the relationship between a learning ultrasound image 50 and a two-dimensional region 52 within the learning ultrasound image 50. The learning ultrasound image 50 is a B-mode image. B-mode is an abbreviation for Brightness mode. The learning ultrasound image 50 shows a puncture instrument 51. The two-dimensional region 52 is the area of ​​the learning ultrasound image 50 where the puncture instrument 51 is located, and it has a rectangular shape. The two-dimensional region 52 is a part of the surface within the learning ultrasound image 50. The two-dimensional region 52 is a continuous region. In other words, the two-dimensional region 52 is not a point cloud. The two-dimensional region 52 contains parts with different brightness levels. The region information regarding the two-dimensional region 52 includes information about the position of the two-dimensional region 52 within the learning ultrasound image 50, information about the size of the two-dimensional region 52, information about the shape of the two-dimensional region 52, and information about the brightness level within the two-dimensional region 52.

[0045] The needle-shaped lancing device 51 is positioned on the diagonal of this two-dimensional region 52. In other words, the two-dimensional region 52 is set up so that the lancing device 51 is positioned on the diagonal. Preferably, the tip of the lancing device 51 is located in the two-dimensional region 52, and the tip of the lancing device 51 is positioned at the corner of the rectangular two-dimensional region 52. Preferably, the dataset further includes information regarding the puncture direction 53 of the lancing device 51 in the training ultrasound image 50. The puncture direction 53 represents the direction toward the tip of the lancing device 51 within the two-dimensional region 52. In Figure 4, the puncture direction 53 is toward the lower left of the paper. The dataset may also include information regarding the position where the tip of the lancing device 51 is located within the two-dimensional region 52.

[0046] The output unit 215 outputs the estimation results from the estimation unit 214. These estimation results are obtained by inputting the ultrasound images acquired by the acquisition unit 213 into a machine learning model. The estimation results include information about the puncture device area estimated by the estimation unit 214. The output unit 215 outputs the estimation results by displaying them on the display unit 26.

[0047] Figure 5 shows an example of the estimation results displayed on the display unit 26. The display unit 26 displays the ultrasound image 60 and the lancing device area 2141 within the ultrasound image 60. The ultrasound image 60 is an image generated by the image generation unit 212. The ultrasound image 60 is a B-mode image. The lancing device area 2141 is the area of ​​the ultrasound image 60 where the presence of a lancing device is estimated. The lancing device area 2141 has a needle shape. The lancing device area 2141 is estimated by the estimation unit 214.

[0048] Figures 6 to 8 each show other examples of estimation results displayed on the display unit 26. As shown in Figure 6, the estimation results may include information regarding the confidence level of the puncture device area 2141 estimated by the estimation unit 214. The confidence level is a value that represents the likelihood of the estimated puncture device area 2141; the higher the confidence level, the more reliable the estimated puncture device area 2141 is. The confidence level is expressed as a percentage from 0% to 100%. When the estimation unit 214 inputs the ultrasound image 60 to the machine learning model, the confidence level is output along with the puncture device area 2141. By outputting information regarding the confidence level, the user can more easily determine whether or not a puncture device is visible in the ultrasound image 60.

[0049] As shown in Figure 7, the estimation results may include information about the position 2142 where the tip of the lancing device is located in the lancing device region 2141. On the display unit 26, the position 2142 is indicated by a circular enclosure. When the estimation unit 214 inputs the ultrasound image 60 to the machine learning model, it outputs information about the lancing device region 2141 along with information about the position 2142 where the tip of the lancing device is located. By outputting information about the position 2142, the user can more accurately understand the status of the lancing device inside the subject's body and perform the puncture smoothly. The estimation results may also include information about the puncture direction in the lancing device region 2141.

[0050] As shown in Figure 8, the estimation results may include information about the estimation region 2143, which includes the puncture device region 2141. The estimation region 2143 is a region corresponding to the two-dimensional region 52 in the training ultrasound image 50 and has a rectangular shape. The puncture device region 2141 is located diagonally across this estimation region 2143. When the estimation unit 214 inputs the ultrasound image 60 to the machine learning model, it outputs information about the puncture device region 2141 along with the estimation region 2143.

[0051] <Overview of processing performed by the image processing device 20> Figure 9 is a flowchart showing the procedure for estimating the puncture device area performed by the image processing device 20. The processing of the image processing device 20 shown in the flowchart of Figure 9 is stored as a program in the storage 24 of the image processing device 20 and is executed by the CPU 21 controlling each part.

[0052] (Step S101) The image processing device 20 first controls the transmission and reception of ultrasound waves by the ultrasound transducer 10. The ultrasound transducer 10 irradiates ultrasound waves from the body surface of the subject into the subject's body and receives ultrasound waves reflected from within the subject's body. The ultrasound transducer 10 converts these ultrasound signals into electrical signals and transmits them to the image processing device 20. A lancing device is inserted into the subject's body.

[0053] (Step S102) The image processing device 20 generates an ultrasonic image 60 based on the electrical signal transmitted from the ultrasonic transducer 10 in step S101.

[0054] (Step S103) The image processing device 20 acquires the ultrasound image 60 generated in step S102.

[0055] (Step S104) The image processing device 20 estimates the puncture device region 2141 by inputting the ultrasound image 60 acquired in step S103 into a machine learning model. At this time, the image processing device 20 calculates the confidence level of the puncture device region 2141.

[0056] (Step S105) The image processing device 20 outputs the estimation result estimated in step S104. This estimation result includes information about the lancing device area 2141 and information about the confidence level of the lancing device area 2141. The image processing device 20 displays the ultrasound image 60, the lancing device area 2141, and the confidence level on the display unit 26. The image processing device 20 performs this estimation process over time. This allows the user to accurately determine the position of the lancing device as it moves within the subject's body.

[0057] <About the learning process> Next, we will explain the machine learning method of the machine learning model used to estimate the puncture device region 2141.

[0058] Figure 10 is a flowchart showing the machine learning method used in a machine learning model.

[0059] In the process shown in Figure 10, machine learning is performed using a large number of pre-prepared datasets as training sample data. This dataset includes training ultrasound images 50 as input, and region information related to a two-dimensional region 52, the puncture direction 53, and the position of the tip of the puncture instrument 51 as output. The learning system uses a standalone high-performance computer with CPU and GPU processors, or a cloud computer. Below, we will describe a learning method using a neural network constructed by combining perceptrons such as deep learning in the learning system. Various methods can be applied to the learning method, including random forests, decision trees, support vector machines (SVMs), logistic regression, k-nearest neighbors, and topic models.

[0060] (Step S111) The learning model loads the training data dataset. If it's the first time, it loads the first set of datasets; if it's the i-th time, it loads the i-th set of datasets.

[0061] (Step S112) The learning model inputs the input data from the loaded dataset into the neural network.

[0062] (Step S113) The learning model compares the estimation results of the neural network with the correct data.

[0063] (Step S114) The learner adjusts its parameters based on the comparison results. The learner adjusts its parameters to minimize the difference in the comparison results by performing a process based on backpropagation.

[0064] (Step S115) If the learning machine has finished processing all the data from set 1 to i (YES), it proceeds to step S116; otherwise, it returns to step S111, loads the next dataset, and repeats the process from step S111 onwards.

[0065] (Step S116) The learning device determines whether to continue learning or not. If it decides to continue (YES), it returns to step S111 and executes the processes for the first to ith groups again in steps S111 to S115. If it decides not to continue (NO), it proceeds to step S117.

[0066] (Step S117) The learning unit terminates (end) by storing the machine learning model constructed during the previous processing. The storage location includes the internal memory of the image processing unit 20. In the estimation process described above, the puncture device area is estimated using the machine learning model thus generated.

[0067] <Effects of the image processing device 20 and the ultrasound diagnostic device 1> In the image processing device 20 and ultrasound diagnostic device 1 according to this embodiment, the training ultrasound image 50 is input to a machine learning model. This machine learning model is trained on a dataset of the training ultrasound image 50 and region information corresponding to a two-dimensional region 52. As a result, accurate estimation results can be output from the ultrasound image 60 input to the machine learning model. Therefore, it becomes possible to more accurately identify the position of the puncture device in the ultrasound image 60. The effects of this will be explained below.

[0068] It is conceivable to use image processing techniques to detect linear brightness distributions similar to those of lancing devices from ultrasound images and identify the location of the lancing device. However, multiple linear brightness distributions similar to those of the lancing device may exist within an ultrasound image. Therefore, it is difficult to identify the location of the lancing device in an ultrasound image with sufficiently high accuracy using image processing techniques. Furthermore, even with machine learning models, if the ground truth labels in the dataset do not contain sufficient information, it may be difficult to accurately estimate the location of the lancing device.

[0069] The lancing device has a predetermined diameter. However, if the correct label is based only on one-dimensional information corresponding to the region where the lancing device exists in the training ultrasound image, for example, information in the length direction of the lancing device, then this information in the diameter direction of the lancing device will not be reflected in the correct label. Therefore, even when using this machine learning model, there is a risk of false positives.

[0070] In contrast, the machine learning model used by the image processing device 20 learns region information corresponding to a rectangular two-dimensional region 52 as the correct label. This allows the model to learn information in two directions: the length direction and the thickness direction of the lancing device. When ultrasound is irradiated in the thickness direction of the lancing device, the part of the lancing device opposite the side irradiated by the ultrasound appears with lower brightness. The two-dimensional region 52 can capture such changes in the brightness of the lancing device. Therefore, by using a machine learning model that has learned region information corresponding to the two-dimensional region 52, it becomes possible to estimate the location of the lancing device more accurately. Consequently, by using the image processing device 20, the user can more accurately identify the location of the lancing device in the ultrasound image 60.

[0071] Furthermore, the dataset that this machine learning model learns from includes information about the puncture direction 53. This allows the image processing device 20 to estimate the puncture direction and the position of the tip of the puncture instrument in the ultrasound image 60. This enables the user to perform punctures on the subject more smoothly.

[0072] Furthermore, this image processing device 20 stores this machine learning model in its storage 24. This allows for faster processing of the acquired ultrasound images 60 to estimate the position of the puncture device. Therefore, users can identify the position of the puncture device as it moves within the subject's body with a sense of real-time accuracy.

[0073] The following describes modified examples of the above embodiments. Components similar to those in the previously described embodiments are denoted by the same reference numerals and their descriptions are omitted.

[0074] <Example 1> Figure 11 is a block diagram showing the functional configuration of the image processing device 20 according to Modification 1. In this image processing device 20, the CPU 21 reads a program stored in the storage 24 and executes processing so that, in addition to the transmission / reception control unit 211, image generation unit 212, acquisition unit 213, estimation unit 214, and output unit 215, a confidence level determination unit 216 also functions. Except for this point, the image processing device 20 according to Modification 1 has the same configuration as the image processing device 20 described in the above embodiment and produces the same effects.

[0075] The confidence determination unit 216 determines whether the confidence level of the lancing device area 2141 estimated by the estimation unit 214 exceeds a predetermined value. The predetermined value is, for example, 80%. When the confidence determination unit 216 determines that the confidence level exceeds the predetermined value, the output unit 215 outputs the estimation result in a first mode. When the confidence determination unit 216 determines that the confidence level is less than or equal to the predetermined value, the output unit 215 outputs the estimation result in a second mode. The second mode is different from the first mode. The output unit 215 displays the lancing device area 2141 in a different way on the display unit 26 between the first mode and the second mode. The output unit 215 may also display the confidence level in a different way on the display unit 26 between the first mode and the second mode.

[0076] Figure 12 shows an example of the estimation result displayed on the display unit 26 in the second embodiment. When the confidence level determination unit 216 determines that the confidence level is less than or equal to a predetermined value, the output unit 215 displays the lancing device area 2141 on the display unit 26 with a dashed line. When the confidence level determination unit 216 determines that the confidence level exceeds a predetermined value, the output unit 215 displays the lancing device area 2141 on the display unit 26 with a solid line, as shown in Figure 6.

[0077] Figure 13 is a flowchart showing the procedure of processing performed in the image processing device 20. The processing of the image processing device 20 shown in the flowchart of Figure 13 is stored as a program in the storage 24 of the image processing device 20 and is executed by the CPU 21 controlling each part.

[0078] (Steps S201~S204) The image processing device 20 executes the processes of steps S201 to S204 in the same manner as the processes of steps S101 to S104 described above.

[0079] (Step S205) The image processing device 20 determines whether the confidence level of the puncture device area 2141 estimated in step S204 exceeds a predetermined value.

[0080] (Step S206) If the image processing device 20 determines in step S205 that the confidence level exceeds a predetermined value (step S205: YES), it outputs the estimation result in the first mode and terminates the process. At this time, the image processing device 20 displays the lancing device area 2151 with a solid line on the display unit 26.

[0081] (Step S207) When the image processing device 20 determines in step S205 that the confidence level is less than or equal to a predetermined value (step S205: NO), it outputs the estimation result in the second mode and terminates the process. At this time, the image processing device 20 displays the puncture device area 2151 with a dashed line on the display unit 26.

[0082] In the image processing device 20 according to Modification 1, the training ultrasound image 50 is input to the machine learning model, similar to the embodiment described above. This machine learning model is trained on a dataset of training ultrasound images 50 and region information corresponding to a two-dimensional region 52. As a result, accurate estimation results can be output from the ultrasound image 60 input to the machine learning model. Therefore, it becomes possible to more accurately identify the position of the puncture device in the ultrasound image 60.

[0083] Furthermore, the image processing device 20 outputs estimation results in different ways depending on the confidence level of the estimated puncture device area 2151. This allows the user to more intuitively grasp the confidence level of the estimated puncture device area 2151. The user will also be more likely to notice if the puncture device is not visible in the acquired ultrasound image 60.

[0084] <Modification 2> Figure 14 is a block diagram showing the functional configuration of the image processing device 20 according to Modification 2. This image processing device 20 functions as a puncture route creation unit 217 in addition to the transmission / reception control unit 211, image generation unit 212, acquisition unit 213, estimation unit 214, output unit 215, and confidence determination unit 216, by having the CPU 21 read a program stored in the storage 24 and execute processing. Except for this point, the image processing device 20 according to Modification 2 has the same configuration as the image processing device 20 described in Modification 1 and produces the same effects.

[0085] The puncture path creation unit 217 creates a puncture path based on the puncture device area 2141 estimated by the estimation unit 214. The puncture path is the predicted path that the puncture device will take, based on the current position and direction of the puncture device. When the confidence level of the puncture device area 2141 estimated by the estimation unit 214 exceeds a predetermined value, the puncture path creation unit 217 creates a puncture path. When the confidence level of the puncture device area 2141 estimated by the estimation unit 214 is less than or equal to a predetermined value, the puncture path creation unit 217 does not create a puncture path. The puncture path creation unit 217 calculates a formula based on the position and inclination of the puncture device area 2141, and creates a puncture path from this formula.

[0086] Figure 15 shows an example of the lancing device area 2141 and lancing route 2144 displayed on the display unit 26. The lancing route 2144 is shown as a dashed line following the lancing device area 2141. The display unit 26 may also display the lancing route 2144 along with a scale indicating length, etc.

[0087] Figure 16 is a flowchart showing the procedure for processing performed in the image processing device 20. The processing of the image processing device 20 shown in the flowchart of Figure 15 is stored as a program in the storage 24 of the image processing device 20 and is executed by the CPU 21 controlling each part.

[0088] (Steps S301~S304) The image processing device 20 executes the processes of steps S301 to S304 in the same manner as the processes of steps S101 to S104 described above.

[0089] (Step S305) The image processing device 20 determines whether the confidence level of the puncture device area 2141 estimated in step S304 exceeds a predetermined value.

[0090] (Steps S306, S307) If the image processing device 20 determines in step S305 that the confidence level exceeds a predetermined value (step S305: YES), it creates a puncture path 2144 based on the puncture device area 2141 estimated in step S304. After this, the image processing device 20 outputs the estimation result and the puncture path 2144 and terminates the process. The image processing device 20 causes the display unit 26 to display the puncture path 2144 following the puncture device area 2141. At this time, the image processing device 20 may output the estimation result in the first embodiment.

[0091] (Step S308) If the image processing device 20 determines in step S305 that the confidence level is less than or equal to a predetermined value (step S305: NO), it outputs the estimation result and terminates the process. At this time, the image processing device 20 may output the estimation result in a second embodiment.

[0092] In the image processing device 20 according to Modification 2, the training ultrasound image 50 is input to the machine learning model, similar to the embodiment described above. This machine learning model is trained on a dataset of training ultrasound images 50 and region information corresponding to a two-dimensional region 52. As a result, accurate estimation results can be output from the ultrasound image 60 input to the machine learning model. Therefore, it becomes possible to more accurately identify the position of the puncture device in the ultrasound image 60.

[0093] Furthermore, the image processing device 20 creates a puncture path 2144 based on the estimated puncture device area 2141. This allows the user to predict the future path of the puncture device area 2151 and accurately understand the positional relationship between the tissues inside the subject's body and the puncture device.

[0094] The image processing device 20 may create a puncture route regardless of the confidence level value. That is, the image processing device 20 according to Modification 2 does not need to have a confidence level determination unit 216.

[0095] The image processing device 20 may decide whether or not to create a puncture path based on the position of the puncture device area 2141 in the ultrasound image 60. The image processing device 20 decides not to create a puncture path when the puncture device area 2141 is located in the center of the ultrasound image 60.

[0096] <Variation 3> Figure 17 is a block diagram showing the functional configuration of the image processing device 20 according to Modification 3. In this image processing device 20, the CPU 21 reads a program stored in the storage 24 and executes processing so that, in addition to the transmission / reception control unit 211, image generation unit 212, acquisition unit 213, estimation unit 214, output unit 215, and confidence determination unit 216, the image storage unit 218 also functions. Except for this point, the image processing device 20 according to Modification 3 has the same configuration as the image processing device 20 described in Modification 1 and produces the same effects.

[0097] The image storage unit 218 stores the ultrasound image 60 acquired by the acquisition unit 213. The image storage unit 218 stores the acquired ultrasound image 60 by storing it in storage 24 or the like. The image storage unit 218 may also store the ultrasound image 60 together with the estimation results estimated by the estimation unit 214. When the confidence level of the puncture device area 2141 estimated by the estimation unit 214 exceeds a predetermined value, the image storage unit 218 stores the ultrasound image 60. When the confidence level of the puncture device area 2141 estimated by the estimation unit 214 is less than or equal to a predetermined value, the image storage unit 218 does not store the ultrasound image 60. The image storage unit 218 may also store the ultrasound image 60 on an external server or the like.

[0098] Figure 18 is a flowchart showing the procedure for processing performed in the image processing device 20. The processing of the image processing device 20 shown in the flowchart of Figure 18 is stored as a program in the storage 24 of the image processing device 20 and is executed by the CPU 21 controlling each part.

[0099] (Steps S401~S404) The image processing device 20 executes the processes of steps S401 to S404 in the same manner as the processes of steps S101 to S104 described above.

[0100] (Step S405) The image processing device 20 determines whether the confidence level of the puncture device area 2141 estimated in step S404 exceeds a predetermined value. If the image processing device 20 determines in step S405 that the confidence level is less than or equal to the predetermined value (step S405: NO), it proceeds to the process in step S407.

[0101] (Step S406) If the image processing device 20 determines in step S405 that the confidence level exceeds a predetermined value (step S405: YES), it saves the ultrasound image 60 acquired in step S401 and proceeds to the process in step S407. The image processing device 20 may also save the ultrasound image 60 together with the estimation result estimated in step S404.

[0102] (Step S407) The image processing device 20 outputs the estimation result and terminates the process. The image processing device 20 may execute the process in step S407 before the process in step S406, or it may execute the processes in steps S406 and S407 simultaneously.

[0103] In the image processing device 20 according to Modification 3, the training ultrasound image 50 is input to the machine learning model, similar to the embodiment described above. This machine learning model is trained on a dataset of training ultrasound images 50 and region information corresponding to a two-dimensional region 52. As a result, accurate estimation results can be output from the ultrasound image 60 input to the machine learning model. Therefore, it becomes possible to more accurately identify the position of the puncture device in the ultrasound image 60.

[0104] Furthermore, this image processing device 20 automatically saves the ultrasound images 60. This allows the user to easily save the ultrasound images 60 even if both of their hands are occupied during puncture. Therefore, the saved ultrasound images 60 can be used for analyzing the condition of the patient, etc.

[0105] The image processing device 20 may store the ultrasound image 60 regardless of the confidence level value. That is, the image processing device 20 according to Modification 3 does not need to have a confidence level determination unit 216. The image processing device 20 according to Modification 3 may further have a puncture route creation unit 217.

[0106] The image processing device 20 may decide whether or not to save the ultrasound image 60 based on the position of the puncture device area 2141 within the ultrasound image 60. The image processing device 20 decides to save the ultrasound image 60 when the puncture device area 2141 is located in the center of the ultrasound image 60. Alternatively, the image processing device 20 may decide whether or not to save the ultrasound image 60 based on its positional relationship with the tissues inside the subject's body. The image processing device 20 decides to save the ultrasound image 60 when the puncture device area 2141 is located near a nerve inside the subject's body.

[0107] <Modification 4> Figure 19 is a block diagram showing the functional configuration of the image processing device 20 according to Modification 4. This image processing device 20 functions as a tissue prediction unit 219 in addition to the transmission / reception control unit 211, image generation unit 212, acquisition unit 213, estimation unit 214, output unit 215, confidence determination unit 216, and puncture route creation unit 217, by having the CPU 21 read a program stored in the storage 24 and execute processing. Except for this point, the image processing device 20 according to Modification 4 has the same configuration as the image processing device 20 according to Modification 2 and produces the same effects.

[0108] The tissue prediction unit 219 predicts the target tissue present in the ultrasound image 60 based on the ultrasound image 60 acquired by the acquisition unit 213. The target tissue is a specific blood vessel and nerve present in the subject's body. The tissue prediction unit 219 uses a machine learning model to predict the target tissue present in the ultrasound image 60. This machine learning model is trained on a dataset of training ultrasound images showing at least a portion of the target tissue and region information corresponding to the two-dimensional region where the target tissue exists in the training ultrasound image. Examples of target tissues include blood vessels and nerves.

[0109] Figure 20 shows an example of a training ultrasound image 80 showing blood vessels and nerves. This training ultrasound image 80 has a two-dimensional region 821 where blood vessels are located and a two-dimensional region 822 where nerves are located. In the machine learning model, a dataset of the training ultrasound image 80 and region information corresponding to the two-dimensional regions 821 and 822 is used for training.

[0110] When the tissue prediction unit 219 predicts the presence of the target tissue within the ultrasound image 60, the output unit 215 outputs the prediction result along with information about the target tissue.

[0111] The output unit 215 may output warning information to alert the user when the distance between the tissue and the puncture device area 2141 is less than or equal to a predetermined value. The output unit 215 outputs the warning information by displaying a warning message or icon on the display unit 26. On the display unit 26, the message or icon may be displayed superimposed on the ultrasound image 60, or it may be displayed outside the ultrasound image 60. The output unit 215 may also output the warning information by outputting a warning sound or warning message on the audio input / output unit 28.

[0112] Figure 21 shows an example of the estimated results and tissue displayed on the display unit 26. The display unit 26 shows the puncture device area 2141 and the blood vessel 2145 within the ultrasound image 60. The display unit 26 also displays a message to alert the user to the blood vessel 2145.

[0113] Figure 22 is a flowchart showing the procedure of processing performed in the image processing device 20. The processing of the image processing device 20 shown in the flowchart of Figure 22 is stored as a program in the storage 24 of the image processing device 20 and is executed by the CPU 21 controlling each part.

[0114] (Steps S501~S503) The image processing device 20 executes the processes of steps S501 to S503 in the same manner as the processes of steps S101 to S103 described above.

[0115] (Step S504) The image processing device 20 predicts the target tissue present in the ultrasound image 60 based on the ultrasound image 60 acquired in step S503.

[0116] (Step S505) The image processing device 20 estimates the puncture device region 2141 by inputting the ultrasound image 60 acquired in step S503 into a machine learning model. At this time, the image processing device 20 calculates the confidence level of the puncture device region 2141. The image processing device 20 may perform the process in step S505 before the process in step S504, or it may perform the processes in steps S504 and S505 simultaneously.

[0117] (Step S506) The image processing device 20 determines whether the confidence level of the puncture device area 2141 estimated in step S505 exceeds a predetermined value.

[0118] (Step S511) If the image processing device 20 determines in step S506 that the confidence level is less than or equal to a predetermined value (step S506: NO), it outputs the estimation result and terminates the process.

[0119] (Step S507) If the image processing device 20 determines in step S506 that the confidence level exceeds a predetermined value (step S506: YES), it creates a puncture path 2144 based on the puncture device area 2141 estimated in step S505.

[0120] (Step S508) The image processing device 20 determines whether the distance between the puncture path 2144 created in step S507 and the puncture device area 2141 estimated in step S505 is less than a predetermined value.

[0121] (Steps S510, S511) In step S508, when the image processing device 20 determines that the distance between the puncture path 2144 and the puncture instrument area 2141 is greater than or equal to a predetermined value (step S508: NO), it outputs the target tissue and the estimation result and terminates the process.

[0122] (Steps S509, S510, S511) If the image processing device 20 determines in step S508 that the distance between the puncture path 2144 and the puncture device area 2141 is less than a predetermined value (step S508: YES), it outputs warning information, target tissue, and estimation results and terminates processing.

[0123] In the image processing device 20 according to Modification 4, the training ultrasound image 50 is input to the machine learning model, similar to the embodiment described above. This machine learning model is trained on a dataset of training ultrasound images 50 and region information corresponding to a two-dimensional region 52. As a result, accurate estimation results can be output from the ultrasound image 60 input to the machine learning model. Therefore, it becomes possible to more accurately identify the position of the puncture device in the ultrasound image 60.

[0124] Furthermore, the image processing device 20 predicts the location of the target tissue as seen in the ultrasound image 60, making it easier for the user to understand the positional relationship between the puncture device and the target tissue within the subject's body. In addition, the image processing device 20 outputs warning information when the distance between the target tissue and the puncture device is close, thus alerting the user.

[0125] The image processing device 20 may create a puncture route 2144 regardless of the confidence value. That is, the image processing device 20 according to Modification 4 does not need to have a confidence determination unit 216. Alternatively, the image processing device 20 does not need to create a puncture route. That is, the image processing device 20 according to Modification 4 does not need to have a puncture route creation unit 217. The image processing device 20 according to Modification 4 may further have an image storage unit 218.

[0126] <Modification 5> Figure 23 is a block diagram showing the functional configuration of the image processing device 20 according to Modification 5. This image processing device 20 functions as a Doppler region setting unit 311 and a Doppler image generation unit 312, in addition to the transmission / reception control unit 211, image generation unit 212, acquisition unit 213, estimation unit 214, output unit 215, and confidence determination unit 216, by having the CPU 21 read a program stored in the storage 24 and execute processing. Except for this point, the image processing device 20 according to Modification 5 has the same configuration as the image processing device 20 described in Modification 1 and produces the same effects.

[0127] The Doppler region setting unit 311 sets the Doppler region within the ultrasound image 60 based on the puncture device region 2141 estimated by the estimation unit 214. The Doppler region is the region from which the Doppler image is generated.

[0128] The Doppler image generation unit 312 generates a Doppler image based on the Doppler signal generated in the Doppler region. The Doppler region is set by the Doppler region setting unit 311. The Doppler signal is generated when the ultrasonic probe 10 irradiates the Doppler region with ultrasound.

[0129] Figure 24 shows an example of a Doppler image 70 displayed on the display unit 26. The Doppler image 70 is displayed alongside the ultrasound image 60. The ultrasound image 60 shows the puncture device area 2141. The Doppler image 70 shows the tissue present in the ultrasound image 60. The Doppler image 70 may also show medication or other substances present in the ultrasound image 60.

[0130] Figure 25 is a flowchart showing the procedure for processing performed in the image processing device 20. The processing of the image processing device 20 shown in the flowchart of Figure 25 is stored as a program in the storage 24 of the image processing device 20 and is executed by the CPU 21 controlling each part.

[0131] (Steps S601~S604) The image processing device 20 executes the processes of steps S601 to S604 in the same manner as the processes of steps S101 to S104 described above.

[0132] (Step S605) The image processing device 20 determines whether the confidence level of the puncture device area 2141 estimated in step S604 exceeds a predetermined value.

[0133] (Step S609) If the image processing device 20 determines in step S605 that the confidence level is less than or equal to a predetermined value (step S605: NO), it outputs the estimation result and terminates the process.

[0134] (Step S606) If the image processing device 20 determines in step S605 that the confidence level exceeds a predetermined value (step S605: YES), it sets a Doppler region within the ultrasound image 60.

[0135] (Step S607) The image processing device 20 generates a Doppler image by irradiating the Doppler region set in step S606 with ultrasound.

[0136] (Steps S608, S609) The image processing device 20 outputs the Doppler image generated in step S608 along with the estimation result, and then terminates the processing.

[0137] In the image processing device 20 according to Modification 5, the training ultrasound image 50 is input to the machine learning model, similar to the embodiment described above. This machine learning model is trained on a dataset of training ultrasound images 50 and region information corresponding to a two-dimensional region 52. As a result, accurate estimation results can be output from the ultrasound image 60 input to the machine learning model. Therefore, it becomes possible to more accurately identify the position of the puncture device in the ultrasound image 60.

[0138] Furthermore, the Doppler region is automatically set in this image processing device 20. This makes it easy to generate Doppler images even when the user's hands are occupied during puncture. Therefore, it becomes easier for the user to understand the positional relationship between the puncture device and the tissue within the subject's body.

[0139] The image processing device 20 may generate Doppler images regardless of the confidence value. That is, the image processing device 20 according to Modification 5 does not need to have a confidence determination unit 216. The image processing device 20 according to Modification 5 may further have at least one of the puncture route creation unit 217, the image storage unit 218, and the tissue prediction unit 219.

[0140] The present invention is not limited to the embodiments and variations described above, and can be modified in various ways within the scope of the claims.

[0141] The ultrasonic transducer 10 and the image processing device 20 may each include components other than those described above, or may not include some of the components described above. The image processing device 20 may be, for example, an information processing device, and only needs to have an information processing function for outputting estimation results from the ultrasonic image 60.

[0142] Furthermore, the ultrasonic probe 10 and the image processing device 20 may each be composed of multiple devices or a single device.

[0143] Furthermore, the functions of each component may be realized by other components. Some or all of the functions of the image processing device 20 may be realized by the ultrasonic probe 10 or an external server, etc. The machine learning model may be stored on an external server, etc.

[0144] Furthermore, the two-dimensional region 52 may have a shape other than a rectangle. The two-dimensional region 52 may have a shape such as an ellipse.

[0145] Furthermore, the processing units in the flowcharts of the above embodiments and their respective modifications are divided according to the main processing content in order to facilitate understanding of each process. The present invention is not limited by how the processing steps are classified. Each process can be further divided into more processing steps. Also, one processing step may perform even more processes.

[0146] The means and methods for performing various processing tasks in the system according to the above embodiment can be implemented by either a dedicated hardware circuit or a programmed computer. The program may be provided on a computer-readable recording medium such as a flexible disk or CD-ROM, or it may be provided online via a network such as the Internet. In this case, the program recorded on the computer-readable recording medium is usually transferred to and stored in a storage unit such as a hard disk. Furthermore, the program may be provided as a standalone application software, or it may be incorporated into the software of the device as a function of the system.

[0147] While embodiments of the present invention have been described and illustrated in detail, the disclosed embodiments are for illustrative purposes only and are not limiting. The scope of the present invention should be interpreted in accordance with the language of the appended claims. [Explanation of symbols]

[0148] 1. Ultrasound diagnostic equipment 10 Ultrasonic probe, 20 Image processing equipment, 21 CPUs, 211 Transmit / receive control unit, 212 Image generation unit, 213 Acquisition Department; 214 Estimation Department, 215 Output section, 216 Confidence determination unit, 217 Puncture route creation section, 218 Image storage section, 219 Organizational Forecasting Department, 311 Doppler area setting unit, 312 Doppler image generation unit, 22 ROMs, 23 RAM, 24 storage, 25 communication interfaces, 26 Display section, 27 Operation reception unit, 28. Audio input / output section.

Claims

1. The lancing device has an acquisition unit that acquires an ultrasound image of the subject into which it has been punctured, The acquired ultrasound image is input to a machine learning model, and the output unit outputs the estimation result. Equipped with, The machine learning model is an information processing device that has learned a dataset of training ultrasound images showing at least a portion of the puncture device and region information corresponding to the two-dimensional region where the puncture device is located in the training ultrasound images.

2. The system further includes an estimation unit that estimates the area in which the puncture device is located in the ultrasound image by inputting the acquired ultrasound image into the machine learning model. The information processing apparatus according to claim 1, wherein the output unit outputs the estimation result, which includes information regarding the estimated puncture device area.

3. The information processing device according to claim 2, wherein the output estimation result further includes information regarding the confidence level of the estimated puncture device area.

4. The system further includes a notification unit that notifies the user of the outputted estimation result. The information processing apparatus according to claim 3, wherein the notification unit is configured to change the notification method to the user according to the degree of confidence.

5. The information processing device according to claim 2, wherein the output estimation result further includes information regarding the position where the tip of the puncture device is located in the estimated puncture device region.

6. The information processing device according to claim 1, wherein the dataset further includes information regarding the puncture direction of the puncture instrument in the learning ultrasound image.

7. The information processing device according to claim 6, wherein the tip of the puncture device is located in the two-dimensional region.

8. The aforementioned two-dimensional region has a quadrilateral shape. The information processing device according to claim 6, wherein the puncture device is arranged on the diagonal of the rectangular shape.

9. The aforementioned learning ultrasound image is a B-mode image. The information processing apparatus according to claim 6, wherein the two-dimensional region contains parts with different brightness levels.

10. The information processing apparatus according to claim 1, further comprising a storage unit in which the machine learning model is stored.

11. The information processing apparatus according to claim 1, wherein the two-dimensional region is provided continuously.

12. A puncture device irradiates ultrasound onto the subject that has been punctured, and a transducer receives the ultrasound reflected by the subject. An image generation unit that generates an ultrasound image of the subject based on the ultrasound, An acquisition unit that acquires the generated ultrasound image, The acquired ultrasound image is input to a machine learning model, and the output unit outputs the estimation result. Equipped with, The machine learning model is an ultrasound diagnostic device that has been trained on a dataset consisting of training ultrasound images in which at least a portion of the puncture device is captured, and region information corresponding to the two-dimensional region in which the puncture device exists in the training ultrasound images.

13. An information processing method performed by an information processing device, The lancing device obtains an ultrasound image of the subject into which it was inserted, The acquired ultrasound images are input into a machine learning model to output the estimation results. Includes, The machine learning model is an information processing method in which a dataset of training ultrasound images showing at least a portion of the puncture device and region information corresponding to the two-dimensional region where the puncture device exists in the training ultrasound images has been trained.

14. An information processing program for causing a computer to execute the information processing method described in claim 13.

Images