Inference apparatus, ultrasound diagnostic apparatus, training apparatus, inference method, display method, and program

Abstract

To provide an inference apparatus capable of performing accurate inference on a nerve root using a trained model, and to provide an ultrasonic diagnostic apparatus, a training apparatus, an inference method, a display method, and a program related to the inference apparatus.SOLUTION: An inference apparatus according to the present disclosure includes: an image acquisition unit configured to acquire ultrasonic image data related to an ultrasonic image generated based on a reflected wave of an ultrasonic wave transmitted to a subject; and an inference unit configured to input the ultrasonic image data as inference target data to a trained model and perform inference on a nerve root in the ultrasonic image based on output data output from the trained model.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an inference device, an ultrasound diagnostic device, a training device, an inference method, a display method, and a program. [Background technology]

[0002] With the recent advances in deep learning technology, machine learning models have come to be used for various purposes. For example, in the medical field, the use of machine learning models for image diagnosis of ultrasound image data has been proposed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2020-519369 [Patent Document 2] JP 2021-164573 A Summary of the Invention [Problem to be solved by the invention]

[0004] Lumbar nerve root blocks are performed on patients who complain of lower back pain. Typically, a nerve root block is an injection of an anesthetic solution into the lumbar nerve root. Typically, nerve root blocks are performed by puncture under fluoroscopy, but the nerve root itself is not imaged. For this reason, doctors determine that the puncture needle has reached the vicinity of the nerve root by touching the puncture needle to the nerve root and observing the patient's pain response.

[0005] However, such a method places a heavy burden on patients, and improvements are therefore required.

[0006] In recent years, a technique has been developed for performing a nerve root block on a cervical nerve root, in which the location of the nerve root is confirmed in real time using an ultrasound diagnostic device while the puncture is performed.

[0007] When attempting to visualize lumbar nerve roots using an ultrasound diagnostic device, the fifth lumbar nerve root (also called L5 nerve root), which is particularly frequently punctured, is located deep from the body surface and is surrounded by bones, making it more difficult to visualize compared to cervical nerve roots. In addition, when observing lumbar nerve roots from the back, the observation is done through muscles such as the multifidus and erector spinae. In patients with aging and fatty muscles, ultrasound is scattered by the fat, making it even more difficult to visualize lumbar nerve roots.

[0008] In view of these circumstances, the present disclosure aims to provide an inference device capable of making accurate inferences regarding nerve roots using a trained model, as well as an ultrasound diagnostic device, a training device, an inference method, a display method, and a program related to the inference device. [Means for solving the problem]

[0009] An inference device according to one aspect of the present disclosure includes an image acquisition unit that acquires ultrasound image data relating to an ultrasound image generated based on reflected waves of ultrasound transmitted to a subject, and an inference unit that inputs the ultrasound image data as inference target data into a trained model and performs inference regarding nerve roots in the ultrasound image based on output data output from the trained model. Effect of the Invention

[0010] According to the present disclosure, a trained model can be used to make accurate inferences regarding nerve roots. [Brief description of the drawings]

[0011] [Figure 1] Schematic diagram showing the training and inference processes in an ultrasound diagnostic system. [Diagram 2] FIG. 1 is a schematic diagram illustrating a first variation of the training and inference processes according to the present disclosure. [Diagram 3] FIG. 1 is a schematic diagram illustrating a second variation of the training and inference processes according to the present disclosure. [Figure 4]FIG. 1 is a diagram showing an example of the appearance of an ultrasound diagnostic apparatus according to an embodiment of the present disclosure. [Diagram 5] FIG. 1 is a block diagram showing a configuration example of an ultrasound diagnostic apparatus according to an embodiment of the present disclosure. [Figure 6] FIG. 1 is a block diagram showing a configuration example of a training device according to an embodiment of the present disclosure. [Figure 7] FIG. 1 is a block diagram showing an example of a hardware configuration of an inference device and a training device according to an embodiment of the present disclosure. [Figure 8] A flowchart for explaining an example of the operation of the training device in the training phase. [Figure 9] FIG. 1 shows an example of an ultrasound image of the lower back. [Figure 10] FIG. 13 shows another example of an ultrasound image of the lumbar region. [Figure 11] FIG. 13 shows another example of an ultrasound image of the lumbar region. [Figure 12] FIG. 13 shows another example of an ultrasound image of the lumbar region. [Figure 13] FIG. 1 shows an example of an ultrasound image of the lower back, including blood flow information. [Figure 14] 1 is a flowchart for explaining an example of an operation of an ultrasound diagnostic apparatus in an inference phase. [Figure 15] FIG. 13 is a diagram showing an example of a display image. [Figure 16] FIG. 11 is a diagram showing another example of a display image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0013] [Summary of the Disclosure] Disclosed below is an ultrasound diagnostic system that includes a training device that trains a machine learning model (trained model) that makes inferences about nerve roots in ultrasound images, an inference device that makes inferences about nerve roots using the trained model, and an ultrasound diagnostic device that generates ultrasound images that are the subject of inference and displays the ultrasound images based on the results of the inference.

[0014] The trained model trained in the training device is trained to output output data when ultrasound image data is input, using ultrasound image data relating to an ultrasound image that includes bones around a nerve root as training data.

[0015] The ultrasound imaging device includes an inference device that holds a trained model that has been trained in a training device. The ultrasound imaging device transmits ultrasound to a subject, generates an ultrasound image based on the reflected ultrasound, and passes ultrasound image data related to the ultrasound image to the inference device. The inference device inputs the ultrasound image data to the trained model and performs inference regarding the nerve root based on the output data. Specifically, the inference device infers the position of the nerve root in the ultrasound image.

[0016] The ultrasound imaging device displays an ultrasound image based on the inference result output by the inference device. Specifically, the ultrasound imaging device displays the position of the nerve root based on the inference result together with the ultrasound image.

[0017] This allows a user (such as a doctor) of the ultrasound diagnostic system to perform a nerve root block while constantly checking the position of the nerve root based on the ultrasound image. In particular, the training device trains the training subject model using ultrasound images that include at least one of the bones of the fifth lumbar vertebra, the fourth lumbar vertebra, or the sacrum that are present around the lumbar nerve root as training data, thereby making it possible to perform a lumbar nerve root block with a relatively small burden on the subject.

[0018] [System Configuration] The system configuration of an ultrasound diagnostic system 1 according to an embodiment of the present disclosure will be described with reference to Fig. 1. Fig. 1 is a schematic diagram showing training processing and inference processing in the ultrasound diagnostic system 1.

[0019] 1, the ultrasound diagnostic system 1 includes a training device 50 and an ultrasound diagnostic device 100. The ultrasound diagnostic device 100 includes an inference device 200.

[0020] The training device 50 stores a machine learning model to be trained, and trains the machine learning model to be trained using training data stored in a training data database (DB) 20. The machine learning model to be trained may be realized as any suitable type of machine learning model, such as a neural network. For example, if the machine learning model to be trained is realized by a neural network, the training device 50 may perform supervised learning using the training data acquired from the training data DB 20, and update parameters of the machine learning model according to any known training algorithm, such as backpropagation.

[0021] When the training of the machine learning model is completed, the trained machine learning model (hereinafter, the trained model 10) is stored in a memory unit or the like of the inference device 200 provided in the ultrasound diagnostic device 100.

[0022] The ultrasound diagnostic device 100 generates ultrasound image data by transmitting and receiving ultrasound signals to and from a subject 30 via an ultrasound probe.

[0023] The inference device 200 inputs ultrasound image data into the trained model 10 and, based on the output data, makes inferences about nerve roots in the ultrasound image, such as inferring the location of the nerve roots in the image.

[0024] Based on the inference result of the inference device 200, the ultrasound diagnostic device 100 displays an ultrasound image and displays information related to the nerve root contained in the ultrasound image.

[0025] Although the system configuration according to an embodiment of the present disclosure has been described with reference to FIG. 1, the system configuration according to the present disclosure is not limited thereto. FIG. 2 is a schematic diagram showing a first modified example of the training process and the inference process according to the present disclosure. In the example shown in FIG. 2, the ultrasound diagnostic device 100 and the inference device 200 are independent of each other. In this case, the ultrasound diagnostic device 100 outputs the generated ultrasound image data to the inference device 200, and the inference device 200 returns output data, which is the inference result performed using the trained model 10, to the ultrasound diagnostic device 100. In this way, the ultrasound diagnostic device 100 can display an ultrasound image and display information related to the nerve root included in the ultrasound image based on the inference result of the inference device 200.

[0026] FIG. 3 is a schematic diagram showing a second modified example of the training process and the inference process according to the present disclosure. In the example shown in FIG. 3, a training device 50 may train a machine learning model to be trained stored in a model database (DB) 40, and a trained model 10 that has completed training may be stored in the model DB 40. An inference device 200 accesses the model DB 40 and acquires output data using the trained model 10. According to such a second modified example, even if the inference device 200 does not have a storage resource for storing the trained model 10, it is possible to obtain an effect equivalent to that of the embodiment shown in FIG. 1 by using the trained model 10 stored in the model DB 40.

[0027] In addition, although not shown in the figures, the present disclosure may include a form in which an ultrasound diagnostic device that generates ultrasound images and an image display device that displays ultrasound images based on the inference results of the inference device 200 are provided separately.

[0028] [Configuration of Ultrasound Diagnostic Apparatus 100] Fig. 4 is a diagram showing an example of the external appearance of the ultrasound diagnostic device 100 according to an embodiment of the present disclosure. Fig. 5 is a block diagram showing an example of the configuration of the ultrasound diagnostic device 100 according to an embodiment of the present disclosure.

[0029] The ultrasound diagnostic device 100 visualizes, as an ultrasound image, at least one of the shape and the dynamics inside the subject 30. The ultrasound diagnostic device 100 is used, for example, to capture ultrasound images (i.e., cross-sectional images) of a detection target site, particularly the lumbar region, and to perform nerve root block on nerve roots present in the detection target site, particularly the lumbar region.

[0030] 4, the ultrasonic diagnostic device 100 includes an ultrasonic diagnostic device main body 1010 and an ultrasonic probe 1020. The ultrasonic diagnostic device main body 1010 and the ultrasonic probe 1020 are connected via a cable 1030.

[0031] The ultrasonic probe (ultrasonic probe) 1020 transmits an ultrasonic beam to a subject 30 (e.g., a patient) and functions as an acoustic sensor that receives reflected waves (ultrasonic echoes) of the transmitted ultrasonic beam reflected inside the body of the subject 30 and converts them into electrical signals.

[0032] The user operates the ultrasound diagnostic device 100 by contacting the transmitting / receiving surface of the ultrasound probe 1020 for transmitting ultrasound beams with the body surface of the subject 30 at a detection target site. Any type of ultrasound probe, such as a convex probe, a linear probe, a sector probe, or a three-dimensional probe, can be used as the ultrasound probe 1020.

[0033] The ultrasonic probe 1020 includes, for example, a plurality of transducers (e.g., piezoelectric elements) arranged in an array, and a channel switching device (e.g., a multiplexer) for controlling the on / off switching of the driving state of the plurality of transducers individually or in blocks.

[0034] Each transducer of the ultrasonic probe 1020 converts a voltage pulse generated by the transmitting unit 1012 of the ultrasonic diagnostic device main body 1010 into an ultrasonic beam and transmits it to the subject 30, receives ultrasonic echoes reflected within the subject 30, converts them into electrical signals, and outputs them to the receiving unit 1013 of the ultrasonic diagnostic device main body 1010.

[0035] In this embodiment, the ultrasound diagnostic device main body 1010 includes, as shown in FIG. 5, an operation input unit 1011, a transmission unit 1012, a reception unit 1013, an ultrasound image generation unit 1014, a display image generation unit 1015, an output unit 1016, a control unit 1017, and an inference device 200.

[0036] The transmitting unit 1012, the receiving unit 1013, the ultrasound image generating unit 1014, and the display image generating unit 1015 are composed of dedicated or general-purpose hardware (electronic circuits) corresponding to each process, such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array), and work in cooperation with the control unit 1017 to realize each function.

[0037] The operation input unit 1011 receives, for example, a command for instructing the operation of the ultrasound diagnostic apparatus 100, or input of information on the subject 30. The operation input unit 1011 may have, for example, an operation panel having a plurality of input switches, a keyboard, and a mouse. The operation input unit 1011 may be configured as a touch panel provided integrally with the output unit 1016.

[0038] The transmission unit 1012 is a transmitter that transmits a voltage pulse as a drive signal to the ultrasonic probe 1020 in accordance with an instruction from the control unit 1017. The transmission unit 1012 may be configured to include, for example, a high-frequency pulse oscillator and a pulse setting unit. The transmission unit 1012 may adjust the voltage pulse generated by the high-frequency pulse oscillator to the voltage amplitude, pulse width, and transmission timing set by the pulse setting unit, and transmit the adjusted voltage pulse for each channel of the ultrasonic probe 1020.

[0039] The transmitting unit 1012 has a pulse setting unit for each of the multiple channels of the ultrasonic probe 1020, and can set the voltage amplitude, pulse width, and transmission timing of the voltage pulse for each of the multiple channels. For example, the transmitting unit 1012 may change the target depth or generate different pulse waveforms by setting appropriate delay times for the multiple channels.

[0040] The receiving unit 1013 is a receiver that receives and processes electrical signals related to ultrasonic echoes generated by the ultrasonic probe 1020 in accordance with instructions from the control unit 1017. The receiving unit 1013 may be configured to include a preamplifier, an AD conversion unit, and a receiving beamformer.

[0041] The receiving unit 1013 amplifies the reception signal related to the weak ultrasonic echo for each channel in the preamplifier, and converts the reception signal into a digital signal in the AD conversion unit. The receiving unit 1013 then combines the reception signals of the multiple channels into one by phasing and adding the reception signals of each channel in the reception beamformer, and can generate acoustic line data.

[0042] The ultrasound image generating unit 1014 acquires the reception signal (acoustic line data) from the receiving unit 1013 and generates an ultrasound image (that is, a tomographic image) of the inside of the subject 30.

[0043] For example, when the ultrasonic probe 1020 transmits a pulsed ultrasonic beam in the depth direction, the ultrasonic image generating unit 1014 accumulates the signal intensity of the ultrasonic echo detected thereafter in a line memory in a time-series continuous manner. Then, as the ultrasonic beam from the ultrasonic probe 1020 scans inside the subject 30, the ultrasonic image generating unit 1014 accumulates the signal intensity of the ultrasonic echo at each scanning position in the line memory in sequence, and generates two-dimensional data in units of frames. Then, the ultrasonic image generating unit 1014 can generate an ultrasonic image representing a two-dimensional structure in a cross section including the ultrasonic transmission direction and the ultrasonic scanning direction by converting the signal intensity of the two-dimensional data into a brightness value.

[0044] In addition, the ultrasound image generating unit 1014 may have, for example, an envelope detection circuit that performs envelope detection on the received signal acquired from the receiving unit 1013, a logarithmic compression circuit that performs logarithmic compression on the signal strength of the received signal detected by the envelope detection circuit, and a dynamic filter that is a bandpass filter whose frequency characteristics change according to the depth and removes noise components contained in the received signal.

[0045] The ultrasound image generating unit 1014 may generate ultrasound image data in other image modes such as B mode, color Doppler mode, M (Motion) mode, and pulse Doppler mode as necessary. B mode is a mode for generating a tomographic image in which the received signal strength of the ultrasound of the subject is represented by brightness. Color Doppler mode is a mode for converting the Doppler shift frequency caused by the blood flow of the subject into velocity and displaying the velocity distribution as a tomographic image.

[0046] Ultrasound image data relating to the ultrasound image generated by the ultrasound image generation unit 1014 is output to the inference device 200.

[0047] As shown in FIG. 5, the inference device 200 includes an image acquisition unit 201 , a blood flow information acquisition unit 202 , an inference unit 203 , and a storage unit 204 .

[0048] The image acquisition unit 201 acquires the ultrasound image data output from the ultrasound image generation unit 1014 .

[0049] When the ultrasound image generating unit 1014 generates an image in color Doppler mode, the blood flow information acquiring unit 202 acquires blood flow information regarding the blood flow flowing through blood vessels included in the ultrasound image.

[0050] The inference unit 203 inputs the ultrasound image data as inference target data to the trained model 10 stored in the storage unit 204, and performs inference regarding the nerve root in the ultrasound image based on the output data regarding the nerve root output from the trained model 10. The content of the inference performed by the inference unit 203 is, for example, inference of the position of the nerve root in the ultrasound image.

[0051] Output data that is the inference result by the inference unit 203 is output to the display image generation unit 1015 .

[0052] The display image generating unit 1015 generates a display image including an ultrasound image and an indication related to a nerve root, based on the ultrasound image data generated by the ultrasound image generating unit 1014 and the output data acquired from the inference device 200. Then, the display image generating unit 1015 sends the data of the generated display image to the output unit 1016. The display image generating unit 1015 may sequentially update the display image every time a new ultrasound image is acquired from the ultrasound image generating unit 1014, and cause the output unit 1016 to display the display image in a moving image format.

[0053] The display image generating unit 1015 may perform predetermined image processing, such as coordinate conversion processing and data interpolation processing, on the ultrasound image output from the ultrasound image generating unit 1014 to generate a display image.

[0054] The output unit 1016 obtains display image data from the display image generation unit 1015 and outputs the display image in accordance with an instruction from the control unit 1017. For example, the output unit 1016 may be configured with a liquid crystal display, an organic EL display, a CRT display, or the like.

[0055] The control unit 1017 controls the operation input unit 1011, the transmission unit 1012, the reception unit 1013, the ultrasound image generation unit 1014, the display image generation unit 1015, the output unit 1016, and the inference device 200 according to their respective functions, thereby performing overall control of the ultrasound diagnostic device 100. Note that in the present disclosure, the control of the inference device 200 does not necessarily have to be performed by the control unit 1017, and for example, the inference device 200 may have a control unit independent of the control unit 1017.

[0056] The control unit 1017 may have a CPU (Central Processing Unit) 1171 as an arithmetic / control device, an SSD (Solid State Drive) 1172 in which basic programs and basic setting data are stored, and a RAM (Random Access Memory) 1173 as a main storage device. The CPU 1171 centrally controls the operation of each functional block of the ultrasound diagnostic device main body 1010 and the inference device 200 by reading out a program corresponding to the processing content from the SSD 172, storing it in the RAM 1173, and executing the stored program.

[0057] [Configuration of training device 50] Next, a configuration of the training device 50 according to an embodiment of the present disclosure will be described. Fig. 6 is a block diagram showing an example of the configuration of the training device 50 according to an embodiment of the present disclosure.

[0058] As shown in FIG. 6, the training device 50 includes a data acquisition unit 51 and a training unit 52.

[0059] The data acquisition unit 51 acquires training data (learning data) used for training a machine learning model to be trained. Specifically, the data acquisition unit 51 acquires a pair of ultrasound image data related to an ultrasound image and a ground truth label indicating the position of a nerve root in the ultrasound image.

[0060] The ultrasound image data as training data acquired by the training device 50 does not necessarily have to be the ultrasound image data generated by the ultrasound diagnostic device 100 described above. The training device 50 may acquire ultrasound image data including a large number of ultrasound images generated in the past by other ultrasound diagnostic devices, and correct labels previously assigned to each of these ultrasound images. The training device 50 may acquire this training data from, for example, a company or organization that provides pairs of ultrasound image data and correct labels, or may use existing data existing on a public network such as the Internet as the training data.

[0061] The correct answer labels may be assigned in advance to multiple ultrasound images as described above, or may be assigned by the user of the ultrasound diagnostic system 1 to each of the ultrasound images generated using the ultrasound diagnostic device 100.

[0062] The data acquisition unit 51 may perform preprocessing such as contrast change and noise removal on the acquired training ultrasound image data. The data acquisition unit 51 may also perform data expansion on the acquired training data to increase the training data. For example, the data acquisition unit 51 may perform data expansion by performing image conversion processing such as enlargement or reduction, position change, and deformation on the training ultrasound image data.

[0063] The training unit 52 uses pairs of ultrasound image data and correct labels as training data to generate a trained model 10 that has been trained to output inference results regarding nerve roots in response to input of ultrasound image data.

[0064] More specifically, the training unit 52 compares output data output by the machine learning model to be trained in response to the input of training ultrasound image data with the correct label, and updates the parameters of the machine learning model to be trained based on the comparison result (error). For example, when the machine learning model is realized by a convolutional neural network, the training unit 52 may continue to adjust the parameters of the machine learning model according to the error between the output result and the correct data according to the backpropagation method until a predetermined completion condition is satisfied.

[0065] In this embodiment, when the training unit 52 determines that the training of the machine learning model is completed, it provides the trained model 10 to the inference device 200 of the ultrasound diagnostic device 100 (see FIG. 1).

[0066] [Hardware Configuration of Inference Device 200 and Training Device 50] FIG. 7 is a block diagram showing an example of a hardware configuration of an inference device 200 and a training device 50 according to an embodiment of the present disclosure.

[0067] The inference device 200 and the training device 50 may each be realized by various types of computing device 300, such as a server device, a personal computer (PC), a workstation, a smartphone, a tablet terminal, etc. That is, the computing device 300 that may constitute the inference device 200 or the training device 50 has a drive device 101, a storage device 102, a memory device 103, a processor 104, a user interface (UI) device 105, and a communication device 106 that are interconnected via a bus B.

[0068] Programs or instructions for implementing various functions and processes of the inference device 200 and the training device 50 may be stored in a removable storage medium such as a CD-ROM (Compact Disk-Read Only Memory) or a flash memory. When the storage medium is set in the drive device 101, the programs or instructions are installed from the storage medium via the drive device 101 into the storage device 102 or the memory device 103. However, the programs or instructions do not necessarily have to be installed from the storage medium, and may be downloaded from any external device via a network or the like.

[0069] The storage device 102 is realized by a storage device such as a hard disk drive (HDD) or an SSD, and stores installed programs or instructions as well as files, data, and the like used to execute the programs or instructions.

[0070] The memory device 103 is realized by a random access memory, a static memory, or the like, and when a program or an instruction is activated, reads and stores the program or instruction, data, or the like from the storage device 102. The storage device 102, the memory device 103, and the removable storage medium may be collectively referred to as a non-transitory storage medium.

[0071] The processor 104 may be realized by one or more CPUs, GPUs (Graphics Processing Units), processing circuits, etc., which may be composed of one or more processor cores, and performs various functions and processing of the inference device 200 and the training device 50 in accordance with programs, instructions, and data such as parameters required to execute the programs or instructions stored in the memory device 103.

[0072] User interface (UI) device 105 may be composed of input devices such as a keyboard, mouse, camera, microphone, etc., output devices such as a display, speaker, headset, printer, etc., and input / output devices such as a touch panel, and realizes an interface between the user and inference device 200 and training device 50. For example, the user operates a GUI (Graphical User Interface) displayed on a display or touch panel by operating a keyboard, mouse, etc. to operate inference device 200 and training device 50.

[0073] The communication device 106 is realized by various communication circuits that execute wired or wireless communication processing with an external device, the Internet, a LAN (Local Area Network), a cellular network, or other communication network.

[0074] The hardware configuration illustrated in FIG. 7 is merely an example, and the inference device 200 and training device 50 according to the present disclosure may be realized by any other suitable hardware configuration.

[0075] [Example] An example of the operation of the ultrasound diagnostic system 1 will be described in detail below.

[0076] <Training Phase> First, a description will be given of an example of the operation of the training device 50 in the training phase. Fig. 8 is a flowchart for describing an example of the operation of the training device 50 in the training phase.

[0077] In step S1, the data acquisition unit 51 of the training device 50 acquires training data.

[0078] The training data is ultrasound image data obtained from various subjects. When it is assumed that the ultrasound diagnostic system 1 is used particularly for lumbar nerve root block, the training data is preferably ultrasound image data of the lumbar regions of various subjects. However, the training data may include ultrasound image data of regions other than the lumbar region.

[0079] Fig. 9 is a diagram showing an example of an ultrasound image of the lumbar region. Fig. 9 shows an example of an ultrasound image obtained by applying an ultrasound probe to the lumbar region of a subject from the back side of the subject along the approximate midline and irradiating the subject with ultrasound. The left side of Fig. 9 corresponds to the subject's foot side, and the right side corresponds to the subject's head side.

[0080] The white areas in Fig. 9 are the subject's muscles, fat, etc. The whiter areas near the interfaces between the white and black areas correspond to bones. Nerve roots are not depicted in Fig. 9.

[0081] In the example shown in Fig. 9, the fourth lumbar vertebra (L4), the fifth lumbar vertebra (L5), and the sacrum are depicted. The L5 nerve root used for the nerve root block is present between the fifth lumbar vertebra and the sacrum (area A shown in Fig. 9). Thus, the ultrasound image of the training data used in the present disclosure includes the nerve root to be treated and the bones that should be around it.

[0082] As described above, the training data is composed of a pair of ultrasound image data and a correct label relating to an ultrasound image as exemplified in Fig. 9. The correct label is data indicating which region of the ultrasound image is the nerve root (particularly the L5 nerve root). The correct label may be assigned in advance to each ultrasound image by the user or a person other than the user.

[0083] The ultrasound image shown in Fig. 9 is an example, and data of other ultrasound images may be used as the training data of the present disclosure. Figs. 10 to 12 are diagrams showing other examples of ultrasound image data as training data.

[0084] 10 and 11 are ultrasound images obtained by placing an ultrasound probe on the midline of a subject from the back side of the subject, similar to the example shown in FIG. 9. Even in ultrasound images of similar parts, the way body tissues are captured in the ultrasound images varies depending on the subject, as shown in FIG. 9, FIG. 10A, FIG. 10B, FIG. 11A, and FIG. 11B. By using a large amount of such diverse ultrasound image data as training data, the performance of the trained model 10 can be improved.

[0085] Figures 11A and 11B show examples of ultrasound images of the lumbar region that appear differently depending on the amount of fat of the subject. Figure 11A is an example of an ultrasound image of a subject with a relatively high amount of fat, and Figure 11B is an example of an ultrasound image of a subject with a relatively low amount of fat. Since fat tends to appear white in ultrasound images, as shown in Figures 11A and 11B, the image of the subject with a relatively high amount of fat appears whiter overall than the image of the subject with a relatively low amount of fat.

[0086] FIG. 12 is a diagram showing examples of ultrasound images with different cross-sectional orientations. FIG. 12A and FIG. 12B show examples of ultrasound images in a transverse section of a subject's lumbar region. These images are obtained when an ultrasound probe is applied to the subject's back along the left-right direction of the subject. In the example shown in FIG. 12, the orientation is different from the examples shown in FIG. 9 to FIG. 11, so that the corresponding tissues shown in the image are different. Pairs of ultrasound images with different orientations and training labels may be used as training data.

[0087] 13 is a diagram showing an example of an ultrasound image of the lumbar region including blood flow information. The blood flow information is information about the blood flow flowing through the blood vessels (lumbar arteries) running parallel to the nerve roots, and the direction of the blood flow is indicated by colors such as red and blue. In this way, when blood flow information is included in an ultrasound image of the lumbar region, it is possible to know that the nerve roots are located near the positions where the blood flow information is present, since the nerve roots run parallel to the blood vessels as described above.

[0088] In the present disclosure, data of various ultrasound images may be used as training data in addition to the ultrasound images described in Figures 9 to 13. In addition, although the ultrasound images illustrated in Figures 9 to 13 are still images, moving images may be used as training data.

[0089] For example, as training data, ultrasound image data of an ultrasound image in which nerve roots are marked by injecting a drug solution for marking nerve roots into the subject when generating the ultrasound image data may be used. With such images, the user does not need to manually assign correct answer labels, and the effort required to generate training data can be significantly reduced.

[0090] Returning to the description of Fig. 8, in step S2, the training unit 52 of the training device 50 uses the training data acquired in step S1 to train the machine learning model to be trained. As described above, the present disclosure does not particularly limit the training method. In the present disclosure, since training is performed using images, an error backpropagation method suitable for images may be used as appropriate.

[0091] In step S3, the training unit 52 determines whether a predetermined completion condition has been met. The predetermined completion condition may be set in advance by, for example, a user of the ultrasound diagnostic system 1. An example of the completion condition is that learning using a predetermined number of pairs of ultrasound images and correct labels has been completed.

[0092] If the training unit 52 determines in step S3 that the training has been completed (step S3: YES), the process proceeds to step S4. If the training unit 52 determines in step S3 that the training has not been completed (step S3: NO), the process returns to step S1, and the training continues. Note that, although the flowchart shown in FIG. 8 shows an example in which the process returns to step S1 when it is determined in step S3 that the training has not been completed, in the present disclosure, if sufficient training data has been acquired in step S1, the process may return from step S3 to step S2.

[0093] In step S4, the training device 50 outputs the trained model 10 (see FIG. 1, etc.) to the inference device 200 of the ultrasound diagnostic device 100. This completes the training phase, and makes it possible to execute the inference phase.

[0094] By carrying out the training in this manner, the machine learning model to be trained becomes a trained model 10 that can output output data related to the nerve root in response to the input of ultrasound image data. Note that the output data may be, for example, data itself indicating the position of the nerve root in the ultrasound image, or various data related to the nerve root, such as data related to the positions of bones present around the nerve root, or data related to the positions of blood vessels running parallel to the nerve root.

[0095] The output data output by the trained model 10 may include the above-mentioned multiple types of data, or may include only one type of data.

[0096] <Inference Phase> Next, a description will be given of an example of the operation of the ultrasound diagnostic apparatus 100 in the inference phase. Fig. 14 is a flowchart for explaining an example of the operation of the ultrasound diagnostic apparatus 100 in the inference phase.

[0097] The inference phase is executed, for example, when a user uses the ultrasound diagnostic apparatus 100 to administer a nerve root block treatment to a subject.

[0098] In step S11, the ultrasound image generating unit 1014 (see FIG. 5) of the ultrasound diagnostic device 100 generates ultrasound image data based on the reflected echo received from the ultrasound probe 1020 pressed against the subject. For example, when a user performs a lumbar nerve root block, the user generates ultrasound image data of the lumbar region.

[0099] In step S12, the ultrasound image generation unit 1014 inputs the generated image data to the inference device 200. If the generated ultrasound image is a moving image, the ultrasound image generation unit 1014 inputs the image data generated for each frame to the inference device 200. As a result, the image acquisition unit 201 of the inference device 200 acquires the ultrasound image data of the inference target.

[0100] In step S13, the inference unit 203 of the inference device 200 inputs the ultrasound image data to be inferred to the trained model 10 stored in the memory unit 204, and acquires the output data output from the trained model 10.

[0101] As described above, the output data may include multiple types of data, such as data indicating the position of the nerve root itself, as well as data indicating the positions of bones that should be present around the nerve root (in the lumbar region, the fourth lumbar vertebra, the fifth lumbar vertebra, and the sacrum, etc.), or data indicating the position of blood vessels running parallel to the nerve root (in the lumbar region, the lumbar artery).

[0102] In step S14, the inference unit 203 performs inference regarding the nerve root in the ultrasound image based on the output data.

[0103] Specifically, when the output data includes the position of the nerve root to be treated, the inference unit 203 identifies the position of the nerve root based on the output data.

[0104] Furthermore, when the output data includes the position of bones that should exist around the nerve root to be treated, the inference unit 203 infers the position of the nerve root based on the output data. For example, in the ultrasound image of the lumbar region illustrated in Fig. 9, the position (area A) where the nerve root should be located is inferred based on the positions of the fourth lumbar vertebra, the fifth lumbar vertebra, and the sacrum, which are indicated by the output data.

[0105] In addition, when the output data includes the position of a blood vessel that should run parallel to the nerve root to be treated, the inference unit 203 infers the position of the nerve root based on the output data. For example, in the example shown in Fig. 13, the position where the nerve root should be located is inferred around the area where blood flow is present.

[0106] In order to improve the accuracy of inferring the nerve root, it is more desirable for the inference unit 203 to infer the position of the nerve root in the ultrasound image by combining multiple types of inference results using multiple types of data included in the output data.

[0107] In step S15, the inference unit 203 outputs the inference result (the position of the nerve root in the ultrasound image) data to the display image generation unit 1015.

[0108] In step S16, the display image generation unit 1015 generates a display image to be displayed on the output unit 1016 based on the ultrasound image data acquired from the ultrasound image generation unit 1014 and the inference result data indicating the position of the nerve root in the ultrasound image of the ultrasound image data.

[0109] Fig. 15 is a diagram showing an example of a display image. The display image shown in Fig. 15 is an image generated by superimposing a display element (mark) indicating the position of the nerve root on the ultrasound image shown in Fig. 9. In the example shown in Fig. 15, a white circle display element indicates the position of the nerve root.

[0110] 15, in the region other than the nerve root in the ultrasound image, labels indicating what each region is ("fourth lumbar vertebra", "fifth lumbar vertebra", "sacrum") are displayed. If not required by the user, such labels do not need to be displayed in the displayed image.

[0111] 15 is an example, and in the present disclosure, various display images may be displayed on the output unit 1016. For example, in order to highlight a display element indicating the position of a nerve root, the display element may be configured with a fluorescent frame.

[0112] Also, a display screen may be generated that displays an ultrasound image that does not display elements or labels indicating the positions of the nerve roots and an ultrasound image that displays display elements or labels indicating the positions of the nerve roots side by side, as shown in, for example, Fig. 16. Although the two images are arranged vertically in Fig. 16, a display screen that displays them horizontally may be generated.

[0113] In step S17, the output unit 1016 outputs (displays) the display image, which allows the user to accurately grasp the position of the nerve root.

[0114] When the ultrasound image data is moving image data, the ultrasound diagnostic device 100 repeats step S11 shown in Fig. 14 for each frame, thereby allowing the user to accurately grasp the position of the nerve root in real time.

[0115] As described above, the inference device 200 infers the position of the nerve root based on the output data output by the trained model in response to the input of ultrasound image data, and the ultrasound diagnostic device 100 generates a display image based on the inference result of the inference device 200, thereby making it possible to generate a display image including a display element that accurately indicates the position of the nerve root. This allows a user who performs a nerve root block treatment on a subject to accurately grasp the position of the nerve root without having to check the subject's reaction. This makes it possible to perform a nerve root block that places less strain on the subject.

[0116] Although the examples of the present disclosure have been described in detail above, the present disclosure is not limited to the specific embodiments described above, and various modifications and variations are possible within the scope of the gist of the present disclosure described in the claims. [Explanation of symbols]

[0117] 1. Ultrasound diagnostic system 10 Pre-trained models 50 training equipment 51 Data Acquisition Section 52 Training Department 100 Ultrasound diagnostic equipment 1010 Ultrasound diagnostic equipment body 1011 Operation input section 1012 Transmitter 1013 Receiving unit 1014 Ultrasound image generating unit 1015 Display image generating unit 1016 Output section 1017 Control section 1020 Ultrasound Probe 1030 Cable 200 Reasoning device 201 Image Acquisition Unit 202 Blood flow information acquisition unit 203 Reasoning section 203 Reasoning section 204 Storage section 300 Computing equipment 101 Drive device 102 Storage device 103 Memory device 104 processors 105 Equipment 106 Communication Equipment

Claims

1. an image acquisition unit that acquires ultrasound image data relating to an ultrasound image generated based on a reflected wave of an ultrasound transmitted to a subject; an inference unit that inputs the ultrasound image data as inference target data into a trained model and performs inference regarding nerve roots in the ultrasound image based on output data output from the trained model; An inference device comprising:

2. The inference unit infers a position of the nerve root based on positions of bones surrounding the nerve root included in the output data output from the trained model. The inference device of claim 1 .

3. a blood flow information acquisition unit for acquiring information on blood flow through a blood vessel included in the ultrasound image, The inference unit infers a position of the nerve root based on the blood flow information. The inference device according to claim 2.

4. The trained model is trained to output the output data when the ultrasound image data is input, using the ultrasound image including a bone present around the nerve root as training data. The inference device of claim 1 .

5. The trained model is trained using the ultrasound image including at least one of the fifth lumbar vertebra, the fourth lumbar vertebra, or the sacrum as training data.

5. The inference device according to claim 4.

6. The trained model is further trained using blood flow information regarding blood flow through a blood vessel running parallel to the nerve root as the training data.

5. The inference device according to claim 4.

7. The trained model is trained using the ultrasound images in which the nerve roots are marked as training data. The inference device of claim 1 .

8. An inference device according to any one of claims 1 to 7; an image generating unit that generates the ultrasonic image; a display unit that displays the ultrasound image and information related to the nerve root based on the inference by the inference unit; An ultrasound diagnostic device comprising:

9. the display unit displays a nerve root position display element indicating a position of the nerve root on the ultrasound image by superimposing the nerve root position display element on the ultrasound image based on the inference by the inference unit. The ultrasonic diagnostic apparatus according to claim 8.

10. The display unit displays the nerve root position display element in an emphasized manner in the ultrasound image. The ultrasonic diagnostic apparatus according to claim 9.

11. the display unit displays the ultrasound image on which the nerve root position display element is superimposed and the ultrasound image on which the nerve root position display element is not superimposed side by side.

10. The ultrasonic diagnostic apparatus according to claim 9.

12. The display unit displays a label based on the inference by the inference unit in an area where no nerve root position display element is displayed in the ultrasound image. The ultrasonic diagnostic apparatus according to claim 9.

13. A training data acquisition unit that acquires pairs of ultrasound image data related to an ultrasound image and a ground truth label indicating a position of a nerve root in the ultrasound image; a training unit that uses the pairs as training data to generate a trained model that is trained to output an inference result regarding the nerve root in response to an input of the ultrasound image data; A training device comprising:

14. The ultrasound images used as the training data include bones present around the nerve root.

14. A training device according to claim 13.

15. The ultrasound images used as the training data include at least one of the bones of the fifth lumbar vertebra, the fifth lumbar vertebra, or the sacrum present around the nerve root.

15. A training device according to claim 14.

16. The ultrasound images used as the training data further include blood flow information flowing through blood vessels running parallel to the nerve root.

15. A training device according to claim 14.

17. An inference method executed by a computer constituting an inference device, comprising: Acquire ultrasound image data relating to an ultrasound image generated based on a reflected wave of the ultrasound transmitted to the subject; inputting the ultrasound image data as inference target data into a trained model, and performing inference regarding nerve roots in the ultrasound image based on an inference result output from the trained model; Reasoning method.

18. A display method executed by a computer constituting an ultrasound diagnostic apparatus, comprising: Transmitting ultrasonic waves to a subject and generating an ultrasonic image based on the reflected waves; acquiring ultrasound image data relating to the generated ultrasound image; inputting the ultrasound image data as inference target data into a trained model, and performing inference regarding nerve roots in the ultrasound image based on output data output from the trained model; generating a display relating to the ultrasound image and the nerve root based on the inference; Display method.

19. A program executed by a computer constituting an inference device, A step of acquiring ultrasound image data relating to an ultrasound image generated based on a reflected wave of an ultrasound transmitted to a subject; inputting the ultrasound image data as inference target data into a trained model, and performing inference regarding nerve roots in the ultrasound image based on output data output from the trained model; A program for causing the computer to execute the above.

20. A program executed by a computer constituting an ultrasound diagnostic apparatus, transmitting ultrasonic waves to a subject and generating an ultrasonic image based on the reflected waves; acquiring ultrasound image data relating to the generated ultrasound image; inputting the ultrasound image data as inference target data into a trained model, and performing inference regarding nerve roots in the ultrasound image based on output data output from the trained model; generating an indication of the nerve root based on the ultrasound image and the inference; A program for causing the computer to execute the above.

Images