Fault determination device, fault determination method, and program for ultrasonic diagnostic device
The ultrasonic diagnostic apparatus employs trained models to analyze multiple data types for accurate failure determination and component identification, addressing the limitations of existing diagnostic methods by integrating judgment results with priority settings and notification.
Patent Information
- Application Number
- JP2020095387
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-06-01
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2040-06-01
AI Technical Summary
Existing ultrasonic diagnostic apparatuses lack a comprehensive and accurate method for determining failures, particularly in the transmission elements, which are diagnosed based on a comparison with predetermined reference signals.
A failure determination device and method that utilizes trained models to analyze various types of data generated by a first ultrasonic diagnostic apparatus in a failure state, determining the failure state of a second apparatus by integrating judgment results from multiple data types, with priorities set based on data type and using a notification unit to inform the operator.
Enables high-accuracy failure determination in ultrasonic diagnostic apparatuses, identifying not only the presence of a failure but also the specific malfunctioning components, and can correct the failure state.
Smart Images

Figure 0007580944000001 
Figure 0007580944000002 
Figure 0007580944000003
Abstract
Description
[Technical field]
[0001] The present invention relates to an ultrasonic diagnostic device failure determination device, a failure determination method, and a program that are capable of determining a failure in an ultrasonic diagnostic device. [Background technology]
[0002] Some ultrasonic diagnostic devices are provided with a self-diagnosis mode for determining whether or not the ultrasonic diagnostic device has a malfunction.
[0003] For example, Patent Document 1 discloses that a self-diagnosis is performed on the functions of each of a plurality of transmitting elements in an ultrasound probe used in an ultrasound diagnostic device, and the functions of any transmitting element that is self-diagnosed as being impaired are restored. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2010-172411 A Summary of the Invention [Problem to be solved by the invention]
[0005] Patent Document 1 only discloses performing self-diagnosis of a specific element (transmitting element) in an ultrasonic probe, and if the received signal does not correspond to a predetermined reference signal, the specific element (transmitting element) is determined to be faulty.
[0006] Therefore, an object of the present invention is to provide a failure determination device for an ultrasonic diagnostic device that can accurately determine a failure. [Means for solving the problem]
[0007] In order to solve the above problem, the failure determination device for an ultrasonic diagnostic device includes a determination unit that determines whether or not the second ultrasonic diagnostic device is in a failure state from multiple types of data generated by a first ultrasonic diagnostic device that is in a failure state, using multiple trained models trained according to the types of data as teacher data, and a notification unit that notifies the determination result of the determination unit, and the determination unit integrates the determination results for each type of data output using the multiple trained models according to a priority indicating a weight used in determining a failure state according to the type of data, and determines whether or not the second ultrasonic diagnostic device is in a failure state. The priority is set so that data generated in a second process upstream of the first process in the second ultrasonic diagnostic device is given priority over data generated in a first process in the second ultrasonic diagnostic device. . Effect of the Invention
[0008] According to the failure determination device for an ultrasonic diagnostic apparatus of the present invention, failure determination can be performed with high accuracy. [Brief description of the drawings]
[0009] [Figure 1] 1 is a diagram showing the overall configuration of an ultrasonic diagnostic apparatus according to the present invention. [Diagram 2] FIG. 4 is a diagram showing a determination unit relating to a learning phase of the present invention. [Diagram 3] FIG. 4 is a diagram showing a determination unit relating to a learning phase of the present invention. [Figure 4] FIG. 1 is a diagram showing the internal configuration of an ultrasonic diagnostic apparatus according to the present invention. [Diagram 5] FIG. 2 is a diagram showing an example of the first ultrasonic diagnostic apparatus of the present invention in a faulty state. [Figure 6] FIG. 2 is a diagram showing a determination unit for an inference phase of the present invention. [Figure 7] FIG. 4 is a diagram showing a determination form of a determination unit relating to an inference phase of the present invention. [Figure 8] FIG. 1 is a diagram showing an example in which a learning device of the present invention is installed outside an ultrasound diagnostic device. [Figure 9] FIG. 2 is a diagram showing a display form of a display unit of the ultrasonic diagnostic apparatus of the present invention. [Figure 10] FIG. 4 is a diagram showing the operation of the learning phase of the present invention. [Figure 11] FIG. 4 is a diagram showing an operation of an inference phase in the first embodiment of the present invention. [Figure 12] FIG. 4 is a diagram showing an operation of an inference phase in the first embodiment of the present invention. [Figure 13] FIG. 11 is a diagram showing an operation of an inference phase in the second embodiment of the present invention. [Figure 14] FIG. 11 is a diagram showing an operation of an inference phase in the second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. EXAMPLES
[0011] 1 shows the configuration of an ultrasonic diagnostic device of the present invention. The ultrasonic diagnostic device includes an ultrasonic probe 100 that is brought into contact with a subject to transmit and receive ultrasonic waves, a device main body 102 that processes ultrasonic signals received by the ultrasonic probe 100 to generate ultrasonic images and perform various measurements, an operation unit 104 for operating the device main body 102, and a display unit 106 that displays ultrasonic images and measurement results.
[0012] The ultrasonic probe 100 is connected to the device body 102. The ultrasonic probe 100 has a plurality of transducers, and can generate ultrasonic waves by driving the plurality of transducers. The ultrasonic probe 100 receives reflected waves from a subject and converts them into electrical signals. The converted electrical signals are transmitted to the device body 102.
[0013] In addition, the ultrasonic probe 100 is provided with an acoustic matching layer on the front side (subject side) of the multiple transducers to match the acoustic impedance of the multiple transducers with that of the subject, and a backing material on the rear side of the multiple transducers to prevent the propagation of ultrasonic waves from the multiple transducers to the rear side.
[0014] The ultrasonic probe 100 is detachably connected to the device body 102. There are various types of ultrasonic probes 100, such as linear type, sector type, convex type, radial type, and three-dimensional scanning type, and an operator can select the type of ultrasonic probe 100 according to the imaging purpose. In addition, the type of sensor applied to the ultrasonic probe 100 is not limited to that using conventional bulk PZT. It is possible to use a type of capacitance probe called CMUT using microfabrication technology, or a type of probe called PMUT that also combines piezoelectric thin film technology.
[0015] The device main body 102 is equipped with a transmission / reception unit 110 that transmits and receives ultrasonic waves to and from the ultrasonic probe 100, a signal processing unit 112 that performs various signal processing using an ultrasonic signal based on a reflected wave signal received by the transmission / reception unit 110, an ultrasonic image generation unit 114 that generates ultrasonic image data using signal processing data signal processed in the signal processing unit 112, a judgment unit 116 that judges failures using data such as the reflected wave signal received by the transmission / reception unit 110, the signal processing data signal processed in the signal processing unit 112, and the ultrasonic image data generated by the ultrasonic image generation unit 114, and a control unit 118 that controls various components of the device main body 102.
[0016] The transmitting / receiving unit 110 controls the transmission and reception of ultrasonic waves performed by the ultrasonic probe 100. The transmitting / receiving unit 110 has a transmitting unit, a transmission delay circuit, etc., and supplies a drive signal to the ultrasonic probe 100. The transmitting unit repeatedly generates a rate pulse of a predetermined repetition frequency (PRF). The transmission delay circuit focuses the ultrasonic waves generated from the ultrasonic probe 100 and provides a delay time for determining the transmission directivity to the rate pulse generated by the transmitting unit. The transmission delay circuit can control the transmission direction of the ultrasonic waves transmitted from the transducer by changing the delay time provided to the rate pulse.
[0017] The transmitting / receiving unit 110 also has an amplifier, an A / D conversion unit, a reception delay circuit, an adder, etc. Various processes are performed on the reflected wave signal received by the ultrasonic probe 100 to generate an ultrasonic signal. The amplifier amplifies the reflected wave signal for each channel and performs gain correction processing. The A / D conversion unit A / D converts the gain-corrected reflected wave signal. The reception delay circuit provides a delay time to the digital data to determine the reception directivity. The adder performs addition processing of the reflected wave signal to which a delay time has been provided by the reception delay circuit. The addition processing by the adder emphasizes the reflected component from a direction corresponding to the reception directivity of the reflected wave signal.
[0018] When performing two-dimensional scanning of a subject, the transmitting / receiving unit 110 causes the ultrasound probe 100 to transmit two-dimensional ultrasound. Then, the transmitting / receiving unit 110 generates a two-dimensional ultrasound signal from the two-dimensional reflected wave signal received by the ultrasound probe 100. When performing three-dimensional scanning of a subject, the transmitting / receiving unit 110 causes the ultrasound probe 100 to transmit three-dimensional ultrasound. Then, the transmitting / receiving unit 110 generates a three-dimensional ultrasound signal from the three-dimensional reflected wave signal received by the ultrasound probe 100.
[0019] The signal processing unit 112 performs various signal processing on the ultrasonic signal output from the transmitting / receiving unit 110. Specifically, the signal processing unit 112 performs signal processing on the ultrasonic signal, such as detection processing and logarithmic compression. The signal processing unit 112 visualizes the amplitude information of the ultrasonic signal and generates signal processing data (raster data). The signal processing unit 112 performs bandpass filter processing on the ultrasonic signal output from the transmitting / receiving unit 110, and then detects the envelope of the output signal. Then, the detected data is compressed by logarithmic transformation. The signal processing unit 112 outputs the signal processing data after the signal processing to the ultrasonic image generating unit 114.
[0020] The ultrasonic image generating unit 114 generates ultrasonic image data using the signal processed data that has been signal processed by the signal processing unit 112. The ultrasonic image generating unit 114 has a digital scan converter and converts the signal processed data into data represented in orthogonal coordinates. Here, the ultrasonic image generating unit 114 orthogonally converts the signal processed data (raster data) into a coordinate system (X, Y) of image data for display. Then, the ultrasonic image generating unit 114 generates ultrasonic image data (B-mode image data) in which signal intensity is expressed by brightness of luminance.
[0021] The determination unit 116 checks data such as the reflected wave signal received by the transmitting / receiving unit 110, various signal processing data (raster data), and ultrasound image data, and performs a failure determination for the ultrasound diagnostic device.
[0022] Specifically, the determination unit 116 has a trained model trained using data generated in a first ultrasound diagnostic device in a faulty state as teacher data. The determination unit 116 then uses the trained model to determine whether or not the second ultrasound diagnostic device is in a faulty state from data generated in the second ultrasound diagnostic device used to diagnose the subject. In the present invention, the first ultrasound diagnostic device is an ultrasound diagnostic device for generating a trained model using data acquired in advance (in the past). The second ultrasound diagnostic device is an ultrasound diagnostic device for which a faulty state is to be determined.
[0023] The operation unit 104 includes, for example, a mouse, a keyboard, a button, a panel switch, a touch command screen, a foot switch, a trackball, and a joystick. The operation unit 104 receives various instructions from an operator of the ultrasound diagnostic apparatus, and transmits the received instructions to the apparatus body 102.
[0024] The display unit 106 displays a GUI for the operator to input various instructions using the operation unit 104, and displays ultrasound images, blood flow images, measurement results, and the like generated in the device main body 102. Furthermore, when the determination unit 116 determines that the second ultrasound diagnostic device is in a faulty state, the display unit 106 displays a message to that effect that the second ultrasound diagnostic device is in a faulty state. Although the display unit 106 is illustrated separately from the notification unit 108, the display unit 106 can also be regarded as the notification unit 108.
[0025] The transmitter / receiver unit 110, signal processing unit 112, ultrasound image generation unit 114, and judgment unit 116 in the device main body 102 may be configured as hardware such as an integrated circuit, or may be modularized software programs.
[0026] The ultrasonic diagnostic device of the present invention uses a trained model that has been trained to determine a fault state using data such as received reflected wave signals, various signal processing data (raster data), and ultrasonic image data, for fault determination. The trained model is, for example, a trained neural network, but any model such as deep learning or a support vector machine may be used. The trained model may be stored in the determination unit 116, or may be connected to the ultrasonic diagnostic device via a network.
[0027] Here, the first ultrasonic diagnostic device and the second ultrasonic diagnostic device will be described. The first ultrasonic diagnostic device is an ultrasonic diagnostic device that has actually failed or that has a simulated failed state. Data such as reflected wave signals, signal processing data, and ultrasonic image data acquired from the first ultrasonic diagnostic device in a failed state are learned as teacher data to generate a trained model. The determination unit 116 uses the trained model to determine the failure state of the second ultrasonic diagnostic device that is the target of failure determination, and notifies the operator.
[0028] The ultrasound image data input to the determination unit 116 (trained model) may be two-dimensional image data or three-dimensional image data (volume data).
[0029] As described above, the trained model is trained to determine the fault state of the second ultrasonic diagnostic device. A trained model for determining the fault state from two-dimensional image data may be used, or a trained model for determining the fault state from three-dimensional image data may be used.
[0030] The details of the determination unit 116 in the ultrasonic diagnostic device of the present invention will be described with reference to Figures 2 to 4. The schematic configuration of the determination unit 116 in Figures 2 to 4 is the same. Different drawings are used to distinguish between the operations in the learning phase and the inference phase. Figures 2 and 3 show the operations of the determination unit 116 related to the learning phase using a first ultrasonic diagnostic device, and Figure 4 shows the operations of the determination unit 116 related to the inference phase using a second ultrasonic diagnostic device.
[0031] As shown in FIG. 2, the judgment unit 116 includes a learning device 200 that learns data acquired from a first ultrasound diagnostic device in a faulty state as teacher data and generates a trained model, and a memory unit 206 that stores the trained model generated in the learning device 200.
[0032] The learning device 200 comprises a teacher data generation unit 202 that generates teacher data related to a fault state from data acquired from a first ultrasonic diagnostic device that is in a fault state, and a learning unit 204 that uses the teacher data generated in the teacher data generation unit 202 to learn the fault state of a second ultrasonic diagnostic device.
[0033] The teacher data generation unit 202 generates teacher data using at least one of the reflected wave signal received by the transceiver unit 110 in the first ultrasound diagnostic device in a faulty state, the signal processing data processed by the signal processing unit 112, and the ultrasound image data generated by the ultrasound image generation unit 114.
[0034] The learning device 200 can learn by causing an actual failure of the first ultrasonic diagnostic device and using data acquired from the first ultrasonic diagnostic device in a failed state. It is also possible to set the first ultrasonic diagnostic device in a state simulating a failure and learn by using data acquired from the first ultrasonic diagnostic device in that state. Setting the first ultrasonic diagnostic device in a state simulating a failure means, for example, creating a disconnection state inside the first ultrasonic diagnostic device.
[0035] 3 shows an embodiment including a learning device that learns the reflected wave signal received by the transmitting / receiving unit 110, the signal processing data that has been signal processed by the signal processing unit 112, and the ultrasound image data generated by the ultrasound image generating unit 114. Here, the learning device generates a trained model according to the type of data generated by the first ultrasound diagnostic device.
[0036] Specifically, the learning device 200 includes a first learning device 220 that performs learning using a reflected wave signal received by the transmitting / receiving unit 110 in the first ultrasonic diagnostic device in a faulty state to generate a first trained model, a second learning device 222 that performs learning using signal processing data that has been signal-processed in the signal processing unit 112 of the first ultrasonic diagnostic device in a faulty state to generate a second trained model, and a third learning device 224 that performs learning using ultrasound image data generated by the ultrasound image generating unit 114 in the first ultrasonic diagnostic device in a faulty state to generate a third trained model. Note that the learning device 200 only needs to generate a plurality of trained models, and is not limited to three trained models (the first trained model, the second trained model, and the third trained model).
[0037] From the reflected wave signal received by the first ultrasonic diagnostic device, it is possible to capture changes in noise level and noise band characteristics. At this time, it is also possible to transmit and receive without touching the ultrasonic probe to the subject. Then, it is possible to determine which circuit or wiring of the element of each channel of the ultrasonic probe is faulty from the amplitude, band characteristics, crosstalk level, and state of multiple reflection of the reflected wave signal, so that the first trained model can be generated by using the reflected wave signal as training data.
[0038] When the element group selected for transmission and reception is changed, for example, data of different scan lines is compared from the signal processing data processed in the first ultrasonic diagnostic device. Then, as a result of comparing the data of the scan lines, it is possible to determine which channel's circuit has a fault, and therefore it is possible to generate a second trained model using this signal processing data as training data.
[0039] Similarly, when the element group selected for transmission and reception is changed, image data of different scan lines, for example, are compared from the ultrasound image data generated in the first ultrasound diagnostic device. As a result of comparing the image data of the scan lines, it is possible to determine which channel's circuit has a failure, and therefore a third trained model can be generated using this ultrasound image data as training data.
[0040] Fig. 4 is a diagram showing the internal configuration of the ultrasound diagnostic device of the present invention. Fig. 4 is a diagram showing the configuration of the transmission / reception unit 110, the signal processing unit 112, the ultrasound image generating unit 114 and their peripherals according to the embodiment of the present invention. Fig. 4 shows a case where the transmission / reception unit 110 is configured with 32 channels, but this is not necessarily limited to 32 channels, and the number of channels of the transmission / reception unit 110 can be appropriately determined depending on the specifications of the ultrasound diagnostic device.
[0041] 4, the transmitting / receiving unit 110 is configured with a transmitting / receiving circuit 710 for transmitting and receiving ultrasonic waves, A / D conversion units 730-1 to 730-32 for converting analog signals to digital signals for reflected wave signals received by the ultrasonic probe 100, data memories 731-1 to 731-32 for storing digital signal data, and a multiplexer 734 for arbitrarily selecting data output ports of the data memories 731-1 to 731-32 and connecting the selected data output port to the ultrasonic image generating unit 114. The transmitting / receiving unit 110 is also configured with multipliers 732-1 to 732-32 for apodization, and an adding circuit 733. The transmitting / receiving unit 110 may also include data memory control circuits 735-1 to 735-32, weighting coefficient supply circuits 736-1 to 736-32, and a gain control circuit 737.
[0042] The ultrasonic transmission waveform generated by the transmission control unit 702 is transmitted from the ultrasonic probe 100 via the transmission and reception circuit 710 .
[0043] The ultrasonic signals acquired by the ultrasonic probe 100 and converted into analog electric signals are passed through the transmission / reception circuit 710 and converted into digital signals in the A / D conversion units 730-1 to 730-32. At this time, the A / D conversion units 730-1 to 730-32 sample the ultrasonic signals using a sampling clock supplied from a clock source (not shown), and convert the analog electric signals into digital signals. An appropriate sampling clock is supplied to the A / D conversion units 730-1 to 730-32 from a clock source (not shown) according to the frequency band of the ultrasonic signals to be acquired.
[0044] Although it depends on the performance of the A / D conversion units 730-1 to 730-32, digital signals having a bit width of about 12 bits to 16 bits are output from the A / D conversion units 730-1 to 730-32. The digital signals output from the A / D conversion units 730-1 to 730-32 are taken into the corresponding data memories 731-1 to 731-32 under the control of the data memory control circuits 735-1 to 735-32. The data memories 731-1 to 731-32 have a capacity capable of storing digital signals for the maximum measurement depth in the subject.
[0045] Furthermore, the A / D conversion units 730-1 to 730-32 are subjected to gain control such as TGC (Time Gain Control) control from a gain control circuit 737. For example, in order to acquire ultrasound image data having uniform contrast regardless of the measurement depth, the amplification gain is increased or decreased according to the time from transmission of the ultrasound to reception of the ultrasound. By performing TGC control, ultrasound image data having uniform contrast can be acquired.
[0046] The control unit 118 performs control to write the digital signals output from the A / D conversion units 730-1 to 730-32 to the data memories 731-1 to 731-32. In addition, the control unit 118 can also perform control to arbitrarily select the data memories 731-1 to 731-32 by controlling the multiplexer 734 to read out the digital signals and transfer them to the ultrasonic image generation unit 114. When performing this control, the ultrasonic image generation unit 114 generates ultrasonic image data without going through the apodization multipliers 732-1 to 732-32, the adder circuit 733, and the signal processing unit 112. In many cases, the control unit 118 sequentially selects all of the data memories 731-1 to 731-32 to read out the digital signals and transfer them to the ultrasonic image generation unit 114.
[0047] Ultrasonic image data is generated in the ultrasonic image generating unit 114. As an algorithm for generating ultrasonic image data, not only delay and sum processing but also any algorithm can be applied to perform image reconstruction.
[0048] Typically, delay-and-sum processing is performed when generating an ultrasound image, but image reconstruction using an algorithm other than delay-and-sum processing may be performed. For example, other image reconstruction methods include back projection in the time domain or Fourier domain, which is commonly used in tomography technology. In this way, image reconstruction using any algorithm other than delay-and-sum processing may be performed.
[0049] The output voltage and waveform of the transmission control unit 702 may be converted into data, and data on a fault state in the data may be used as training data. In addition, attention may be paid to waveform data that is reflected when the waveform output by the transmission control unit 702 hits the ultrasonic probe 100, and data on a fault state in the reflected waveform data may be used as training data to generate a trained model.
[0050] It is also possible to determine which channel of the transmission / reception circuit 710 has a fault. Specifically, the peak level of the waveform data, the rising and falling characteristics of the waveform, the bandwidth characteristics of the waveform, crosstalk, etc. are checked. As a result of these checks, the cause of the fault can be isolated to a fault in the transmission circuit power supply, a fault in the driver circuit of the transmission circuit, damage to the transmission circuit board, etc., but the items that can be detected as faults are not necessarily limited to these. Anything that can cause deterioration of the transmission characteristics of the ultrasound diagnostic device can be used.
[0051] Furthermore, the method of ultrasonic transmission for generating data to which the fault judgment is applied is not limited to a specific method. For example, in addition to a focused transmission beam, a transmission waveform such as a plane wave or a diverging wave can be used. As long as the transmission waveform is effective for ultrasonic image diagnosis, fault judgment, etc., the transmission method is not limited to a specific method.
[0052] In this way, the ultrasonic diagnostic apparatus of the present invention can determine not only that a malfunction has occurred, but also which part is malfunctioning.
[0053] In addition, in order to determine the detailed state of a fault, data calculated as a result of processing data by adding, subtracting, multiplying, and dividing, or any combination thereof, may be used as training data. This includes cases where a Fourier transform is performed on transmitted waveforms, received waveforms, or image data, or differential processing such as edge detection is performed, and the data is used as training data. Alternatively, the results of inference using artificial intelligence on data generated within an ultrasound diagnostic device can be used as training data.
[0054] Learning device 200 may perform learning using, for example, digital signals stored in data memories 731-1 to 731-32 output from multiplexer 734. Alternatively, learning may be performed using data output from adder circuit 733 and signal processing unit 112. The data used as teacher data by learning device 200 may be data from any part of the processing flow performed for generating an ultrasound image in an ultrasound diagnostic device. The types and combinations of the data are not limited to specific ones.
[0055] In this way, the learning unit 204 can learn the characteristics of the fault state of the ultrasonic diagnostic device using any data processed by the ultrasonic diagnostic device.
[0056] The data generated by the ultrasonic diagnostic device will be exemplified. For example, the reflected wave signal data corresponds to the digital signals output from the A / D conversion units 730-1 to 730-32, or the digital signals output from the multiplexer 734 and stored in the data memories 731-1 to 731-32. The signal processing data corresponds to the output data of the signal processing unit 112. The ultrasonic image data corresponds to the ultrasonic image data generated by the ultrasonic image generating unit 114, but when the ultrasonic image generating unit 114 is composed of a plurality of signal processing blocks and image processing blocks, the output of each signal processing block and image processing block also corresponds to the ultrasonic image data. Here, any part of the data flow of the ultrasonic diagnostic device may be used for failure judgment as long as it is useful for failure judgment, and is not limited to a specific one.
[0057] 5 shows an example of the first ultrasonic diagnostic device in a faulty state. In the transmission / reception circuit 710, all of the transducers 100-1 to 100-32 in the ultrasonic probe should be connected to the transmission control unit 702 in a normal state. Here, a faulty state (here, a disconnection state) can be created by setting a part 701-3 of the transmission / reception circuit 710 in an open state. In the same way, it is possible to make the operation settings of other circuits of the first ultrasonic diagnostic device different from the normal state, create a pseudo faulty state, and learn the faulty state of the first ultrasonic diagnostic device.
[0058] In addition, in an external device, the reflected wave signal, signal processing data, and ultrasound image data generated from the first ultrasound diagnostic device in a faulty state can be generated by simulation and used for learning. The data generated by simulation may be data that represents a faulty state for any combination of two-dimensional or three-dimensional still images or video images actually captured when the ultrasound diagnostic device is not faulty. Alternatively, the data may be data that represents a faulty state for data obtained by performing a simulation using one or more simulation models with different sizes, locations, and physical properties of wire phantoms or cysts.
[0059] The learning unit 204 uses, for example, a neural network and includes multiple layers. The multiple layers have multiple intermediate layers between an input layer and an output layer. Although not shown, the multiple intermediate layers are composed of a convolution layer, a pooling layer, an upsampling layer, a synthesis layer, and the like. The convolution layer is a layer that performs convolution processing on an input value group. In the convolution layer, the input ultrasound image (region of interest) is convolved to extract features of the ultrasound image (region of interest).
[0060] The pooling layer is a layer that performs processing to make the number of output value groups less than the number of input value groups by thinning out or combining input value groups. The upsampling layer is a layer that performs processing to make the number of output value groups greater than the number of input value groups by duplicating input value groups or adding values interpolated from the input value groups. The synthesis layer is a layer that inputs value groups such as the output value group of a certain layer or the pixel values that make up an ultrasound image (region of interest) from multiple sources and performs processing to combine them by concatenating or adding them. The number of intermediate layers can be changed at any time depending on the learning content.
[0061] In this manner, the learning device 200 (learning unit 204) uses a neural network to learn by associating data on the fault state of the ultrasound diagnostic device as teacher data, thereby generating a learned model.
[0062] The judgment unit 116 related to the inference phase will be described with reference to Fig. 6. A trained model related to target data used when performing fault judgment in the second ultrasonic diagnostic device is generated in the learning device 200. The storage unit 206 is connected to the learning device 200. The trained model trained to judge a fault state is stored in the storage unit 206. Specifically, the storage unit 206 stores a trained model trained to judge a fault state from data acquired from the second ultrasonic diagnostic device.
[0063] For example, the memory unit 206 stores a first trained model generated in a first learning device 220, a second trained model generated in a second learning device 222, and a third trained model generated in a third learning device 224.
[0064] Then, the reflected wave signal received by the transmitting / receiving unit 110 in the second ultrasonic diagnostic device is output to the inference unit 208. The inference unit 208 reads out from the storage unit 206 a first trained model that has been trained to determine a fault state, and uses the first trained model to determine a fault state from the reflected wave signal received by the transmitting / receiving unit 110. Furthermore, signal processing data that has been signal-processed in the signal processing unit 112 in the second ultrasonic diagnostic device is output to the inference unit 208. The inference unit 208 reads out from the storage unit 206 a second trained model that has been trained to determine a fault state, and uses the second trained model to determine a fault state from the signal processing data that has been signal-processed in the signal processing unit 112. Furthermore, ultrasonic image data generated in the ultrasonic image generating unit 114 in the second ultrasonic diagnostic device is output to the inference unit 208. The inference unit 208 reads out from the storage unit 206 a third trained model that has been trained to determine a fault state, and uses the third trained model to determine a fault state from the ultrasonic image data output by the ultrasonic image generating unit 114.
[0065] Here, the transmitter / receiver 110, the signal processor 112, the ultrasound image generator 114, and the inference unit 208 are linked, but the present invention is not limited to this. Any data or any combination of data used in the ultrasound diagnostic device can be linked to the inference unit 208, and a fault state can be determined.
[0066] For example, any of reflected wave signal data, signal processing data, and ultrasound image data acquired from an ultrasound diagnostic device, or any combination thereof, can be used for fault determination by linking it to the inference unit 208. Furthermore, the data linked to the inference unit 208 for fault determination is not limited to a specific type of data, as long as it is useful for fault determination.
[0067] The determination form of the determination unit 116 regarding the inference phase will be described with reference to FIG. 7. Cases 1 to 12 shown in FIG. 7 are examples of determination using a reflected wave signal, signal processing data, and ultrasonic image data. Here, the determination unit 116 sets a priority according to the type of data generated by the second ultrasonic diagnostic device, and determines whether the second ultrasonic diagnostic device is in a faulty state according to the priority. The priority is set for each of the reflected wave signal, the signal processing data, and the ultrasonic image data, for example. Note that the priority may also be set for other signals and data generated by the second ultrasonic diagnostic device. The operator can set the priority via the operation unit. The priority is weighted according to the processing content of the data, and the weight can be set in the order of the reflected wave signal, the signal processing data, and the ultrasonic image data, for example.
[0068] 7 indicates a state determined to be a fault state from the reflected wave signal, a state determined to be a fault state from the signal processing data, or a state determined to be a fault state from the ultrasound image data. The symbol ◯ indicates a state determined to be a normal state from the reflected wave signal, a state determined to be a normal state from the signal processing data, or a state determined to be a normal state from the ultrasound image data.
[0069] Cases 1 to 8 are forms in which the state of the second ultrasonic diagnostic device is determined using three pieces of data (reflected wave signals, signal processing data, and ultrasonic image data). Here, three pieces of data will be described, but the present invention is not limited to these pieces of data and can also be applied to other pieces of data.
[0070] In case 1, the reflected wave signal, the signal processing data, and the ultrasound image data are all the symbol ×. The determination unit 116 determines that the second ultrasound diagnostic apparatus is in a faulty state.
[0071] In case 2, the reflected wave signal and the signal processing data are indicated by the symbol x, and the ultrasound image data are indicated by the symbol o. The judgment of the reflected wave signal and the signal processing data is different from the judgment of the ultrasound image data, but the judgment of the upstream processing in the ultrasound diagnostic device takes precedence. Here, the priority relationship is as follows: judgment of the reflected wave signal > judgment of the signal processing data > judgment of the ultrasound image data. Since the reflected wave signal and the signal processing data are indicated by the symbol x, the judgment unit 116 judges that the second ultrasound diagnostic device is in a faulty state.
[0072] In case 3, the reflected wave signal and the ultrasound image data are represented by the symbol x, and the signal processing data is represented by the symbol o. Using the above priority relationship, the determination unit 116 determines that the second ultrasound diagnostic device is in a faulty state because the reflected wave signal is represented by the symbol x.
[0073] In case 4, the reflected wave signal is the symbol ×, and the ultrasound image data and the signal processing data are the symbols O. Since the reflected wave signal is the symbol × but the other two pieces of data are the symbol O, and there is a possibility that only the reflected wave signal is affected by others, the determination unit 116 does not determine that the second ultrasound diagnostic device is in a faulty state, but determines that there is a possibility of a fault.
[0074] In case 5, the reflected wave signal, the signal processing data, and the ultrasound image data are all marked with the symbol .O slashed. The determination unit 116 determines that the second ultrasound diagnostic apparatus is in a normal state.
[0075] In case 6, the reflected wave signal and the signal processing data are indicated by the symbol ◯, and the ultrasound image data are indicated by the symbol ×. The judgment of the reflected wave signal and the signal processing data is different from the judgment of the ultrasound image data, but the judgment of the upstream processing in the ultrasound diagnostic device takes precedence. Since the reflected wave signal and the signal processing data are indicated by the symbol ◯, the judgment unit 116 judges that the second ultrasound diagnostic device is in a normal state.
[0076] In case 7, the reflected wave signal and the ultrasound image data are represented by the symbol ◯, and the signal processing data is represented by the symbol ×. Using the above priority relationship, the determination unit 116 determines that the second ultrasound diagnostic device is in a normal state because the reflected wave signal is represented by the symbol ◯.
[0077] In case 8, the reflected wave signal is indicated by the symbol ◯, and the ultrasound image data and the signal processing data are indicated by the symbol ×. Since the reflected wave signal is indicated by the symbol ◯ while the other two pieces of data are indicated by the symbol × and there is a possibility that only the reflected wave signal is affected by others, the determination unit 116 does not determine that the second ultrasound diagnostic device is in a normal state, but determines that there is a possibility of a malfunction.
[0078] Cases 9 to 12 are forms in which the state of the second ultrasonic diagnostic device is determined using two sets of data (signal processing data, ultrasonic image data).
[0079] In cases 9 and 10, the signal processing data is the symbol x. The judgment of the upstream processing in the ultrasound diagnostic device has priority, and the priority relationship is such that the judgment of the signal processing data > the judgment of the ultrasound image data. Since the signal processing data is the symbol x, the judgment unit 116 judges that the second ultrasound diagnostic device is in a faulty state.
[0080] In cases 11 and 12, the signal processing data is the symbol ◯. The judgment of the upstream processing in the ultrasound diagnostic device has priority, and the priority relationship is such that the judgment of the signal processing data > the judgment of the ultrasound image data. Since the signal processing data is the symbol ◯, the judgment unit 116 judges that the second ultrasound diagnostic device is in a normal state.
[0081] The learning device 200 may be installed outside the ultrasound diagnostic device. Fig. 8 shows an example in which the learning device 200 is installed outside the ultrasound diagnostic device.
[0082] The learning device 200 may be located, for example, in a network within a hospital or in a cloud outside the hospital. The learning device 200 is connected to multiple ultrasound diagnostic devices 500, 502, and 504. Here, a form in which there are three multiple ultrasound diagnostic devices is shown, but there may be four or more multiple ultrasound diagnostic devices.
[0083] For example, the learning device 200 learns data of a fault state generated in an ultrasound diagnostic device 500 as teacher data and generates a trained model. The learning device 200 also learns data of a fault state generated in an ultrasound diagnostic device 502 different from the ultrasound diagnostic device 500 as teacher data and updates the trained model. Similarly, the learning device 200 learns data of a fault state generated in an ultrasound diagnostic device 504 different from the ultrasound diagnostic device 500 and the ultrasound diagnostic device 502 as teacher data and updates the trained model. The trained model generated (updated) by the learning device 200 is transmitted to each of the multiple ultrasound diagnostic devices 500, 502, and 504. The multiple ultrasound diagnostic devices 500, 502, and 504 each store the latest trained model generated by the learning device 200.
[0084] In this way, the learning device 200 can learn the fault state data set in the multiple ultrasound diagnostic devices 500, 502, 504 as teacher data. Therefore, the learning device 200 can generate trained models corresponding to the multiple ultrasound diagnostic devices 500, 502, 504. The learning device 200 can also learn the fault state data generated in the multiple ultrasound diagnostic devices 500, 502, 504 collectively as teacher data. Here, the multiple ultrasound diagnostic devices 500, 502, 504 can be either the first ultrasound diagnostic device or the second ultrasound diagnostic device.
[0085] Next, the notification form (display form) of the notification unit 108 (display unit 106) will be described with reference to Fig. 9. The notification unit 108 notifies that the determination unit has determined that the second ultrasonic diagnostic device is in a faulty state. In this case, in addition to the faulty state, the notification unit 108 may also notify the location of the fault.
[0086] The inference unit 208 uses a trained model that has been trained to determine faults using data generated from the second ultrasound diagnostic device to check the ultrasound image data 600 displayed on the display unit 106 and the data inside the ultrasound diagnostic device, and determines whether or not a fault exists.
[0087] Then, when the inference unit 208 determines that the second ultrasonic diagnostic device is in a faulty state, the notification unit 108 notifies the operator. The notification unit 108 may function as a display function of the display unit 106, or may notify using an alarm sound or the like. When the second ultrasonic diagnostic device determines that it is in a faulty state, not only is the fault determination displayed on the display unit 106 to inform the operator, but also a contact number for a repair request is displayed on the display unit 106. The notification unit 108 may notify a call center. Note that, regarding the notification by the notification unit 108, any approach may be adopted so as not to affect the ultrasonic diagnosis, and is not limited to a specific approach.
[0088] The operation of the ultrasonic diagnostic apparatus in the learning phase will be described with reference to FIG.
[0089] S700: In the first ultrasonic diagnostic device, data of a fault state is acquired. For example, data generated in the first ultrasonic diagnostic device may be reflected wave signals, signal processing data, ultrasonic image data, etc., which are actually faulty or simulate a fault state. In addition, simulated data of a fault state generated by an external device of the first ultrasonic diagnostic device may be input to the first ultrasonic diagnostic device.
[0090] S701: The learning device 200 learns data of a fault state as training data and generates a trained model. After S701, the operation of the learning phase ends.
[0091] Next, the operation of the ultrasonic diagnostic apparatus in the inference phase will be described with reference to FIG.
[0092] FIG. 11 shows a case where failure determination is performed on data before it becomes ultrasonic image data in the second ultrasonic diagnostic device.
[0093] S800: The transmitting / receiving unit 110 causes the ultrasound probe 100 to transmit and receive ultrasound. The ultrasound probe 100 may transmit and receive ultrasound in the air.
[0094] S801: The determination unit 116 performs failure determination using at least one of reflected wave signal data, signal processing data, and ultrasonic image data acquired by causing the ultrasonic probe 100 to transmit and receive ultrasonic waves.
[0095] If a malfunction is determined, the process proceeds to S803, where an alarm is issued using the notification unit 108, and thereafter the process proceeds to S802, where the ultrasound image generating unit 114 generates an ultrasound image.
[0096] If no malfunction is detected, the process proceeds to S802, where the ultrasound image generating unit 114 generates an ultrasound image.
[0097] S802: The ultrasound image generating unit 114 generates an ultrasound image using the data generated in S801. In this step, image processing can also be performed to improve the visibility of the ultrasound image.
[0098] S806: The display unit 106 displays the ultrasound image generated by the ultrasound image generation unit 114, and the notification unit 108 (display unit 106) displays a warning if a malfunction is determined. After S806, the operation of the inference phase ends.
[0099] Next, the operation of the ultrasonic diagnostic apparatus in the inference phase will be described with reference to FIG.
[0100] FIG. 12 shows a case where a fault determination is performed on ultrasonic image data generated by a second ultrasonic diagnostic device.
[0101] S800: The operator causes the transmitting / receiving unit 110 to transmit and receive ultrasonic waves to and from the ultrasonic probe 100. The ultrasonic probe 100 may transmit and receive ultrasonic waves in the air.
[0102] S802: The ultrasound image generating unit 114 generates ultrasound image data by performing various signal processing on the reflected wave signal from the transmitting / receiving unit 110. In this step, in addition to the signal processing, image processing for improving the visibility of the ultrasound image can also be performed.
[0103] S804: The determination unit 116 performs a fault determination using the ultrasound image data generated by the ultrasound image generation unit 114 in S802. If a fault is determined, the process proceeds to S805, where the notification unit 108 (display unit 106) displays a warning, and then the process proceeds to S806, where the ultrasound image is displayed.
[0104] If no malfunction is detected, the process proceeds to S806, where the display unit 106 displays an ultrasound image.
[0105] S806: The display unit 106 displays a warning if a malfunction is determined, in addition to the ultrasound image generated by the ultrasound image generating unit 114. After S806, the operation of the inference phase ends.
[0106] As described above, the failure determination device for an ultrasonic diagnostic device of the present invention includes a determination unit 116 that determines whether or not the second ultrasonic diagnostic device is in a failure state from data generated by the second ultrasonic diagnostic device using a trained model trained using data generated by the first ultrasonic diagnostic device in a failure state as teacher data, and a notification unit 108 that notifies the determination result of the determination unit 116. Thus, according to the present invention, failure determination of the ultrasonic diagnostic device can be performed with high accuracy.
[0107] The trained model may be generated using an ultrasound diagnostic device in a normal state. The determination unit 116 may determine whether the second ultrasound diagnostic device is in a faulty state from data generated in the second ultrasound diagnostic device using a trained model trained using data generated in the third ultrasound diagnostic device in a normal state as teacher data. The device includes a learning device that learns the reflected wave signal received by the transmission / reception unit 110 in the third ultrasound diagnostic device in a normal state, the signal processing data processed in the signal processing unit 112, and the ultrasound image data generated by the ultrasound image generating unit 114 to generate a trained model. The determination unit 116 uses the trained model to determine whether the second ultrasound diagnostic device is in a faulty state from the data generated in the second ultrasound diagnostic device. The notification unit 108 then notifies the determination result of the determination unit 116. EXAMPLES
[0108] An ultrasonic diagnostic device in the second embodiment of the present invention will be described with reference to Figures 13 and 14. The second embodiment is different from the first embodiment in that, when the second ultrasonic diagnostic device is in a faulty state, the control unit 118 performs a process of correcting data of the faulty state of the second ultrasonic diagnostic device. For example, the control unit 118 corrects data of the faulty state of the second ultrasonic diagnostic device by using data of a normal state that is not determined to be a faulty state in the second ultrasonic diagnostic device.
[0109] Although not shown in the drawings, in the learning phase of the second embodiment of the present invention, data in a normal state of the first ultrasonic diagnostic device is associated with data in a faulty state, and the learning device 200 is caused to learn the data.
[0110] The operation of the second ultrasonic diagnostic device in the inference phase will be described with reference to Fig. 13. Fig. 13 shows a case where a fault determination is performed on data generated by the second ultrasonic diagnostic device and not yet converted into ultrasonic image data.
[0111] S800: The operator causes the transmitting / receiving unit 110 to transmit and receive ultrasonic waves to and from the ultrasonic probe 100. The ultrasonic probe 100 may transmit and receive ultrasonic waves in the air.
[0112] S801: The ultrasonic diagnostic apparatus performs fault determination using the reflected wave signals and signal processing data acquired by transmitting and receiving ultrasonic waves to and from the ultrasonic probe 100.
[0113] If a failure is determined, the process proceeds to S803, where the notification unit 108 (display unit 106) displays a warning. The control unit 118 then corrects the data that is in a failure state in the process of S813, and the process proceeds to S802, where the ultrasound image generating unit 114 generates an ultrasound image. In this manner, it is possible to easily recover the ultrasound diagnostic device from the failure state.
[0114] If no malfunction is detected, the process proceeds to S802, where the ultrasound image generating unit 114 generates an ultrasound image.
[0115] S802: The ultrasound image generating unit 114 generates an ultrasound image using the data generated in S801. In this step, image processing is also performed to improve the visibility of the ultrasound image.
[0116] S806: The display unit 106 displays the ultrasound image generated by the ultrasound image generating unit 114, and the notification unit 108 displays a warning if a malfunction is determined. After S806, the operation of the inference phase ends.
[0117] FIG. 14 shows a case where a fault determination is performed on ultrasonic image data generated by an ultrasonic diagnostic device.
[0118] S800: The operator causes the transmitting / receiving unit 110 to transmit and receive ultrasonic waves to and from the ultrasonic probe 100. The ultrasonic probe 100 may transmit and receive ultrasonic waves in the air.
[0119] S802: The ultrasound image generating unit 114 generates ultrasound image data by performing various signal processing on the reflected wave signal from the transmitting / receiving unit 110. In this step, in addition to the signal processing, image processing for improving the visibility of the ultrasound image is also performed.
[0120] S804: The ultrasound diagnostic device performs a fault determination using the ultrasound image data generated by the ultrasound image generating unit 114. If a fault is determined, the process proceeds to S805, where the notification unit 108 displays a warning on the display unit 106. The control unit then corrects the data of the fault state in the process of S815, and the process proceeds to S806 to display the ultrasound image. In this manner, it is possible to easily recover the ultrasound diagnostic device from the fault state.
[0121] If no malfunction is detected, the process proceeds to S806, where the display unit 106 displays an ultrasound image.
[0122] S806: The display unit 106 displays the ultrasound image generated by the ultrasound image generation unit 114. The notification unit 108 displays a warning if a malfunction is determined. After S806, the operation of the inference phase ends.
[0123] As described above, the ultrasonic diagnostic device failure determination device in this embodiment determines a failure state of the second ultrasonic diagnostic device and corrects the data generated by the second ultrasonic diagnostic device, thereby enabling the ultrasonic diagnostic device to easily recover from the failure state.
[0124] A computer program for realizing the functions of the first and second embodiments can be supplied to a computer via a network or a storage medium (not shown), and the computer program can be executed. The computer program is a computer program for causing a computer to execute the above-mentioned ultrasonic image display method. In other words, the computer program is a program for causing a computer to realize the functions of an ultrasonic diagnostic device. The storage medium stores the computer program.
[0125] In addition, in the matters described in the first and second embodiments, when different types of ultrasound probes 100 are connected to the ultrasound diagnostic apparatus, individual learning and inference may be performed for each type of ultrasound probe 100 so as to be optimal. [Explanation of symbols]
[0126] 100 Ultrasonic transducer 104 Operation section 106 Display section 110 Transmitter / receiver 112 Signal Processing Unit 114 Ultrasound image generation unit 116 Judgment section 118 Control Unit 200 Learning Device 202 Teacher Data Generation Unit 204 Learning Department 206 Memory section 208 Reasoning section
Claims
1. a determination unit that determines whether or not the second ultrasonic diagnostic device is in a faulty state from the multiple types of data generated in the second ultrasonic diagnostic device by using multiple trained models trained according to the types of data, with each of the multiple types of data generated in the first ultrasonic diagnostic device being used as teacher data; A notification unit that notifies a result of the determination by the determination unit, the determination unit integrates the determination results for each type of data output using the multiple trained models according to a priority indicating a weight used to determine a fault state according to the type of data, and determines whether or not the second ultrasonic diagnostic device is in a fault state; the priority is set so that data generated in a second process upstream of the first process in the second ultrasound diagnostic device is given priority over data generated in a first process in the second ultrasound diagnostic device.
2. 2. The ultrasonic diagnostic apparatus fault determination device according to claim 1, further comprising a learning device that learns each of the multiple types of data generated by the first ultrasonic diagnostic apparatus as teacher data and generates the multiple trained models.
3. 3. The fault determination device for an ultrasonic diagnostic device according to claim 2, wherein the learning device generates the trained model by using a neural network to learn by associating the data generated by the first ultrasonic diagnostic device as teacher data.
4. The failure determination device for an ultrasonic diagnostic device according to claim 2 , further comprising a storage unit that stores a trained model generated by the learning device.
5. 3. The failure determination device for an ultrasonic diagnostic device according to claim 2, wherein the learning device is installed outside the first ultrasonic diagnostic device.
6. 6. The failure determination device for an ultrasonic diagnostic device according to claim 5, wherein the learning device learns the data generated in a plurality of first ultrasonic diagnostic devices in a failure state as training data.
7. 3. The fault determination device for an ultrasonic diagnostic device according to claim 2, wherein the learning device generates a trained model according to each of the types of data generated by the first ultrasonic diagnostic device.
8. 2. The failure determination device for an ultrasonic diagnostic device according to claim 1, wherein the data generated by the second ultrasonic diagnostic device is at least one of a reflected wave signal, signal processing data, and ultrasonic image data.
9. 2. The failure determination device for an ultrasonic diagnostic apparatus according to claim 1, wherein the priority is set by an operator.
10. 2. The failure determination device for an ultrasonic diagnostic apparatus according to claim 1, wherein the priority is weighted according to the processing content of the data.
11. 2. The failure determination device for an ultrasonic diagnostic device according to claim 1, wherein the priority is weighted in the order of reflected wave signal, signal processing data, and ultrasonic image data.
12. 2. The failure determination device for ultrasonic diagnostic devices according to claim 1, further comprising a control unit that performs processing for correcting data on the failure state of the second ultrasonic diagnostic device when the second ultrasonic diagnostic device is in a failure state.
13. The fault determination device for an ultrasound diagnostic device according to claim 1, wherein the plurality of trained models include a first trained model trained using a reflected wave signal generated in the first ultrasound diagnostic device as teacher data, a second trained model trained using signal processing data generated in the first ultrasound diagnostic device as teacher data, and a third trained model trained using ultrasound image data generated in the first ultrasound diagnostic device as teacher data, and the priority is set in the order of the reflected wave signal, the signal processing data, and the ultrasound image data.
14. a step of determining whether or not the second ultrasonic diagnostic device is in a faulty state from the multiple types of data generated in the second ultrasonic diagnostic device using multiple trained models trained according to the types of data, with each of the multiple types of data generated in the first ultrasonic diagnostic device being used as teacher data; In accordance with a priority indicating a weight used for determining a fault state according to the type of data, the determination results for each type of data output using the multiple trained models are integrated, and the second ultrasonic diagnostic device is determined to be in a fault state or not; The fault determination method, wherein the priority is set so as to give priority to data generated in a second process upstream of the first process in the second ultrasonic diagnostic device over data generated in a first process in the second ultrasonic diagnostic device.
15. A program for causing a computer to execute the failure determination method according to claim 14.
Citation Information
Patent Citations
Fault diagnosis system based on circuit breaker operating mechanism
CN104267336A
Ultrasonic diagnosis device and self-diagnosis method therefor
JP1998262967A
Ultrasonic diagnostic apparatus
JP2008289697A
Medical diagnostic device adaptable to earthquake
JP2009172238A
Ultrasonic diagnostic apparatus
JP2010172411A
