Ultrasonic diagnostic device and image processing device
By detecting bounding boxes in ultrasonic images to infer measurement points, the problem of inaccurate measurement under the influence of brightness is solved, and the accuracy of ultrasonic images is improved.
Patent Information
- Application Number
- CN202110576727.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-05-28
- Filing Date
- 2021-05-26
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-05-26
AI Technical Summary
In ultrasonic images, brightness-based measurement methods are easily affected by high brightness areas, resulting in inaccurate inference of measurement locations, which in turn affects the accurate measurement of fetal parameters.
The detection unit detects the first rectangle covering the measurement part from the ultrasonic image, and the inference unit infers the measurement point based on the rectangular position information, and outputs relevant information through the output unit to improve the measurement accuracy.
Through the use of bounding boxes, interference from high-brightness areas is avoided, the accuracy of ultrasonic image measurement is improved, and the accurate measurement of fetal parameters is ensured.
Smart Images

Figure CN113729777B_ABST
Abstract
Description
[0001] Reference to related applications
[0002] This application claims the benefit of priority from Japanese Patent Application No. 2020-93551 filed on May 28, 2020, the entire contents of which are hereby cited. Technical Field
[0003] The embodiments disclosed in this specification and the drawings relate to an ultrasonic diagnostic apparatus and an image processing apparatus. Background Art
[0004] Ultrasonic images are used to confirm the development of the fetus. For example, ultrasonic diagnostic equipment can measure parameters such as the biparietal diameter (BPD), head circumference (HC), abdominal circumference (AC), femur length (FL), and humerus length (HL) of the fetus by using ultrasonic images.
[0005] Here, when measuring parameters such as BPD, HC, AC, FL, and HL, there is a known method of inferring a measurement point for measuring a measurement site by image processing based on brightness. However, in this method, if there is a portion with a higher brightness than the contour of the actual measurement site in the ultrasonic image, the measurement point may be inferred under the influence of the portion with a higher brightness. Summary of the invention
[0006] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to improve the accuracy of measurement using ultrasonic images. However, the problems solved by the embodiments disclosed in this specification and the drawings are not limited to the above-mentioned problems. Problems corresponding to the effects brought about by the various structures shown in the embodiments described later can also be positioned as other problems.
[0007] The ultrasonic diagnostic apparatus of this embodiment includes a detection unit, an inference unit, and an output unit. The detection unit detects a first rectangle that encloses a measurement site from an ultrasonic image. The inference unit infers a measurement point for measuring the measurement site based on position information of the first rectangle. The output unit outputs information related to the measurement point.
[0008] Effects of the Invention
[0009] According to the ultrasonic diagnostic apparatus of the embodiment, the accuracy of measurement using ultrasonic images is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] Figure 1 This is a block diagram showing an example of the configuration of the ultrasonic diagnostic apparatus according to the present embodiment.
[0011] Figure 2 This is a diagram for explaining problems when measuring a measurement site.
[0012] Figure 3 This is a diagram for explaining processing by the image processing circuit of the ultrasonic diagnostic apparatus according to the present embodiment.
[0013] Figure 4 This is a diagram for explaining processing by the image processing circuit of the ultrasonic diagnostic apparatus according to the present embodiment.
[0014] Figure 5 This is a diagram for explaining processing by the image processing circuit of the ultrasonic diagnostic apparatus according to the present embodiment.
[0015] Figure 6 This is a diagram for explaining processing by the image processing circuit of the ultrasonic diagnostic apparatus according to the present embodiment.
[0016] Figure 7 This is a diagram for explaining processing by the image processing circuit of the ultrasonic diagnostic apparatus according to the present embodiment.
[0017] Figure 8 This is a flowchart showing the processing procedure of the ultrasonic diagnostic apparatus according to the present embodiment.
[0018] Fig. 9 This is a diagram for explaining processing by the image processing circuit of the ultrasonic diagnostic apparatus according to the present embodiment.
[0019] Fig.10 This is a diagram for explaining processing by the image processing circuit of the ultrasonic diagnostic apparatus according to the present embodiment.
[0020] Fig.11 This is a diagram for explaining processing by the image processing circuit of the ultrasonic diagnostic apparatus according to the present embodiment.
[0021] Fig.12 This is a diagram for explaining processing by the image processing circuit of the ultrasonic diagnostic apparatus according to the present embodiment.
[0022] Fig.13 It is a block diagram showing a configuration example of a learned model generating device in this embodiment. DETAILED DESCRIPTION
[0023] Hereinafter, the ultrasonic diagnostic apparatus and the image processing apparatus according to the embodiment will be described with reference to the attached drawings. In addition, the embodiment is not limited to the following embodiment. In addition, the content described in one embodiment is also applicable to other embodiments in principle.
[0024] Figure 1 FIG. 1 is a block diagram showing a configuration example of the ultrasonic diagnostic apparatus 1 according to the present embodiment. Figure 1 As shown, the ultrasonic diagnostic apparatus 1 of this embodiment includes an apparatus body 100, an ultrasonic probe 101, an input device 102, and a display 103. The ultrasonic probe 101, the input device 102, and the display 103 are connected to the apparatus body 100, respectively.
[0025] The ultrasonic probe 101 performs transmission and reception of ultrasonic waves (ultrasonic scanning). For example, the ultrasonic probe 101 contacts the body surface of the subject P (for example, the abdomen of a pregnant woman), and performs transmission and reception of ultrasonic waves for an area in the uterus of the pregnant woman that includes at least a part of the fetus. The ultrasonic probe 101 has a plurality of piezoelectric vibrators. The plurality of piezoelectric vibrators are piezoelectric elements that have a piezoelectric effect that converts electrical signals (pulse voltage) and mechanical vibrations (vibrations based on sound) into each other, and generate ultrasonic waves based on a driving signal (electrical signal) supplied from the device body 100. The generated ultrasonic waves are reflected at the mismatching surface of the acoustic impedance in the subject P, and are received by the plurality of piezoelectric vibrators as reflected wave signals (electrical signals) including components scattered by scatterers in the tissue. The ultrasonic probe 101 sends the reflected wave signals received by the plurality of piezoelectric vibrators to the device body 100.
[0026] In addition, in this embodiment, the ultrasonic probe 101 can also use any type of ultrasonic probe, such as a 1D array probe having multiple piezoelectric vibrators arranged in one dimension in a specified direction, a 2D array probe having multiple piezoelectric vibrators arranged two-dimensionally in a grid pattern, or a mechanical 4D probe that scans a three-dimensional area by mechanically swinging multiple piezoelectric vibrators arranged in one dimension.
[0027] The input device 102 includes a mouse, a keyboard, a button, a panel switch, a touch command screen, a foot switch, a wheel switch, a trackball, a joystick, etc., and accepts various setting requests from the operator of the ultrasonic diagnostic device 1 and transmits the accepted various setting requests to the device body 100.
[0028] The display 103 displays a GUI (Graphical User Interface) used by the operator of the ultrasonic diagnostic apparatus 1 to input various setting requests using the input device 102, or displays ultrasonic image data (data) generated in the apparatus body 100. In addition, the display 103 displays various messages (messages) to notify the operator of the processing status of the apparatus body 100. In addition, the display 103 is an example of a "display unit".
[0029] The device body 100 is a device that generates ultrasonic image data based on the reflected wave signal received by the ultrasonic probe 101. The ultrasonic image data generated by the device body 100 may be two-dimensional ultrasonic image data generated based on the two-dimensional reflected wave signal, or may be three-dimensional ultrasonic image data generated based on the three-dimensional reflected wave signal.
[0030] like Figure 1 As shown, the device body 100 includes, for example, a transceiver circuit 110, a B-mode processing circuit 120, a Doppler processing circuit 130, an image processing circuit 140, an image memory 150, a storage circuit 160, and a control circuit 170. The transceiver circuit 110, the B-mode processing circuit 120, the Doppler processing circuit 130, the image processing circuit 140, the image memory 150, the storage circuit 160, and the control circuit 170 are connected to each other in a communicable manner. In addition, the device body 100 is connected to a network 2 within the hospital.
[0031] The transceiver circuit 110 controls the transmission of ultrasonic waves by the ultrasonic probe 101. For example, the transceiver circuit 110 applies the above-mentioned drive signal (drive pulse) to the ultrasonic probe 101 at a timing (timing) given a predetermined transmission delay time for each transducer based on the instruction of the control circuit 170. As a result, the transceiver circuit 110 causes the ultrasonic waves to be bundled into a beam-shaped ultrasonic beam and transmitted to the ultrasonic probe 101.
[0032] In addition, the transceiver circuit 110 controls the reception of the reflected wave signal of the ultrasonic probe 101. As described above, the reflected wave signal is a signal obtained after the ultrasonic wave transmitted from the ultrasonic probe 101 is reflected by the internal tissue of the subject P. For example, based on the instruction of the control circuit 170, the transceiver circuit 110 adds a predetermined delay time to the reflected wave signal received by the ultrasonic probe 101 and performs addition processing. As a result, the reflection component from the direction corresponding to the reception directivity of the reflected wave signal is emphasized. Then, the transceiver circuit 110 converts the reflected wave signal after the addition processing into an in-phase signal (I signal, I: In-phase) and a quadrature signal (Q signal, Q: Quadrature-phase) of the baseband frequency band. Then, the transceiver circuit 110 sends the I signal and the Q signal (hereinafter referred to as IQ signal) as reflected wave data to the B-mode processing circuit 120 and the Doppler processing circuit 130. Furthermore, the transmission and reception circuit 110 may convert the reflected wave signal after the addition process into an RF (Radio Frequency) signal and transmit it to the B-mode processing circuit 120 and the Doppler processing circuit 130. The IQ signal and the RF signal become signals (reflected wave data) including phase information.
[0033] The B-mode processing circuit 120 performs various signal processing on the reflected wave data generated by the transceiver circuit 110 based on the reflected wave signal. The B-mode processing circuit 120 performs logarithmic amplification, envelope detection processing, etc. on the reflected wave data received from the transceiver circuit 110, and generates data (B-mode data) that expresses the signal strength of each sample point (observation point) in terms of brightness. The B-mode processing circuit 120 sends the generated B-mode data to the image processing circuit 140.
[0034] In addition, the B-mode processing circuit 120 performs signal processing for harmonic imaging (harmonic imaging) for imaging high-order harmonic components. As harmonic imaging, contrast harmonic imaging (CHI: Contrast Harmonic Imaging) and tissue harmonic imaging (THI: Tissue Harmonic Imaging) are known. In contrast harmonic imaging and tissue harmonic imaging, as scanning methods, amplitude modulation (AM: Amplitude Modulation), phase modulation (PM: Phase Modulation) called "pulse subtraction method" or "pulse inversion method", and AMPM that combines AM and PM to obtain both the effect of AM and the effect of PM are known.
[0035] The Doppler processing circuit 130 generates data obtained by extracting motion information based on the Doppler effect of the moving body at each sampling point in the scanning area according to the reflected wave data generated by the transceiver circuit 110 based on the reflected wave signal, as Doppler data. Here, the motion information of the moving body is information such as the average speed, variance value, and energy value of the moving body. The moving body is, for example, blood flow, tissue such as the heart wall, and contrast agent. The Doppler processing circuit 130 sends the generated Doppler data to the image processing circuit 140.
[0036] For example, when the moving body is a blood flow, the motion information of the blood flow is information such as the average velocity, variance value, and energy of the blood flow (blood flow information). The blood flow information is obtained by, for example, a color Doppler method.
[0037] In the color Doppler method, first, multiple ultrasonic waves are transmitted and received on the same scanning line. Next, an MTI (Moving Target Indicator) filter is used to pass signals of a specific frequency band for the data column representing the reflected wave data at the same position (the same sampling point), and attenuate signals of other frequency bands. That is, the signal (clutter component) from stationary tissue or slow-moving tissue is suppressed. Thus, a blood flow signal related to blood flow is extracted from the signal representing the data column of the reflected wave data. Moreover, in the color Doppler method, blood flow information such as the average velocity, variance value, and energy of the blood flow is inferred based on the extracted blood flow signal, and the inferred blood flow information is generated as Doppler data.
[0038] The image processing circuit 140 performs image data (ultrasonic image data) generation processing, various image processing on the image data, etc. For example, the image processing circuit 140 generates two-dimensional B-mode image data that represents the intensity of the reflected wave by brightness based on the two-dimensional B-mode data generated by the B-mode processing circuit 120. In addition, the image processing circuit 140 generates two-dimensional Doppler image data in which blood flow information is visualized based on the two-dimensional Doppler data generated by the Doppler processing circuit 130. The two-dimensional Doppler image data is velocity image data representing the average velocity of the blood flow, variance image data representing the variance value of the blood flow, energy image data representing the energy of the blood flow, or image data combining them. The image processing circuit 140 generates color Doppler image data that displays blood flow information such as the average velocity, variance value, and energy of the blood flow in color, or generates Doppler image data that displays one blood flow information in grayscale as Doppler image data.
[0039] Here, the image processing circuit 140 generally converts (scan converts) the scan line signal sequence of the ultrasonic scan into the scan line signal sequence of the video format represented by television, etc., and generates ultrasonic image data for display. Specifically, the image processing circuit 140 performs coordinate conversion according to the ultrasonic scanning method of the ultrasonic probe 101, thereby generating ultrasonic image data for display. In addition, in addition to scan conversion, the image processing circuit 140 performs various image processing, such as image processing (smoothing processing) for regenerating an average image of brightness using a plurality of image frames after scan conversion and image processing (edge emphasis processing) using a differential filter in the image. In addition, the image processing circuit 140 synthesizes character information of various parameters, scales, body marks, etc. in the ultrasonic image data.
[0040] That is, the B-mode data and the Doppler data are ultrasonic image data before scan conversion processing, and the data generated by the image processing circuit 140 is ultrasonic image data for display after scan conversion processing. In addition, the B-mode data and the Doppler data are also called raw data. The image processing circuit 140 generates two-dimensional ultrasonic image data for display based on the two-dimensional ultrasonic image data before scan conversion processing.
[0041] The image processing circuit 140 generates three-dimensional B-mode image data by performing coordinate conversion on the three-dimensional B-mode data generated by the B-mode processing circuit 120. The image processing circuit 140 generates three-dimensional Doppler image data by performing coordinate conversion on the three-dimensional Doppler data generated by the Doppler processing circuit 130.
[0042] In addition, the image processing circuit 140 performs rendering processing on the volume image data in order to generate various two-dimensional image data for displaying the volume image data on the display 103. As a rendering processing performed by the image processing circuit 140, there is, for example, a processing of performing multi-planar reconstruction (MPR: Multi Planer Reconstruction) to generate MPR image data from the volume image data. In addition, as a rendering processing performed by the image processing circuit 140, there is, for example, a volume rendering (VR: Volume Rendering) processing for generating two-dimensional image data reflecting information of a three-dimensional image. In addition, as a rendering processing performed by the image processing circuit 140, there is, for example, a surface rendering (SR: Surface Rendering) processing for generating two-dimensional image data from which only surface information of a three-dimensional image is extracted.
[0043] The image processing circuit 140 stores the generated image data and the image data obtained after performing various image processing in the image memory 150. In addition, the image processing circuit 140 may also generate information indicating the display position of each image data, various information for assisting the operation of the ultrasonic diagnostic apparatus 1, and supplementary information related to the diagnosis such as patient information, together with the image data, and store them in the image memory 150.
[0044] In addition, the image processing circuit 140 of this embodiment executes an image generation function 141, a detection function 142, an inference function 143, and a display control function 144. In addition, the detection function 142 is an example of a detection unit. The inference function 143 is an example of an inference unit. The display control function 144 is an example of an output unit.
[0045] Here, as Figure 1 The image processing circuit 140 shown in the figure has the image generation function 141, the detection function 142, the inference function 143, and the display control function 144, each of which is executed by the components, and is recorded in the storage circuit 160 in the form of a program that can be executed by a computer. The image processing circuit 140 is a processor that reads out and executes each program from the storage circuit 160 to realize the function corresponding to each program. In other words, the image processing circuit 140 that has read out the status of each program will have Figure 1 The functions shown in the image processing circuit 140 are shown in FIG. The processing contents of the image generation function 141, the detection function 142, the estimation function 143, and the display control function 144 executed by the image processing circuit 140 will be described later.
[0046] In addition, Figure 1 In the figure, it is described that each processing function performed by the image generation function 141, the detection function 142, the inference function 143, and the display control function 144 is implemented in a single image processing circuit 140, but a plurality of independent processors can also be combined to form a processing circuit, and each processor executes a program to implement the function.
[0047] The term "processor" used in the above description refers to circuits such as CPU (Central Processing Unit), GPU (Graphics Processing Unit), Application Specific Integrated Circuit (ASIC)), programmable logic devices (such as Simple Programmable Logic Device (SPLD), Complex Programmable Logic Device (CPLD) and Field Programmable Gate Array (FPGA)). In the case where the processor is, for example, a CPU, the processor reads out and executes a program stored in the storage circuit 60 to implement its functions. On the other hand, in the case where the processor is, for example, an ASIC, instead of storing the program in the storage circuit 60, the program is directly programmed into the circuit of the processor. In addition, the processors of the present embodiment are not limited to the case where each processor is constructed as a single circuit, and multiple independent circuits may be combined to form a processor and implement its functions. Moreover, it is also possible to Figure 1 Multiple components in a processor are combined into one to realize its functions.
[0048] The image memory 150 and the storage circuit 160 are, for example, semiconductor storage elements such as RAM (Random Access Memory) and flash memory, or storage devices such as a hard disk and an optical disk.
[0049] The image memory 150 is a memory for storing image data such as B-mode image data and Doppler image data generated by the image processing circuit 140 as ultrasonic image data. In addition, the image memory 150 may also store image data such as B-mode data generated by the B-mode processing circuit 120 and Doppler data generated by the Doppler processing circuit 130 as ultrasonic image data. The ultrasonic image data stored in the image memory 150 may be called by an operator after diagnosis, for example, and become ultrasonic image data for display via the image processing circuit 140.
[0050] The storage circuit 160 stores various data such as control programs for ultrasonic transmission and reception, image processing, and display processing, diagnostic information (e.g., patient ID, doctor's findings, etc.), diagnostic protocols, and various body marks. In addition, the data stored in the storage circuit 160 can be transmitted to an external device via an interface unit (not shown). In addition, the external device is, for example, a PC (Personal Computer) used by a doctor performing image diagnosis, a storage medium such as a CD, a DVD, or a printer. In addition, if the ultrasonic diagnostic device 1 can be accessed on the network 2, the storage circuit 160 may not be built into the ultrasonic diagnostic device 1.
[0051] In addition, the storage circuit 160 stores a learned model 400. The learned model 400 is a model generated by learning using ultrasonic image data when ultrasonic scanning was performed in the past and learning data representing an area including a measurement site in the ultrasonic image data. Examples of such learning include learning using AI (Artificial Intelligence) based on a neural network and machine learning. Through such learning, the learned model 400 is given the following functions: receiving input of ultrasonic image data as input data, and outputting output data representing the result of detecting an area including a measurement site in the ultrasonic image data. Here, the measurement site includes the head, abdomen, thigh, upper arm, etc. of the fetus. The use and generation of the learned model 400 will be described later.
[0052] The control circuit 170 controls the overall processing of the ultrasonic diagnostic apparatus 1. Specifically, the control circuit 170 controls the processing of the transceiver circuit 110, the B-mode processing circuit 120, the Doppler processing circuit 130, the image processing circuit 140, etc. based on various setting requests input from the operator via the input device 102, various control programs and various data read from the storage circuit 160.
[0053] In addition, the transceiver circuit 110, B-mode processing circuit 120, Doppler processing circuit 130, image processing circuit 140 and control circuit 170 built into the device body 100 are sometimes composed of hardware (hardware) of a processor (CPU (Central Processing Unit), MPU (Micro-Processing Unit), integrated circuit, etc.), but sometimes are composed of programs that are modularized in software.
[0054] The ultrasonic diagnostic apparatus 1 thus constructed is used, for example, to confirm the development of a fetus in the uterus of a pregnant woman. The ultrasonic probe 101 performs ultrasonic transmission and reception (ultrasonic scanning) on a region including a part of the fetus (measurement site) in the uterus of a pregnant woman, and the image processing circuit 140 generates an ultrasonic image after imaging the region including the measurement site based on the scanning result. For example, in the ultrasonic diagnostic apparatus 1, instead of manual measurement using a caliper, the measurement sites such as the head and abdomen of the fetus are automatically detected from the ultrasonic image, and the measurement points for measuring parameters such as the biparietal diameter (BPD) of the fetal head, the fetal head circumference (HC), the abdominal circumference (AC), the femur length (FL), and the humerus length (HL) of the fetus are inferred. In addition, in the ultrasonic diagnostic apparatus 1, the estimated fetal weight (EFW) can be calculated by using the parameters measured using the above-mentioned measurement points.
[0055] The overall configuration of the ultrasonic diagnostic apparatus 1 according to the present embodiment has been described above. With this configuration, the ultrasonic diagnostic apparatus 1 according to the present embodiment can improve the accuracy of measurement using ultrasonic images.
[0056] For example, there is a known method of inferring the measurement points used to measure the measurement part by image processing based on brightness when measuring parameters such as BPD, HC, AC, FL, and HL. Specifically, when measuring parameters such as BPD, HC, and AC based on an ultrasonic image, an ellipse is used to approximate the contour of the measurement part such as the head and abdomen, and an ellipse-shaped high-brightness area is detected in the ultrasonic image based on the brightness value, thereby inferring the measurement points used to measure the measurement part. In this case, the measurement points are the end points of the major axis and the end points of the minor axis of the ellipse. Similarly, when measuring parameters such as FL and HL based on an ultrasonic image, a rod-shaped area equivalent to the measurement part such as the femur and humerus is detected in the ultrasonic image based on the brightness value, thereby inferring the measurement points used to measure the measurement part. In this case, the measurement points are the two end points of the rod-shaped area. However, when this method is applied, there are the following problems.
[0057] Figure 2 1 is a diagram for explaining a problem when measuring a measurement site. For example, when measuring HC based on an ultrasonic image, Figure 2 As shown in FIG. 1 , the ellipse HC100 obtained by approximating the contour of the measurement part using brightness detection is used to infer the measurement points P101 to P103 used to measure the measurement part. However, in the ultrasonic image, if there is a portion with a brightness higher than the brightness of the contour of the actual measurement part represented by the ellipse HC100, the measurement point may be inferred due to the influence of the portion with higher brightness. For example, Figure 2As shown in FIG. 1 , when there is a portion having a higher brightness than the brightness of the contour represented by the ellipse HC100 and the ellipse HC200 is detected so as to include the portion, a point P201 located outside the measurement portion is estimated as the measurement point. In this way, when the measurement portion is detected based on brightness, the correct measurement point may not be estimated based on the high brightness portion outside the measurement portion, so that parameters such as BPD, HC, AC, FL, and HL may not be measured with high accuracy.
[0058] Therefore, in the ultrasonic diagnostic apparatus 1 of this embodiment, the detection function 142 detects a first rectangle enclosing the measurement site from the ultrasonic image. The inference function 143 infers a measurement point for measuring the measurement site based on the position information of the first rectangle. The display control function 144 outputs information related to the measurement point.
[0059] Below, use Figure 3 to Figure 8 , each function of the image generation function 141, the detection function 142, the inference function 143, and the display control function 144 is explained.
[0060] Figure 3 to Figure 7 This is a diagram for explaining the processing of the image processing circuit 140 of the ultrasonic diagnostic apparatus 1 according to the present embodiment. Figure 3 to Figure 7 This is an explanatory diagram of a case where the ultrasonic diagnostic apparatus 1 uses an ultrasonic image to measure the abdominal circumference (AC) of a fetus. In this case, the measurement site is the abdomen of the fetus. Figure 8 1 is a flowchart showing the processing procedure of the ultrasonic diagnostic apparatus 1 according to the present embodiment. Figure 8 , a flowchart for explaining the overall operation (image processing method) of the ultrasonic diagnostic apparatus 1 is shown, and a description is given of which step of the flowchart each component corresponds to.
[0061] Figure 8 Step S101 is a step performed by the ultrasonic probe 101. In step S101, the ultrasonic probe 101 performs ultrasonic scanning. Specifically, the ultrasonic probe 101 contacts the body surface of the subject P (the abdomen of the pregnant woman), performs ultrasonic scanning on a region in the uterus of the pregnant woman that includes a part of the fetus (the measurement site), and collects reflected wave signals of the above region as a result of the ultrasonic scanning.
[0062] Figure 8 Step S102 is a step in which the image processing circuit 140 calls out the program corresponding to the image generation function 141 from the storage circuit 160 and executes the program. In step S102, the image generation function 141 generates an ultrasonic image.
[0063] Specifically, the image generation function 141 generates an image based on the reflected wave signal obtained by the ultrasonic probe 101. Figure 3 The ultrasonic image 200 is ultrasonic image data that images a region including a measurement site. Here, the image generation function 141 may generate the ultrasonic image 200 by using the B-mode data generated by the B-mode processing circuit 120 to generate the B-mode image data, or may generate the ultrasonic image 200 by using the ultrasonic image data stored in the image memory 150.
[0064] Figure 8 Step S103 is a step in which the image processing circuit 140 calls out the program corresponding to the detection function 142 from the storage circuit 160 and executes the program. In step S103, the detection function 142 detects a first bounding box that surrounds the measurement site from the ultrasonic image 200.
[0065] Specifically, first, the detection function 142 detects the measurement part from the ultrasonic image 200 using the learned model 400 read from the storage circuit 160. The learned model 400 receives the ultrasonic image data as input data, and outputs the result obtained after detecting the measurement part in the ultrasonic image data, that is, output data. For example, the learned model 400 outputs a rectangle that encloses an ellipse obtained by approximating the contour of the abdomen as the measurement part as output data. This rectangle is also called a bounding box (circumscribed rectangle). The detection function 142 detects the measurement part by inputting the ultrasonic image 200 into the learned model 400 as a bounding box that represents the position and size of the fetal abdomen whose contour is approximated by an ellipse and encloses the fetal abdomen. Figure 4 The first bounding box 210 is shown as the first bounding box 210. The first bounding box 210 is an example of a "first rectangle".
[0066] In addition, the detection function 142 detects a predetermined structure from the ultrasonic image 200 using the learned model 400. The learned model 400 receives the ultrasonic image data as input data, and outputs the result of detecting the predetermined structure in the ultrasonic image data, that is, output data. When the measurement site is the abdomen of the fetus, the predetermined structure is the spine of the fetus. For example, Figure 4 As shown, the detection function 142 inputs the ultrasonic image 200 to the learned model 400, detects the spine 202, and outputs it as output data.
[0067] Here, the inference function 143 infers a temporary measurement point for measuring the abdomen of the fetus based on the position information of the first bounding box 210. Figure 4As shown, the inference function 143 infers points 211 to 213 inscribed in the first bounding box 210 and the ellipse obtained by approximating the contour of the fetus' abdomen as temporary measurement points. The measurement points 211 to 213 become the endpoints of the major axis and the minor axis of the ellipse inscribed in the first bounding box 210, and become the midpoints of the long side and the midpoints of the short side of the first bounding box 210.
[0068] Figure 8 Step S104 is a step in which the image processing circuit 140 calls out and executes a program corresponding to the detection function 142 from the storage circuit 160. In step S104, the detection function 142 detects the orientation of the measurement site based on the position information of the first boundary frame 210 and the position information of a predetermined structure.
[0069] Specifically, first, the detection function 142 is as follows Figure 4 As shown, in the ultrasonic image 200, a line segment L210 connecting the center C210 of the first boundary frame 210 and the center of the fetal spine 202 as a predetermined structure is detected. Next, the detection function 142 detects the direction of the line segment L210 as the direction of the measurement site, that is, the fetal abdomen. Figure 4 In the example shown, the direction of the line segment L210 is shown to be inclined clockwise or counterclockwise by an angle θ based on the longitudinal direction or the lateral direction of the screen displayed by the ultrasonic image 200. The longitudinal direction and the lateral direction of the screen displayed by the ultrasonic image 200 correspond to the column direction and the row direction of the pixels of the ultrasonic image 200, respectively. Figure 4 As shown, the direction of the line segment L210 is inclined counterclockwise by an angle θ with respect to the vertical direction of the screen on which the ultrasonic image 200 is displayed.
[0070] Figure 8 Step S105 is a step in which the image processing circuit 140 calls out the program corresponding to the detection function 142 from the storage circuit 160 and executes it. In step S105, the detection function 142 performs processing to determine whether the measurement site detected in step S104 is tilted.
[0071] As described above, the direction of the line segment L210 detected by the detection function 142 can be regarded as the orientation of the abdomen of the fetus as the measurement site. For example, the abdomen depicted in the ultrasonic image 200 can be regarded as tilted counterclockwise by an angle θ based on the vertical direction of the screen displayed by the ultrasonic image 200. In this case, in the determination process of step S105, the detection function 142 determines that the measurement site detected in step S104 is tilted (step S105; yes).
[0072] Figure 8Step S106 is a step in which the image processing circuit 140 calls out the program corresponding to the detection function 142 from the storage circuit 160 and executes it. In step S106, the detection function 142 performs a rotation process on the ultrasonic image 200 based on the orientation of the measurement site detected in step S104. Specifically, the detection function 142 performs the rotation process so that the direction of the line segment L210 becomes the vertical direction or the horizontal direction of the screen on which the ultrasonic image 200 is displayed.
[0073] By performing the rotation processing, the front-back direction of the abdomen of the fetus as the measurement part becomes more likely to be the longitudinal direction or the transverse direction of the screen, and the left-right direction becomes the transverse direction or the longitudinal direction of the screen. That is, by performing the rotation processing, the major axis or the minor axis of the ellipse detected as the contour of the abdomen becomes more likely to be the longitudinal direction or the transverse direction of the screen displayed by the ultrasonic image 200. Moreover, by performing the rotation processing, the possibility that one side of the boundary box becomes the longitudinal direction or the transverse direction of the screen displayed by the ultrasonic image 200 becomes more likely.
[0074] For example Figure 5 As shown, the detection function 142 performs rotation processing so that the direction of the line segment L210 becomes the vertical direction of the screen displayed by the ultrasonic image 200. That is, the direction of the line segment L210 is tilted counterclockwise by an angle θ with respect to the vertical direction of the screen displayed by the ultrasonic image 200, and therefore the detection function 142 rotates the ultrasonic image 200 clockwise by the angle θ.
[0075] Figure 8 Step S107 is a step in which the image processing circuit 140 calls out the program corresponding to the detection function 142 from the storage circuit 160 and executes it. In step S107, the detection function 142 detects a second bounding box that surrounds the measurement site from the ultrasonic image 200 after the rotation process executed in step S106.
[0076] Specifically, the detection function 142 detects the position and size of the fetal abdomen as a bounding box that represents the position and size of the fetal abdomen and covers the fetal abdomen by inputting the rotated ultrasonic image 200 into the learned model 400. Figure 5 In this case, the bounding box is changed from the first bounding box 210 to the second bounding box 220. For example, the second bounding box 220 is an example of a "second rectangle".
[0077] Figure 8 Step S108 is a step in which the image processing circuit 140 calls out and executes a program corresponding to the estimation function 143 from the storage circuit 160. In step S108, the estimation function 143 estimates the measurement points for measuring the abdomen of the fetus based on the position information of the second bounding box 220.
[0078] Specifically, if Figure 5 As shown, the inference function 143 infers the points 221 to 223 inscribed in the second bounding box 220 and the ellipse 224 obtained by approximating the contour of the fetal abdomen as measurement points. The points 221 to 223 become the endpoints of the major axis and the minor axis of the ellipse 224, and become the midpoints of the long side and the midpoints of the short side of the second bounding box 220. In this case, the inference function 143 updates the measurement points 211 to 213 based on the position information of the first bounding box 210 to the measurement points 221 to 223 based on the position information of the second bounding box 220 as measurement points for measuring the fetal abdomen.
[0079] In addition, in the determination process of step S105, when the direction of the line segment L210 detected by the detection function 142 is the vertical direction or the horizontal direction of the screen displayed by the ultrasonic image 200, the abdomen depicted in the ultrasonic image 200 can be regarded as not tilted. In this case, the detection function 142 determines that the measurement part detected in step S104 is not tilted (step S105; No), and does not execute steps S106 and S107, but executes step S108. That is, the detection function 142 does not perform the rotation process on the ultrasonic image 200, and in step S108, the inference function 143 sets the measurement points 211 to 213 based on the position information of the first bounding box 210 as the measurement points for measuring the abdomen of the fetus.
[0080] Figure 8 Step S109 is a step in which the image processing circuit 140 calls out a program corresponding to the display control function 144 from the storage circuit 160 and executes the program. In step S109, the display control function 144 outputs information related to the measurement points 221 to 223 on the screen.
[0081] For example, the display control function 144 causes the display 103 to display an image in which marks indicating the positions of the measurement points 221 to 223 are drawn on the ultrasonic image 200 before the rotation process as information related to the measurement points 221 to 223. Specifically, first, the display control function 144 controls the display 103 to display the image in which the marks indicating the positions of the measurement points 221 to 223 are drawn on the ultrasonic image 200 before the rotation process. Figure 5 As shown in FIG. 1 , marks indicating the positions of the measurement points 221 to 223 are drawn on the ultrasonic image 200 after the rotation processing. Here, the ultrasonic image 200 on which marks indicating the positions of the measurement points 221 to 223 are drawn is a Figure 4 The ultrasonic image 200 shown is an image after being rotated clockwise by an angle θ. Figure 6As shown in FIG. 1 , the display control function 144 rotates the ultrasonic image 200 on which the marks indicating the positions of the measurement points 221 to 223 are drawn counterclockwise by an angle θ. Figure 7 As shown, marks indicating the positions of measurement points 221 to 223 are drawn on the image of the ultrasonic image 200 before the rotation process is performed on the display 103. In addition, the display 103 displays an ellipse 224 in which the measurement points 221 to 223 are the endpoints of the major axis and the minor axis, superimposed on the image.
[0082] Here, the display control function 144 may also display the information related to the measurement points 221 to 223. Figure 7 When the image shown is displayed on the display 103, the operator is notified that the measuring points 221 to 223 are estimated. For example, a lamp such as an LED (Light Emitting Diode) may be provided on the input device 102 or the display 103, and the display control function 144 may light the lamp to notify the operator that the measuring points 221 to 223 are estimated, and then Figure 7 The image shown is displayed on the display 103. For example, the display control function 144 may display the message that the measurement points 221 to 223 are estimated on the display 103 through a message, and then Figure 7 The image shown is displayed on the display 103. For example, the display 103 may have a speaker (not shown), and the display control function 144 may output a predetermined sound such as a beep, and then Figure 7 The image shown is displayed on the display 103 .
[0083] in addition, Figure 7 The mark displayed on the image shown can be changed by the operator. For example, the operator can move the mark displayed on the image by operating the input device 102.
[0084] Thus, the estimation function 143 can measure the abdominal circumference (AC) of the fetus by using the ultrasonic image 200 and the measurement points 221 to 223. For example, the estimation function 143 measures the circumference of the ellipse 224 formed by the measurement points 221 to 223 in the ultrasonic image 200 as AC.
[0085] Thus, in the ultrasonic diagnostic apparatus 1 of the present embodiment, the detection function 142 detects the first bounding frame 210 that surrounds the measurement site from the ultrasonic image 200, and the estimation function 143 estimates the measurement points for measuring the measurement site based on the position information of the first bounding frame 210. Specifically, the detection function 142 detects a predetermined structure from the ultrasonic image 200 together with the first bounding frame 210. Next, the detection function 142 performs rotation processing on the ultrasonic image 200 based on the position information of the predetermined structure and the position information of the first bounding frame 210, and detects the second bounding frame 220 that surrounds the measurement site from the rotated ultrasonic image 200. Next, the estimation function 143 updates the measurement points 211 to 213 based on the position information of the first bounding frame 210 to the measurement points 221 to 223 based on the position information of the second bounding frame 220 as the measurement points for measuring the measurement site. Then, the display control function 144 outputs information related to the measurement points 221 to 223.
[0086] Thus, in the ultrasonic diagnostic apparatus 1 of the present embodiment, the measurement point is estimated by detecting a bounding box that surrounds the entire measurement part instead of detecting an ellipse that matches the contour of the measurement part based on the high-brightness area, thereby improving the accuracy of measurement using the ultrasonic image 200. In addition, in the ultrasonic diagnostic apparatus 1 of the present embodiment, the measurement point is estimated based on the entire measurement part using the bounding box, so that the measurement point can be estimated efficiently compared to estimating measurement points that are difficult to feature one by one.
[0087] In addition, although not limited to detection using the learned model 400, in part detection, when the posture of the part to be detected is not tilted, the part can be detected at a more accurate position. In this embodiment, the measurement part is detected using a bounding box, and a predetermined structure that can infer the posture of the measurement part is detected, and the image of the detection object is rotated so that one side of the bounding box that surrounds the measurement part becomes vertical or horizontal. In addition, in this embodiment, the bounding box detection process is performed again based on the image after the rotation process, so that the part can be detected at a more accurate position, and as a result, the inference accuracy of the measurement point can also be improved.
[0088] In the above-mentioned embodiment, the case of measuring the abdominal circumference (AC) of the fetus is described, but it is not limited to this. This embodiment can also be applied when measuring parameters such as the biparietal diameter (BPD) of the fetus, the fetal head circumference (HC), the femur length (FL), and the humerus length (HL).
[0089] First, the case of measuring parameters such as BPD and HC will be described. Fig. 9It is a diagram for explaining the processing of the image processing circuit of the ultrasonic diagnostic apparatus according to the present embodiment and is an explanatory diagram when measuring parameters such as BPD and HC.
[0090] For example, after executing steps S101 and S102, in step S103, the detection function 142 detects a first bounding box that encloses the measurement site from the ultrasonic image. Specifically, the detection function 142 inputs the ultrasonic image 500 to the learned model 400, and detects a first bounding box that encloses the fetal head and represents the position and size of the fetal head whose outline is approximated by an ellipse. Fig. 9 In addition, the detection function 142 detects a predetermined structure by inputting the ultrasonic image 500 to the learned model 400. For example, when the measurement part is the fetal head, the predetermined structure is the fetal cavum pellucidum 502. Here, the inference function 143 infers a temporary measurement point for measuring the fetal head based on the position information of the first bounding box 510. For example, Fig. 9 As shown, the inference function 143 infers points 511 to 514 inscribed in the first bounding box 510 and the ellipse obtained by approximating the contour of the fetal head as temporary measurement points.
[0091] In step S104, the detection function 142 detects the orientation of the measurement part based on the position information of the first boundary frame 510 and the position information of the predetermined structure. Fig. 9 As shown, a line segment L510 connecting the center C510 of the first boundary frame 510 and the central part of the fetal cavum pellucida 502 as a predetermined structure is detected in the ultrasonic image 500. Next, the detection function 142 detects the direction of the line segment L510 as the orientation of the measurement part, that is, the head of the fetus. For example, when the longitudinal direction or the transverse direction of the screen displayed by the ultrasonic image 500 is used as a reference, the direction of the line segment L510 becomes the transverse direction of the screen displayed by the ultrasonic image 500.
[0092] In step S105 , the detection function 142 performs processing for determining whether the measurement site detected in step S104 is inclined.
[0093] In the determination process of step S105, when the direction of the line segment L510 detected by the detection function 142 is the vertical direction or the horizontal direction of the screen displayed by the ultrasonic image 500, the head depicted in the ultrasonic image 500 can be regarded as not tilted. In this case, the detection function 142 determines that the measurement site detected in step S104 is not tilted (step S105; No), and does not perform the processes of steps S106 and S107, but performs the process of step S108. That is, the detection function 142 does not perform the rotation process on the ultrasonic image 500, but in step S108, the inference function 143 sets the measurement points 511 to 514 based on the position information of the first bounding box 310 as the measurement points for measuring the head of the fetus.
[0094] In step S109 , the display control function 144 causes the display 103 to display an image in which marks indicating the positions of the measurement points 511 to 514 are drawn on the ultrasonic image 500 as information on the measurement points 511 to 514 .
[0095] Thus, the estimation function 143 can measure the HC by using the ultrasonic image 500 and the measurement points 511 to 514. For example, the estimation function 143 measures the circumference of an ellipse formed by the measurement points 511 to 514 in the ultrasonic image 500 as the HC.
[0096] Thus, the estimation function 143 can measure the HC by using the ultrasonic image 500 and the measurement points 511 to 514. For example, the estimation function 143 measures the circumference of an ellipse formed by the measurement points 511 to 514 in the ultrasonic image 500 as the HC.
[0097] In addition, the estimation function 143 can measure the BPD by using the ultrasonic image 500 and the measurement points 513 and 514. For example, the estimation function 143 measures the length of the line segment connecting the measurement point 513 and the measurement point 514 in the ultrasonic image 500 as the BPD.
[0098] In addition, in the determination process of step S105 , when the detection function 142 determines that the measurement site detected in step S104 is inclined (step S105 ; Yes), the following processes of steps S106 to S109 are executed.
[0099] First, in step S106, the detection function 142 performs a rotation process on the ultrasonic image 500 based on the orientation of the measurement site detected in step S104. Specifically, the detection function 142 performs a rotation process in such a way that the direction of the line segment L510 becomes the longitudinal direction or the transverse direction of the screen displayed by the ultrasonic image 500. Next, in step S107, the detection function 142 detects a second bounding box that encompasses the measurement site from the ultrasonic image 500 after the rotation process performed in step S106. Specifically, the detection function 142 detects the second bounding box as a bounding box that represents the position and size of the fetal head whose outline is approximated by an ellipse and encompasses the fetal head by inputting the rotated ultrasonic image 500 into the learned model 400.
[0100] Next, in step S108, the inference function 143 infers the measurement points for measuring the head of the fetus based on the position information of the second bounding box. Specifically, the inference function 143 infers the points where the ellipse obtained by approximating the contour of the head of the fetus and the second bounding box are inscribed as measurement points. In this case, the inference function 143 updates the measurement points 511 to 514 based on the position information of the first bounding box 510 as measurement points for measuring the head of the fetus to measurement points based on the position information of the second bounding box. Then, in step S109, the display control function 144 causes the mark indicating the position of the updated measurement point to be drawn on the image of the ultrasonic image 500 as information related to the measurement point and is displayed on the display 103.
[0101] In step S103, when the measurement site is the head of the fetus, the predetermined structure is the cavum septum pellucidum 502 of the fetus, but the present invention is not limited thereto.
[0102] For example Fig.10 As shown in FIG. 1 , when the measurement site is the head of a fetus, the predetermined structure may also be the quadrigeminal cistern 503 of the fetus. In this case, in step S104, first, the detection function 142 is as follows: Fig.10As shown, a line segment L510 connecting the center C510 of the first bounding box 510 and the central part of the quadrigeminal cistern 503 of the fetus as a predetermined structure is detected in the ultrasonic image 500. Next, the detection function 142 detects the direction of the line segment L510 as the orientation of the measurement site, that is, the head of the fetus. After that, the processing after step S105 is performed. For example, when the measurement site is not tilted (step S105; No), the ultrasonic image 500 is not rotated, and the measurement points 511 to 514 are estimated based on the position information of the first bounding box 310 (step S108), and the information related to the measurement points 511 to 514 is displayed on the display 103 (step S109). On the other hand, if the measurement site is tilted (step S105; Yes), the ultrasonic image 500 is rotated (step S106), and the second bounding box covering the measurement site is detected from the rotated ultrasonic image 500 (step S107). Then, the measurement point is inferred based on the position information of the second boundary box (step S108), and information related to the measurement point is displayed on the display 103 (step S109). Fig.10 In the example shown, even when the predetermined structure is the quadrigeminal cistern 503 of the fetus, the orientation of the head of the fetus can be detected, similar to the case where the predetermined structure is the cavum septum pellucidum 502 .
[0103] In addition, for example Fig.11 As shown in FIG. 1 , when the measurement site is the head of a fetus, the predetermined structure may also be the cavum septum pellucidum 502 and the cistern of the quadrigeminal body 503 of the fetus. In this case, in step S104, first, the detection function 142 is as follows: Fig.11 As shown, a line segment L510 connecting the center C510 of the first bounding box 510 and the central parts of the fetal cavum septum pellucidum 502 and the cistern quadrigeminalis 503 as the predetermined structures is detected in the ultrasonic image 500. Next, the detection function 142 detects the direction of the line segment L510 as the orientation of the measurement site, that is, the head of the fetus. Thereafter, the processing after step S105 is performed. For example, when the measurement site is not tilted (step S105; No), the ultrasonic image 500 is not rotated, and the measurement points 511 to 514 are estimated based on the position information of the first bounding box 310 (step S108), and the information related to the measurement points 511 to 514 is displayed on the display 103 (step S109). On the other hand, if the measurement site is tilted (step S105; Yes), the ultrasonic image 500 is rotated (step S106), and the second bounding box covering the measurement site is detected from the rotated ultrasonic image 500 (step S107). Then, the measurement point is inferred based on the position information of the second boundary box (step S108), and information related to the measurement point is displayed on the display 103 (step S109). Fig.11 In the example shown, the predetermined structures are the cavum septum pellucidum 502 and the cistern quadrigeminal 503 of the fetus. Therefore, the orientation of the fetal head can be detected more accurately than when the predetermined structure is only one of the cavum septum pellucidum 502 and the cistern quadrigeminal 503 .
[0104] Next, the case of measuring FL is described. Fig.12 It is a diagram for explaining the processing of the image processing circuit of the ultrasonic diagnostic apparatus according to the present embodiment and is an explanatory diagram when FL is measured.
[0105] For example, after executing steps S101 and S102, in step S103, the detection function 142 detects a first bounding box that encloses the measurement site from the ultrasonic image. Specifically, after the ultrasonic image data is input, the learned model 400 outputs a bounding box that circumscribes the rod-shaped region corresponding to the femur as the measurement site as output data. The detection function 142 detects a first bounding box that encloses the fetus' thigh as a bounding box that represents the position and size of the fetus' thigh that is similar to the rod-shaped region by inputting the ultrasonic image 600 into the learned model 400. Fig.12 Here, the detection function 142 does not execute steps S104 to S107 when the measurement site is the femur or the humerus.
[0106] In step S108, the inference function 143 infers the measurement point for measuring the thigh of the fetus based on the position information of the first bounding box 610. Fig.12 As shown, the estimation function 143 estimates as measurement points points 611 and 612 that are inscribed in the first bounding box 610 and the stick-shaped region corresponding to the thigh of the fetus. The measurement points 611 and 612 correspond to both ends of the femur.
[0107] In step S109 , the display control function 144 causes the display 103 to display an image in which marks indicating the positions of the measurement points 611 , 612 are drawn on the ultrasonic image 600 as information on the measurement points 611 , 612 .
[0108] Thus, the estimation function 143 can measure FL by using the ultrasonic image 600 and the measurement points 611 and 612. For example, the estimation function 143 measures the length of the line segment connecting the measurement point 611 and the measurement point 612 in the ultrasonic image 600 as FL.
[0109] When measuring HL, for example, steps S101 to S103 , S108 , and S109 are similarly executed, whereby the estimation function 143 measures the length of the line segment connecting the two measurement points in the ultrasonic image as HL.
[0110] Here, the process of generating the learned model 400 by machine learning will be described.
[0111] The learned model 400 is generated, for example, by another device (hereinafter referred to as a learned model generation device) different from the ultrasonic diagnostic device 1, and is stored in the ultrasonic diagnostic device 1 (for example, stored in the storage circuit 160). Here, the learned model generation device can be implemented by the ultrasonic diagnostic device 1, and for example, the learned model 400 can also be generated by the ultrasonic diagnostic device 1 (for example, the inference function 143) and stored in its storage circuit 160. The following description takes the case where the learned model 400 is generated by the learned model generation device and stored in the ultrasonic diagnostic device 1 as an example.
[0112] Fig.13 is a block diagram showing an example of the configuration of the learned model generation device 300 in this embodiment. Fig.13 As shown, the learned model generation device 300 includes an input device 302, a display 303, an image processing circuit 340, and a storage circuit 360. The learned model generation device 300 is used as a browser for generating the learned model 400.
[0113] The input device 302 includes a mouse, keyboard, buttons, panel switches, touch command panel, foot switch, scroll wheel, track ball, joystick, etc., and receives various setting requests from the operator of the learned model generation device 300, and transmits the received various setting requests to the image processing circuit 340. The display 303 is a monitor that is referenced by the operator of the learned model generation device 300.
[0114] The image processing circuit 340 controls the overall processing of the learned model generation device 300. For example, the image processing circuit 340 Fig.13 As shown, a learning data acquisition function 341 and a learned model generation function 342 are executed.
[0115] The storage circuit 360 is, for example, a semiconductor storage element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. Here, each processing function executed by the learning data acquisition function 341 and the learning model generation function 342 as components of the image processing circuit 340 is recorded in the storage circuit 360 in the form of a program that can be executed by a computer. The image processing circuit 340 is a processor that reads each program from the storage circuit 360 and executes it to realize the function corresponding to each program. In addition, the storage circuit 360 stores a learning program as an algorithm of a neural network.
[0116] In the process of generating the learned model 400, first, the learning data acquisition function 341 of the learned model generation device 300 acquires a plurality of data sets obtained when ultrasonic scanning was performed in the past. The plurality of data sets respectively include ultrasonic image data collected when ultrasonic scanning of the measurement site was performed in the past, and learning data representing the region including the measurement site in the ultrasonic image data. In addition, since the algorithm of the neural network is an algorithm that gradually learns based on experience, the data set used for learning does not need to be the same subject.
[0117] The learned model generation function 342 of the learned model generation device 300 uses the learning program 502 read from the storage circuit 160 to learn the pattern of the learning data based on the plurality of data sets. At this time, the learned model generation function 342 generates a learned model 400 having the following functions: receiving input data (ultrasonic image data) and outputting output data indicating the result of detecting the area including the measurement part in the ultrasonic image data. For example, the generated learned model 400 is stored in the ultrasonic diagnostic device 1 (stored in the storage circuit 160). The learned model 400 in the storage circuit 160 can be read by the detection function 142 of the ultrasonic diagnostic device 1, for example.
[0118] In the process using the learned model 400, the detection function 142 of the ultrasonic diagnostic apparatus 1 inputs the ultrasonic image data as input data. Then, the detection function 142 detects the region including the measurement site from the ultrasonic image 200 as the ultrasonic image data using the learned model 400 read out from the storage circuit 160, and outputs output data as the detection result.
[0119] In addition, in this embodiment, the case where the learned model 400 is used is described, but the learned model 400 may not be used. For example, in this embodiment, after the measurement part such as the abdomen is detected by image processing using brightness, a bounding box that encloses the measurement part is detected. In this case, in this embodiment, the measurement part may not be detected as a figure suitable for the measurement part such as an ellipse, but as a part surrounded by an arbitrary curve. In addition, in this embodiment, a predetermined structure may also be detected by image processing using brightness instead of using the learned model 400.
[0120] (Other embodiments)
[0121] The embodiment is not limited to the above-mentioned embodiment. For example, the image processing circuit 140 may be a workstation provided separately from the ultrasonic diagnostic apparatus 1. In this case, the workstation has the same processing circuit as the image processing circuit 140 and executes the above-mentioned processing.
[0122] In addition, the components of each device illustrated in the embodiments are conceptual functions and do not necessarily need to be physically configured as shown. That is, the specific manner of dispersing and merging each device is not limited to the diagram, and all or part of it can be functionally or physically dispersed and merged in arbitrary units according to various loads or usage conditions. Moreover, all or any part of the processing functions performed by each device can be implemented by using a CPU and a program analyzed and executed by the CPU, or as hardware of wired logic.
[0123] In addition, the display method described in the above embodiment can be implemented by executing a pre-prepared image processing program by a computer such as a personal computer or a workstation. The image processing program can be distributed via a network such as the Internet. In addition, the image processing program can also be recorded on a non-temporary recording medium that can be read by a computer, such as a hard disk, a floppy disk (FD), a CD-ROM, an MO, or a DVD, and read out from the recording medium and executed by the computer.
[0124] According to at least one of the embodiments described above, it is possible to improve the accuracy of measurement using ultrasonic images.
[0125] In addition, although several embodiments are described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other ways, and various omissions, substitutions, changes, and combinations of embodiments can be performed without departing from the scope of the subject matter of the invention. These embodiments and their variations are included in the scope and subject matter of the invention, and are also included in the invention described in the claims and their equivalents.
Claims
1. An ultrasonic diagnostic apparatus, comprising: a detection unit, which detects a first rectangle covering a measurement site of a fetus included in the ultrasonic image, from an ultrasonic image, and detects a predetermined structure from the ultrasonic image, rotates the ultrasonic image based on position information of the first rectangle and position information of the predetermined structure, and detects a second rectangle covering the measurement site from the rotated ultrasonic image; an inference unit, which infers a plurality of measurement points for measuring the measurement site based on the position information of the second rectangle; and an output unit, which outputs information related to the plurality of measurement points. 2 . The ultrasonic diagnostic apparatus according to claim 1 , wherein the estimation unit estimates a temporary measurement point for measuring the measurement site based on the position information of the first rectangle, and updates the temporary measurement point to the plurality of measurement points based on the position information of the second rectangle. 3 . The ultrasonic diagnostic apparatus according to claim 2 , wherein the detection unit performs the rotation process so that a direction of a line segment connecting the predetermined structure and the center of the first rectangle is in a vertical direction or a horizontal direction of a screen displaying the ultrasonic image. 4 . The ultrasonic diagnostic apparatus according to claim 3 , wherein the detection unit does not perform the rotation processing when the direction of the line segment is in the vertical direction or the horizontal direction of the screen displaying the ultrasonic image.
5. The ultrasonic diagnostic apparatus according to any one of claims 2 to 4, wherein the detection unit performs the rotation processing in a manner such that the major axis or the minor axis of the ellipse is in the vertical direction or the horizontal direction of the screen displaying the ultrasonic image when the contour of the measurement site is approximated by an ellipse.
6. The ultrasonic diagnostic apparatus according to any one of claims 2 to 4, wherein the detection unit performs the rotation process so that one side of the rectangle is in the vertical direction or the horizontal direction of the screen displaying the ultrasonic image when the contour of the measurement site is approximated by a rectangle. 7 . The ultrasonic diagnostic apparatus according to claim 1 , wherein the output unit causes the display unit to display an image in which marks indicating positions of the plurality of measurement points are drawn on the ultrasonic image. 8 . The ultrasonic diagnostic apparatus according to claim 2 , wherein the output unit causes the display unit to display an image in which marks indicating the positions of the plurality of measurement points are drawn on the ultrasonic image before the rotation process. 9 . The ultrasonic diagnostic apparatus according to claim 2 , wherein when the measurement site is the abdomen of a fetus, the predetermined structure is a spine of the fetus. 10 . The ultrasonic diagnostic apparatus according to claim 2 , wherein when the measurement site is the head of a fetus, the predetermined structure is at least one of a cavum septum pellucidum and a cistern quadrigeminal of the fetus. 11 . The ultrasonic diagnostic apparatus according to claim 1 , wherein the estimation unit measures a circumference of an ellipse formed by the plurality of measurement points or a length of a line segment connecting the plurality of measurement points.
12. An image processing device, comprising: a detection unit, which detects a first rectangle that covers a measurement part of a fetus contained in the ultrasonic image from an ultrasonic image, and detects a specified structure from the ultrasonic image, rotates the ultrasonic image based on position information of the first rectangle and position information of the specified structure, and detects a second rectangle that covers the measurement part from the rotated ultrasonic image; an inference unit, which infers a plurality of measurement points for measuring the measurement part based on the position information of the second rectangle; and an output unit, which outputs information related to the plurality of measurement points.
