Ultrasonic diagnosis support apparatus, ultrasonic diagnostic apparatus, ultrasonic diagnosis support method, and program

Abstract

To provide an ultrasonic diagnosis support apparatus, an ultrasonic diagnostic apparatus, an ultrasonic diagnosis support method, and a program capable of easily specifying a probe scanning position for a subject, and improving objectivity and reproducibility of an ultrasonic diagnosis.SOLUTION: An ultrasonic diagnosis support apparatus includes: an ultrasonic image generation unit for generating an ultrasonic image on the basis of an ultrasonic signal from an ultrasonic probe for transmitting / receiving an ultrasonic wave; a camera image generation unit for generating a camera image on the basis of an imaging signal from a camera attached to the ultrasonic probe; and a diagnosis information managing unit for storing camera attaching information indicating a camera attaching mode for the ultrasonic probe in association with the ultrasonic image and the camera image.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an ultrasonic diagnosis support device, an ultrasonic diagnostic device, an ultrasonic diagnosis support method, and a program.

Background Art

[0002] Conventionally, as one of medical imaging diagnostic devices, an ultrasonic diagnostic device is known that transmits ultrasonic waves toward a subject, receives the reflected waves, and performs predetermined signal processing on the received signals to visualize the shape, properties, or dynamics inside the subject as an ultrasonic image. Since the ultrasonic diagnostic device can acquire an ultrasonic image by a simple operation of applying an ultrasonic probe to the body surface or inserting it into the body, it is safe and imposes a small burden on the subject.

[0003] The ultrasonic probe is configured independently of the diagnostic device main body and is communicably connected to the diagnostic device main body by wire or wirelessly. The user can freely scan the diagnostic target site by operating the ultrasonic probe. On the other hand, due to the high degree of freedom, when referring to the ultrasonic image after diagnosis or sharing it with others, it is difficult to know the probe scanning position, and it is difficult to ensure the reproducibility of the ultrasonic image. To address such drawbacks, an ultrasonic diagnostic device has been proposed that images the vicinity of the ultrasonic probe, specifically, the contact portion between the subject and the ultrasonic probe, with a camera and manages the ultrasonic image and the camera image in association with each other (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the ultrasonic diagnostic apparatus described in Patent Document 1, the camera image obtained by the probe viewpoint camera has a narrow imaging range and low bird's-eye viewability. Therefore, it is not possible to specify the probe scanning position from the camera image, and there is a risk that the diagnostic target site depicted in the ultrasonic image cannot be accurately grasped. For example, when landmarks that can specify the probe scanning position (for example, the boundary between the subject and the background or the characteristics of the body surface) are not included in the imaging range of the probe viewpoint camera, it is difficult to grasp the probe scanning position from the camera image.

[0006] Also, Patent Document 1 describes that the probe scanning position is specified by using a probe viewpoint camera and a panoramic camera in combination. However, since a panoramic camera is required, the cost increases. Moreover, it is necessary to ensure that there are no obstacles between the probe scanning position and the panoramic camera, which poses a problem in terms of convenience.

[0007] An object of the present disclosure is to provide an ultrasonic diagnostic support device, an ultrasonic diagnostic apparatus, an ultrasonic diagnostic support method, and a program that can easily specify the probe scanning position with respect to a subject and improve the objectivity and reproducibility of ultrasonic diagnosis.

Means for Solving the Problems

[0008] The ultrasonic diagnostic support device according to the present invention includes an ultrasonic image generation unit that generates an ultrasonic image based on an ultrasonic signal from an ultrasonic probe that transmits and receives ultrasonic waves, a camera image generation unit that generates a camera image based on an imaging signal from a camera attached to the ultrasonic probe, and a diagnostic information management unit that stores camera attachment information indicating the attachment mode of the camera with respect to the ultrasonic probe in association with the ultrasonic image and the camera image.

[0009] The ultrasonic diagnostic apparatus according to the present invention includes an ultrasonic diagnostic apparatus main body to which the above ultrasonic diagnostic support device is applied, the ultrasonic probe, and the camera.

[0010] The ultrasonic diagnosis support method according to the present invention generates an ultrasonic image based on an ultrasonic signal from an ultrasonic probe that transmits and receives ultrasonic waves, generates a camera image based on an imaging signal from a camera attached to the ultrasonic probe, and stores camera attachment information indicating an attachment mode of the camera to the ultrasonic probe in association with the ultrasonic image and the camera image.

[0011] The program according to the present invention causes a computer to execute a process of generating an ultrasonic image based on an ultrasonic signal from an ultrasonic probe that transmits and receives ultrasonic waves, execute a process of generating a camera image based on an imaging signal from a camera attached to the ultrasonic probe, and execute a process of storing camera attachment information indicating an attachment mode of the camera to the ultrasonic probe in association with the ultrasonic image and the camera image.

Advantages of the Invention

[0012] According to the present invention, the probe scanning position with respect to the subject can be easily specified, and the objectivity and reproducibility of ultrasonic diagnosis can be improved.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0015] FIG. 1 is a diagram showing the appearance of an ultrasonic diagnostic apparatus 1 according to an embodiment of the present invention. FIGS. 2A and 2B are diagrams showing an example of the attachment mode of the camera 22 to the ultrasonic probe 21. FIG. 2A shows an example in which the camera 22 is attached to a surface 211 (hereinafter referred to as the "major axis surface 211") along the major axis direction of the outer peripheral surface of the ultrasonic probe 21. FIG. 2B shows an example in which the camera 22 is attached to a surface 212 (hereinafter referred to as the "minor axis surface 212") along the minor axis direction of the outer peripheral surface of the ultrasonic probe 21. FIG. 3 is a block diagram showing the main part of the control system of the ultrasonic diagnostic apparatus 1.

[0016] <Ultrasonic diagnostic apparatus 1> The ultrasonic diagnostic apparatus 1 is used to visualize the shape, properties, or dynamics within a subject as an ultrasonic image and perform image diagnosis. As shown in FIG. 1, the ultrasonic diagnostic apparatus 1 includes an ultrasonic diagnostic apparatus main body 10, an ultrasonic probe 21, and a camera 22. The ultrasonic probe 21 and the camera 22 are communicably connected to the ultrasonic diagnostic apparatus main body 10 via, for example, a cable 23. Note that the ultrasonic probe 21 and the camera 22 may be communicably connected to the ultrasonic diagnostic apparatus main body 10 by wireless communication.

[0017] <ultrasonic probe 21> The ultrasonic probe 21 transmits ultrasonic waves to the subject, receives ultrasonic echoes (reflected ultrasonic waves) reflected by the subject, converts them into received signals, and transmits them to the ultrasonic diagnostic apparatus main body 10.

[0018] Any electronic scanning type probe such as a convex probe, a linear probe, or a sector probe, or a mechanical scanning type probe such as a mechanical sector probe can be applied to the ultrasonic probe 21. The ultrasonic probe 21 may have a puncture needle guide portion for attaching a puncture needle and guiding the puncture direction.

[0019] The ultrasonic probe 21 has, for example, an acoustic lens, an acoustic matching layer, a vibrator array, and a backing material in order from the ultrasonic transmission / reception side (the surface side of the probe head) (all not shown). Note that a protective layer may be disposed on the surface (ultrasonic transmission / reception surface) of the acoustic lens.

[0020] The acoustic lens is a lens that converges ultrasonic waves in the slice direction. The slice direction is a direction orthogonal to the scan direction in which a plurality of vibrators are arranged. For example, when a material having a lower sound velocity than that of a living body is used for the acoustic lens, the acoustic lens has a kamaboko-like shape in which the central portion in the slice direction bulges. Hereinafter, the slice direction will be referred to as the "minor axis direction", and the scan direction will be referred to as the "major axis direction".

[0021] The acoustic matching layer is an intermediate substance for efficiently introducing ultrasonic waves into the subject, and matches the acoustic impedance between the vibrator and the subject. The vibrator array is composed of, for example, a plurality of strip-shaped vibrators arranged in a single row in the scanning direction. The backing material attenuates unnecessary vibrations generated by the vibrator array.

[0022] The ultrasonic probe 21 has two long-axis surfaces 211 facing each other in the short-axis direction and two short-axis surfaces 212 facing each other in the long-axis direction. The camera 22 is attached along the long-axis surface 211 or the short-axis surface 212.

[0023] One of the short-axis surfaces 212 of the ultrasonic probe 21 is provided with a protrusion (not shown) called an orientation mark. Generally, in ultrasonic diagnosis, based on the orientation mark, how to contact the ultrasonic probe with the diagnostic target site is determined. The user can recognize how the ultrasonic probe 21 is in contact with the diagnostic target site by the orientation mark, and can easily understand how the diagnostic target site is depicted in the ultrasonic image.

[0024] <Camera 22> The camera 22 is attached to the ultrasonic probe 21. The camera 22 images the vicinity of the probe scanning position on the body surface of the subject and transmits the imaging signal to the ultrasonic diagnostic apparatus main body 10.

[0025] The camera 22 is a known imaging device and has an optical lens, a lens driving device, an imaging element, etc. The optical lens preferably includes a wide-angle lens having a wider angle of view than a standard lens. The lens driving device has, for example, an autofocus function of moving the optical lens in the optical axis direction to perform focusing. The imaging element is composed of, for example, a CCD (charge-coupled device) type image sensor, a CMOS (complementary metal oxide semiconductor) type image sensor, etc. The imaging element images the subject image formed by the optical lens.

[0026] The camera 22 is attached to the ultrasonic probe 21 so that the vicinity of the probe scanning position is imaged. The camera 22 is detachably attached to the outer peripheral surface of the ultrasonic probe 21 using an appropriate fixture. Further, the attachment mode of the camera 22 to the ultrasonic probe 21 can be changed. The attachment mode includes the attachment position and the attachment posture.

[0027] The attachment position includes the circumferential position on the outer peripheral surface of the ultrasonic probe 21, that is, on the major axis surface 211 and the minor axis surface 212 of the ultrasonic probe 21. The attachment position may include the vertical position (distance from the scanning surface) with respect to the ultrasonic probe 21.

[0028] The attachment posture includes the rotation (roll) around the optical axis of the camera 22 and / or the rotation (yaw and pitch) around two axes orthogonal to the optical axis.

[0029] The user can appropriately adjust the attachment mode of the camera 22 so that the imaging range includes landmarks that can identify the probe scanning position.

[0030] In the present embodiment, the attachment position of the camera 22 is fixed in advance on each of the two major axis surfaces 211 and the two minor axis surfaces 212 of the ultrasonic probe 21, and is appropriately selected from among the four attachment positions. Further, the attachment posture is such that the optical axis of the camera 22 is orthogonal to the scanning surface of the ultrasonic probe 21 and can only rotate around the optical axis. That is, for the attachment posture of the camera 22, yaw and pitch are invariant.

[0031] Note that the attachment position of the camera 22 may be continuously variable along the outer peripheral surface of the ultrasonic probe 21. Further, the attachment posture of the camera 22 may be rotatable around the yaw axis and the pitch axis.

[0032] <Ultrasonic diagnostic apparatus main body 10 (ultrasonic diagnosis support apparatus)> The ultrasonic diagnostic apparatus main body 10 visualizes the shape, property, or dynamics in the subject as an ultrasonic image (B-mode image) using the reception signal from the ultrasonic probe 21. In the present embodiment, the ultrasonic diagnostic support apparatus according to the present invention is applied to the ultrasonic diagnostic apparatus main body 10.

[0033] The ultrasonic diagnostic apparatus main body 10 includes a system control unit 11, a transmission / reception unit 12, an ultrasonic image generation unit 13, a camera image generation unit 14, a display image generation unit 15, a storage unit 17, an operation unit 18, a display unit 19, and the like.

[0034] The system control unit 11 includes a CPU 111 (Central Processing Unit) as an arithmetic / control device, a ROM 112 (Read Only Memory) and a RAM 113 (Random Access Memory) as main storage devices, and the like. Basic programs and basic setting data are stored in the ROM 112. Further, an ultrasonic diagnostic support program executed at the time of diagnosis is stored in the ROM 112.

[0035] In the present embodiment, the system control unit 11 functions as a camera information setting unit that sets camera attachment information. The camera attachment information is information indicating the attachment mode of the camera 22 to the ultrasonic probe 21. The camera attachment information includes position information indicating the attachment position of the camera 22 and attitude information indicating the attachment attitude. The system control unit 11 functions as a camera information setting unit and acquires the camera attachment information based on, for example, an operation signal from the operation unit 18.

[0036] Further, the system control unit 11 functions as a diagnostic information management unit that manages diagnostic information such as ultrasonic images. The system control unit 11 functions as a diagnostic information management unit, associates the acquired camera attachment information with the ultrasonic image and the camera image, and stores it in the storage unit 17.

[0037] The CPU 111 reads a program corresponding to the processing content from the ROM 112, expands it in the RAM 113, and executes the expanded program, thereby centrally controlling the operations of the functional blocks of the ultrasonic diagnostic apparatus main body 10. The functions of the respective functional blocks are realized, for example, by the cooperation of the respective hardware components constituting the functional blocks and the system control unit 11. Note that some or all of the functions of the respective functional blocks may be realized by the system control unit 11 executing a program.

[0038] The transmission / reception unit 12 includes a transmission circuit and a reception circuit.

[0039] The transmission circuit generates a transmission signal (drive signal) according to an instruction from the system control unit 11 and outputs it to the ultrasonic probe 21. The transmission circuit includes, for example, a clock generation circuit, a delay circuit, a pulse generation circuit, a pulse width setting unit, and the like.

[0040] The reception circuit receives the reception signal from the ultrasonic probe 21 according to an instruction from the system control unit 11 and outputs it to the ultrasonic image generation unit 13. The reception circuit includes, for example, an amplifier, an A / D conversion circuit, an in-phase addition circuit, and the like.

[0041] The ultrasonic image generation unit 13 generates an ultrasonic image indicating the internal state of the subject based on the reception signal obtained by transmitting and receiving ultrasonic waves according to an instruction from the system control unit 11. The ultrasonic image is, for example, a B-mode tomogram, a blood flow spectrum by Doppler, a blood flow distribution image, and the like. The data of the generated ultrasonic image is stored, for example, in the storage unit 17.

[0042] The camera image generation unit 14 generates a camera image based on the imaging signal from the imaging element of the camera 22 according to an instruction from the system control unit 11. The vicinity of the probe scanning position is depicted in the camera image. The user can specify the probe scanning position based on the camera image. The data of the generated camera image is stored, for example, in the storage unit 17 in association with the ultrasonic image.

[0043] The display image generation unit 15 generates a display image based on the image data from the ultrasonic image generation unit 13 and the camera image generation unit 14 in accordance with the instructions of the system control unit 11. The display image includes, for example, an ultrasonic image and a camera image. The display image generation unit 15 includes a DSC (Digital Scan Converter) or the like that performs coordinate conversion and pixel interpolation according to the type of the ultrasonic probe 21. The data of the generated display image is output to the display unit 19.

[0044] The ultrasonic image generation unit 13, the camera image generation unit 14, and the display image generation unit 15 are composed of dedicated or general-purpose hardware (electronic circuits) such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), and a PLD (Programmable Logic Device) according to each process, and cooperate with the system control unit 11 to realize each function.

[0045] The operation unit 18 receives, for example, a command for instructing the start of diagnosis or the like or an input of information about the subject. The operation unit 18 has, for example, an operation panel having a plurality of input switches, a keyboard, a trackball, and the like. Note that the operation unit 18 may be configured by a touch panel provided integrally with the display unit 19.

[0046] The user can instruct the execution of the ultrasonic diagnosis process using the operation unit 18. In addition, the user can set the camera attachment information using the operation unit 18. The specific method for setting the camera attachment information will be described later.

[0047] The display unit 19 is composed of, for example, a liquid crystal display, an organic EL display, a CRT display, or the like. The display unit 19 displays an ultrasonic image and a camera image based on the display data from the display image generation unit 15 in accordance with the instructions of the system control unit 11.

[0048] Although illustration is omitted, the ultrasonic diagnostic apparatus main body 10 may have a communication unit that transmits and receives various types of information to and from an external device (for example, a personal computer or cloud storage on the Internet) connected to a communication network such as a wired / wireless LAN (Local Area Network). The communication unit may include various interfaces such as a NIC (Network Interface Card), a MODEM (MOdulator-DEModulator), and a USB (Universal Serial Bus). In addition, a communication interface for short-range wireless communication such as NFC (Near Field Communication) or Bluetooth (registered trademark) can also be applied to the communication unit.

[0049] <Diagnostic screen> FIGS. 4A and 4B are diagrams showing an example of a diagnostic screen 40 displayed during diagnosis. FIG. 4A shows the entire diagnostic screen 40, and FIG. 4B extracts and shows camera attachment information 43 included in the diagnostic screen 40.

[0050] As shown in FIG. 4A, the diagnostic screen 40 includes an ultrasonic image 41, a camera image 42, and camera attachment information 43. The probe scanning position is depicted in the camera image 42. In addition, in the camera attachment information 43, how the camera 22 is attached to the ultrasonic probe 21 is shown by an icon.

[0051] As shown in FIG. 4B, the icon indicating the camera attachment information 43 has a probe icon 44 indicating the ultrasonic probe 21 and a camera icon 45 indicating the camera 22. Note that the probe icon 44 and the camera 22 shown in FIG. 4B are an example of the case where the camera attachment information 43 is displayed as an icon. The drawing mode of the icon (for example, shape, color, etc.) only needs to be able to visually recognize the attachment mode of the camera 22 with respect to the ultrasonic probe 21, and is not limited to that shown in FIG. 4B.

[0052] Note that the camera attachment information 43 may be displayed as text information together with the icon or instead of the icon.

[0053] The probe icon 44 is arranged at a fixed position within the display area of the camera attachment information 43. The probe icon 44 has a form that can evoke the scanning surface of the ultrasonic probe 21. In FIG. 4B, the probe icon 44 is drawn with a rectangular frame that can distinguish the major axis plane 211 and the minor axis plane 212 of the ultrasonic probe 21. The long side of the probe icon 44 corresponds to the major axis plane 211 of the ultrasonic probe 21. The short side of the probe icon 44 corresponds to the minor axis plane 212 of the ultrasonic probe 21.

[0054] The probe icon 44 has a probe reference mark 44a indicating the reference plane among the outer peripheral surfaces (the major axis plane 211 and the minor axis plane 212) of the ultrasonic probe 21. The probe reference mark 44a is arranged corresponding to, for example, the surface where the orientation mark is provided (here, one of the minor axis planes 212).

[0055] The camera icon 45 is arranged adjacent to the long side or the short side of the probe icon 44. In FIG. 4B, the camera icon 45 is drawn with a rectangular frame representing the imaging area. The camera icon 45 has a camera reference mark 45a indicating the reference for the mounting posture of the camera 22. The camera reference mark 45a is arranged at a position, for example, indicating the upper side in the camera image 42 of the diagnostic screen 40.

[0056] The camera icon 45 is arranged around the probe icon 44 based on the camera attachment information preset by the camera information setting unit (system control unit 11). The user can know the mounting position of the camera 22 with respect to the ultrasonic probe 21 from the position of the camera icon 45 with respect to the probe icon 44. Also, the user can know the mounting posture of the camera 22, that is, the rotation around the optical axis, from the rotation mode of the camera icon 45.

[0057] For example, in the case of the camera attachment information 43 shown in FIG. 4B, the user can know that the camera 22 is disposed on one of the long axis surfaces 211 of the ultrasonic probe 21. Further, the user can know that the vertical direction of the camera image 42 coincides with the short axis direction of the ultrasonic probe 21, and the upper side of the camera image 42 is the distal side with respect to the ultrasonic probe 21.

[0058] As shown in FIG. 4A, the ultrasonic image 41, the camera image 42, and the camera attachment information 43 are displayed on the same diagnostic screen 40. With the diagnostic screen 40, the user can understand how the camera image 42 is captured by the camera 22 attached to the ultrasonic probe 21, and can easily and accurately grasp the probe scanning position with respect to the subject. Therefore, the objectivity of the diagnosis based on the ultrasonic image 41 is improved. Further, since the probe scanning position can be easily reproduced with reference to the camera image 42 and the camera attachment information 43, the reproducibility of ultrasonic diagnosis is improved.

[0059] <Setting of Camera Attachment Information> In the present embodiment, before starting ultrasonic diagnosis, the camera attachment information is set in advance. The camera attachment information may be set manually by the user, for example, or may be set automatically. The camera attachment information is set based on the actual attachment mode of the camera 22 with respect to the ultrasonic probe 21.

[0060] <Example 1 of Setting of Camera Attachment Information> FIG. 5 is a flowchart showing an example of ultrasonic diagnosis support processing executed when setting camera attachment information manually. This processing is realized, for example, in the ultrasonic diagnostic apparatus 1 by the CPU 111 of the system control unit 11 executing an ultrasonic diagnosis support program stored in the ROM 112 when the diagnostic mode is started.

[0061] In the first setting example, the system control unit 11 presents an icon indicating the attachment mode of the camera 22 with respect to the ultrasonic probe 21, and sets the camera attachment information based on an icon operation using the icon.

[0062] Specifically, in step S101, the system control unit 11 controls the display unit 19 to display the setting screen 50 (processing as a camera information setting unit). As shown in FIGS. 6A and 6B, a probe icon 44 and a camera icon 45 are displayed on the setting screen 50. The probe icon 44 and the camera icon 45 are the same as those constituting the camera mounting information 43 displayed on the diagnostic screen 40.

[0063] For example, the user uses a drawing tool 51 such as a cursor displayed on the setting screen 50 to change the position of the camera icon 45 relative to the probe icon 44 and the rotation mode of the camera icon 45 according to the actual mounting mode of the camera 22. The user can intuitively recognize the mounting mode of the camera 22 through the icons. Also, the user can confirm the camera mounting information by performing a confirmation operation of clicking the confirmation button 52 displayed on the setting screen 50.

[0064] In step S102, the system control unit 11 determines whether an icon operation has been performed by the user based on the operation signal from the operation unit 18 (processing as a camera information setting unit). If an icon operation has been performed (\"YES\" in step S102), the process proceeds to step S103. If no icon operation has been performed (\"NO\" in step S102), the process proceeds to step S104.

[0065] In step S103, the system control unit 11 acquires the operation content performed using the drawing tool 51 and reflects it in the display mode of the camera icon 45 (processing as a camera information setting unit). For example, in the setting screen 50 shown in FIG. 6A, when the drawing tool 51 is moved to a position along the short side of the probe icon 44 in alignment with the camera icon 45, the setting screen 50 shown in FIG. 6B is obtained.

[0066] In step S104, the system control unit 11 determines whether a confirmation operation has been performed by the user based on the operation signal from the operation unit 18 (processing as a camera information setting unit). If the confirmation operation has been performed ( "YES" in step S104), the process proceeds to step S105. If the confirmation operation has not been performed ( "NO" in step S104), the process returns to step S102 and the processes of steps S102 to S104 are repeated.

[0067] In step S105, the system control unit 11 sets the camera mounting information based on the position coordinates and rotation mode of the camera icon 45 confirmed on the setting screen 50 (processing as a camera information setting unit). The set camera mounting information is temporarily stored in, for example, the RAM 113.

[0068] Next, in step S111, the system control unit 11 determines whether a diagnosis start operation has been performed based on the operation signal from the operation unit 18. The diagnosis start operation is, for example, an operation in which the user presses the diagnosis start button of the operation unit 18. When the diagnosis start operation is performed ( "YES" in step S111), the process proceeds to step S112.

[0069] In step S112, the system control unit 11 associates and stores the ultrasonic image and the camera image. The data of the ultrasonic image and the camera image are temporarily stored in, for example, the RAM 113. The ultrasonic image and the camera image may be still images or moving images.

[0070] In step S113, the system control unit 11 determines whether a diagnosis end operation has been performed. The diagnosis end operation is, for example, an operation in which the user presses the diagnosis end button of the operation unit 18. When the diagnosis end operation is performed ( "YES" in step S113), the process proceeds to step S114. When the diagnosis end operation is not performed ( "NO" in step S113), the process returns to step S112 and the processes of steps S112 and S113 are repeated.

[0071] In step S114, the system control unit 11 associates the ultrasonic image and the camera image stored in step S112 with the camera mounting information set in step S105, and stores them in, for example, the storage unit 17. The stored ultrasonic image, camera image, and camera mounting information are read out, for example, at the time of diagnosis and displayed on the diagnosis screen 40 (see FIG. 4A).

[0072] <Example 2 of setting camera mounting information> FIG. 7 is a flowchart showing another example of the ultrasonic diagnosis support process executed when manually setting camera mounting information. Compared with the flowchart shown in FIG. 5, mainly, the processes as the camera information setting unit in steps S201 to S205 are different.

[0073] In setting example 2, a plurality of options indicating the mounting mode of the camera 22 with respect to the ultrasonic probe 21 are presented, and the camera mounting information is set based on an operation of selecting from among the plurality of options.

[0074] Specifically, in step S201, the system control unit 11 controls the display unit 19 to display the setting screen 60 (process as the camera information setting unit). As shown in FIGS. 7A and 7B, on the setting screen 60, four candidate position buttons 61 to 64 that are candidates for the mounting position of the camera 22 are arranged around the probe icon 44.

[0075] The user can visually recognize the options indicating the mounting mode of the camera 22 with respect to the ultrasonic probe 21 by the candidate position button 61. The user selects the mounting position of the camera 22, for example, from among the candidate position buttons 61 to 64 displayed on the setting screen 60.

[0076] In step S202, the system control unit 11 determines whether a selection operation has been performed by the user based on the operation signal from the operation unit 18 (processing as the camera information setting unit). When a selection operation has been performed (in step S202, "YES"), the process proceeds to step S203. When no selection operation has been performed (in step S202, "NO"), the process proceeds to step S204.

[0077] In step S203, the system control unit 11 reflects the selection operation on the setting screen 60 (processing as the camera information setting unit). For example, on the setting screen 60 shown in FIG. 8A, when a selection operation is performed on the candidate position button 61, the setting screen 60 shown in FIG. 8B is obtained. In FIG. 8B, the candidate position button 61 has changed to a camera icon 45, indicating that the candidate position L1 has been selected as the mounting position of the camera 22.

[0078] In step S204, the system control unit 11 determines whether a confirmation operation has been performed by the user based on the operation signal from the operation unit 18 (processing as the camera information setting unit). When a confirmation operation has been performed (in step S204, "YES"), the process proceeds to step S205. When no confirmation operation has been performed (in step S204, "NO"), the process returns to step S202 and the processes of steps S202 to S204 are repeated.

[0079] In step S205, the system control unit 11 sets the camera mounting information based on the candidate position confirmed on the setting screen 60 (processing as the camera information setting unit). The set camera mounting information is temporarily stored in, for example, the RAM 113.

[0080] Since the processing as the diagnostic information management unit in steps S211 to S214 is the same as that in steps S111 to S114 in FIG. 5, the description thereof is omitted.

[0081] In addition, in this second setting example, after the camera icon 45 is displayed, the rotation mode of the camera icon 45 may be changed by icon operations as in the first setting example (see Fig. 8C). Further, in the second setting example, instead of the candidate position buttons 61 to 64, options indicating the mounting position of the camera 22 may be presented by text information.

[0082] <Third Setting Example of Camera Mounting Information> Fig. 9 is a flowchart showing an example of ultrasonic diagnosis support processing executed when automatically setting camera mounting information. Compared with the flowchart shown in Fig. 5, the processing as the camera information setting unit in steps S301 to S304 is mainly different.

[0083] In the third setting example, the system control unit 11 acquires a camera image captured by the camera 22 with the ultrasonic probe 21 installed at a predetermined position on a calibration sheet having identification information capable of identifying the mounting mode of the camera 22 with respect to the ultrasonic probe 21, and sets the camera mounting information based on the identification information included in the camera image.

[0084] Specifically, the calibration sheet 70 shown in Fig. 10 is used. The calibration sheet 70 has a probe installation frame 75 indicating the installation position of the ultrasonic probe 22. Different identification information 71 to 74 (for example, QR markers) is arranged at four locations around the probe installation frame 75 that are candidates for the mounting position of the camera 22. Information indicating the mounting position of the camera 22 is associated with the identification information 71 to 74.

[0085] In step S301, the system control unit 11 controls the display unit 19 to display an instruction to the user to install the ultrasonic probe 21 on the calibration sheet 70. The user installs the ultrasonic probe 21 in accordance with the displayed instruction, aligning it with the probe installation frame 75 of the calibration sheet 70.

[0086] In step S302, the system control unit 11 analyzes the camera image acquired by the camera 22 (processing as a camera information setting unit).

[0087] In step S303, the system control unit 11 determines whether or not identification information has been detected based on the analysis result of the camera image (processing as a camera information setting unit). If the identification information is detected (in step S303, “YES”), the process proceeds to step S304. If the identification information is not detected (in step S303, “NO”), the process returns to step S302 and the processes of steps S302 and S303 are repeated.

[0088] In step S304, the system control unit 11 sets camera mounting information based on the detected identification information (processing as a camera information setting unit). The set camera mounting information is temporarily stored in the RAM 113, for example.

[0089] The processing as the diagnostic information management unit in steps S311 to S314 is the same as that in steps S111 to S114 in FIG. 5, and thus the description thereof is omitted.

[0090] Note that in this setting example 3, after the mounting position of the camera 22 is determined based on the detection results of the identification information 71 to 74, the camera icon 45 may be displayed as in setting example 1, and the rotation mode of the camera icon 45 may be changed by an icon operation.

[0091] As described above, the ultrasonic diagnostic apparatus main body 10 according to the embodiment includes the following characteristic matters alone or in appropriate combination.

[0092] That is, the ultrasonic diagnostic apparatus main body 10 (ultrasonic support apparatus) of the present embodiment includes an ultrasonic image generation unit 13 that generates an ultrasonic image based on an ultrasonic signal from an ultrasonic probe 21 that transmits and receives ultrasonic waves, a camera image generation unit 14 that generates a camera image based on an imaging signal from a camera 22 attached to the ultrasonic probe 21, and a diagnostic information management unit that stores camera attachment information indicating the attachment mode of the camera 22 to the ultrasonic probe 21 in association with the ultrasonic image and the camera image. The diagnostic information management unit is realized by the system control unit 11 of the ultrasonic diagnostic apparatus main body 10.

[0093] In addition, the ultrasonic diagnostic support method according to the embodiment generates an ultrasonic image based on an ultrasonic signal from an ultrasonic probe 21 that transmits and receives ultrasonic waves, generates a camera image based on an imaging signal from a camera attached to the ultrasonic probe 21, and stores camera attachment information indicating the attachment mode of the camera to the ultrasonic probe in association with the ultrasonic image and the camera image (steps S111 to S114 in FIG. 5).

[0094] In the embodiment, the ultrasonic diagnostic support apparatus according to the present invention is realized by the system control unit 11 executing an ultrasonic diagnostic support program. That is, the ultrasonic diagnostic support program causes the system control unit 11 (computer) to perform a process of generating an ultrasonic image based on an ultrasonic signal from an ultrasonic probe 21 that transmits and receives ultrasonic waves, a process of generating a camera image based on an imaging signal from a camera 22 attached to the ultrasonic probe 21, and a process of storing camera attachment information indicating the attachment mode of the camera 22 to the ultrasonic probe 21 in association with the ultrasonic image and the camera image (steps S111 to S114 in FIG. 5).

[0095] The ultrasonic diagnostic support program can be provided, for example, via a computer-readable portable storage medium (including, for example, an optical disk, a magneto-optical disk, and a memory card) in which the program is stored. Also, for example, the ultrasonic diagnostic support program can be provided by downloading from a server that holds the program via a network.

[0096] Further, the ultrasonic diagnostic apparatus 1 according to the embodiment includes an ultrasonic diagnostic apparatus main body 10 to which an ultrasonic diagnostic support apparatus is applied, an ultrasonic probe 21, and a camera 22.

[0097] According to the embodiment, since it is possible to grasp how the camera 22 is attached to the ultrasonic probe 21 from the camera attachment information, it is possible to grasp what kind of camera image is obtained in a state where the ultrasonic probe 21 is in contact with the subject. Therefore, since the probe scanning position when the ultrasonic image is acquired can be easily specified from the camera image, the objectivity of the diagnosis based on the ultrasonic image is improved. In addition, since the probe scanning position can be easily reproduced with reference to the camera image and the camera attachment information, the reproducibility of ultrasonic diagnosis is improved. Note that the attachment position and attachment posture of the camera 22 with respect to the ultrasonic probe 21 are preferably adjusted so that landmarks capable of specifying the probe scanning position are drawn in the camera image.

[0098] In the ultrasonic diagnostic apparatus main body 10 (ultrasonic diagnostic support apparatus), the camera attachment information includes position information indicating the attachment position of the camera 22 with respect to the ultrasonic probe 21 and posture information indicating the attachment posture of the camera 22. Since the user can more accurately grasp how the camera 22 is attached to the ultrasonic probe 21, the accuracy of the probe scanning position specified from the camera image is improved.

[0099] In the ultrasonic diagnostic apparatus main body 10 (ultrasonic diagnostic support apparatus), the system control unit 11 as a diagnostic information management unit displays the associated ultrasonic image, camera image, and camera attachment information on the same screen. More specifically, the system control unit 11 (diagnostic information management unit) superimposes and displays the camera attachment information on the camera image. The user can easily grasp the relationship between the ultrasonic image and the probe scanning position on the same screen.

[0100] In addition, the ultrasonic diagnostic apparatus main body 10 (ultrasonic diagnostic support apparatus) includes a camera information setting unit capable of setting camera information. The camera information setting unit is realized by the system control unit 11 of the ultrasonic diagnostic apparatus main body 10. The ultrasonic diagnostic apparatus main body 10 can easily acquire the attachment mode of the camera 22 with respect to the ultrasonic probe 21.

[0101] In the ultrasonic diagnostic apparatus main body 10 (ultrasonic diagnostic support apparatus), the system control unit 11 as the camera information setting unit presents an icon indicating the attachment mode of the camera 22 with respect to the ultrasonic probe 21, and sets the camera attachment information based on an icon operation using the icon. The user can easily set the attachment mode of the camera 22 by an intuitive operation using the icon.

[0102] In the ultrasonic diagnostic apparatus main body 10 (ultrasonic diagnostic support apparatus), the system control unit 11 as the camera information setting unit presents a plurality of options indicating the attachment mode of the camera 22 with respect to the ultrasonic probe 21, and sets the camera attachment information based on an operation of selecting from the plurality of options. The user can easily set the attachment mode of the camera 22 by a selection operation from the plurality of options.

[0103] In the ultrasonic diagnostic apparatus main body 10 (ultrasonic diagnostic support apparatus), the system control unit 11 as the camera information setting unit acquires a camera image captured by the camera 22 in a state where the ultrasonic probe 21 is installed at a predetermined position on a calibration sheet having identification information capable of identifying the attachment mode of the camera 22 with respect to the ultrasonic probe 21, and sets the camera attachment information based on the identification information included in the camera image. The user can easily set the attachment mode of the camera 22 only by installing the ultrasonic probe 21 at a predetermined position on the calibration sheet.

[0104] In the ultrasonic diagnostic apparatus 1, the camera 22 can change its attachment position and / or attachment posture with respect to the ultrasonic probe 21. Further, the camera 22 is detachable from the ultrasonic probe 21. Also, the camera 22 preferably has a wide-angle lens. The user can easily adjust the attachment position and attachment posture of the camera 22 with respect to the ultrasonic probe 21 so that landmarks capable of specifying the probe scanning position are drawn in the camera image.

[0105] As described above, the invention made by the present inventor has been specifically described based on the embodiments. However, the present invention is not limited to the above embodiments and can be modified without departing from the gist thereof.

[0106] In the embodiment, the ultrasonic diagnosis support apparatus of the present invention is applied to the ultrasonic diagnostic apparatus main body 10. However, part or all of the functions as the ultrasonic diagnosis support apparatus may be incorporated in the ultrasonic probe 21.

[0107] Also, the attachment position of the camera 22 may be continuously variable in each of the two major axis planes 211 and the two minor axis planes 212 of the ultrasonic probe 21. Further, the attachment posture of the camera 22 may be variable with respect to yaw and pitch. In this case, the camera attachment information is drawn in a three-dimensional display.

[0108] Also, a detection device such as an acceleration sensor or a gyro sensor may be incorporated in the ultrasonic probe 21 and / or the camera 22 so that the attachment mode of the camera 22 with respect to the ultrasonic probe 21 can be automatically acquired.

[0109] The embodiments disclosed this time should be considered as illustrative in all respects and not restrictive. The scope of the present invention is shown not by the above description but by the claims, and it is intended that all modifications within the meaning and scope equivalent to the claims are included.

Explanation of reference numerals

[0110] 1 Ultrasonic diagnostic apparatus 10 Ultrasonic diagnostic apparatus main body (ultrasonic diagnostic support apparatus) 21 Ultrasonic probe 22 Camera

Claims

1. An ultrasonic image generation unit that generates an ultrasonic image based on an ultrasonic signal from an ultrasonic probe that transmits and receives ultrasonic waves, A camera image generation unit that generates a camera image based on an imaging signal from a camera attached to the ultrasonic probe, A diagnostic information management unit that stores camera attachment information indicating an attachment mode of the camera to the ultrasonic probe in association with the ultrasonic image and the camera image, An ultrasonic diagnostic support device comprising the above.

2. The camera attachment information includes position information indicating an attachment position of the camera to the ultrasonic probe and attitude information indicating an attachment attitude of the camera, The ultrasonic diagnostic support device according to Claim 1.

3. The diagnostic information management unit displays the associated ultrasonic image, camera image, and camera attachment information on the same screen, The ultrasonic diagnostic support device according to Claim 1.

4. The diagnostic information management unit superimposes and displays the camera attachment information on the camera image, The ultrasonic diagnostic support device according to Claim 3.

5. Comprising a camera information setting unit capable of setting the camera attachment information, The ultrasonic diagnostic support device according to Claim 1.

6. The camera information setting unit presents an icon indicating an attachment mode of the camera to the ultrasonic probe, and sets the camera attachment information based on an icon operation using the icon, The ultrasonic diagnostic support device according to Claim 5.

7. The camera information setting unit presents a plurality of options indicating an attachment mode of the camera to the ultrasonic probe, and sets the camera attachment information based on an operation of selecting from among the plurality of options, The ultrasonic diagnostic support device according to Claim 5.

8. The camera information setting unit acquires the camera image captured by the camera with the ultrasonic probe installed at a predetermined position of a calibration sheet having identification information capable of identifying an attachment mode of the camera to the ultrasonic probe, and sets the camera attachment information based on the identification information included in the camera image, The ultrasonic diagnostic support device according to Claim 5.

9. An ultrasonic diagnostic device main body to which the ultrasonic diagnostic support device according to Claim 1 is applied, The ultrasonic probe, The camera, An ultrasonic diagnostic device comprising the above.

10. The camera is capable of changing an attachment position and / or an attachment attitude with respect to the ultrasonic probe, The ultrasonic diagnostic apparatus according to claim 9.

11. The camera is detachable from the ultrasonic probe. The ultrasonic diagnostic apparatus according to claim 9.

12. The camera has a wide-angle lens. The ultrasonic diagnostic apparatus according to claim 9.

13. Generate an ultrasonic image based on an ultrasonic signal from an ultrasonic probe that transmits and receives ultrasonic waves, Generate a camera image based on an imaging signal from a camera attached to the ultrasonic probe, Store camera attachment information indicating an attachment mode of the camera to the ultrasonic probe in association with the ultrasonic image and the camera image. An ultrasonic diagnostic support method.

14. Cause a computer to Perform a process of generating an ultrasonic image based on an ultrasonic signal from an ultrasonic probe that transmits and receives ultrasonic waves, Perform a process of generating a camera image based on an imaging signal from a camera attached to the ultrasonic probe, Perform a process of storing camera attachment information indicating an attachment mode of the camera to the ultrasonic probe in association with the ultrasonic image and the camera image. A program for causing the above to be executed.

Images