Ultrasonic image search device and its program

Abstract

To simply acquire an ultrasonic image suitable for diagnosis of echo examination.SOLUTION: An ultrasonic image retrieval apparatus 13 is capable of retrieving an object image visualizing a desired examination cross section for a cardiac echo examination, out of a plurality of cardiac ultrasonic images obtained while varying the position and posture of an ultrasonic probe P in contact with the chest surface of an analyte. The retrieval apparatus 13 comprises an image analysis part 16 which performs image analysis of each ultrasonic image, thereby estimates a proper arrangement made of a desired position and desired posture of the ultrasonic probe P capable of obtaining the object image, and extracts as the object image the ultrasonic image obtained in the proper arrangement. The image analysis part 16 comprises a probe arrangement identification part 19 which performs image analysis of acquiring detection information including an inference value expressing the presence probability of a predetermined portion within the heart needing visualization in the object image, with respect to each ultrasonic image, and identifies the proper arrangement of the ultrasonic probe P on the surface of the cardiac apex of the analyte.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an apparatus for searching for an ultrasonic image suitable for diagnosis in ultrasonic inspection and a program thereof.

Background Art

[0002] In recent years, in Japan, echocardiography (heart echo examination), which non-invasively and highly accurately supports a doctor's diagnosis for heart diseases that rank among the top in terms of mortality, has attracted attention. The echocardiography is performed to evaluate the form and movement of a subject's heart by applying ultrasonic waves to the subject's heart, converting the reflected sound waves into electrical signals, and depicting them as images. In this echocardiography, in order to make an appropriate diagnosis, since it requires an advanced technique to accurately and clearly depict an ultrasonic image of the heart that is a two-dimensional examination cross-section, corresponding knowledge and experience are required for examiners such as doctors and technicians who perform the examination. However, the number of hospitals with experienced examiners is limited, and currently, many medical institutions are lacking examiners for performing echocardiography. Furthermore, in echocardiography, the ultrasonic probe held by the examiner is applied to the body surface of a subject such as a patient at various angles, and the subject is forced to maintain an unnatural posture. Therefore, as the examination time lengthens, the burden on the subject increases. Thus, the present inventors have hitherto developed an ultrasonic inspection robot that automates echocardiography in which an examiner manually operates an ultrasonic probe while searching for an appropriate ultrasonic image (see Patent Document 1). In this ultrasonic inspection robot, while rotating a subject in a sitting position in the body side direction and the front-rear direction, an ultrasonic probe disposed relative to the subject's chest is automatically operated along the chest surface of the subject. As a result, the conventional complicated manual operation of the ultrasonic probe by the examiner becomes unnecessary, and the subject can undergo echocardiography with a reduced examination burden.

[0003] Incidentally, in the ultrasonic inspection robot, when controlling the operation of the ultrasonic probe by the robot, it is necessary to specify the position and orientation of the ultrasonic probe at which an ultrasonic image of an appropriate inspection section for determining the presence or absence of heart disease by echocardiography can be obtained. Here, as inspection sections for determining the presence or absence of heart disease diagnosis, it is necessary to draw basic sections called the parasternal left margin left ventricular long axis section, the parasternal left margin left ventricular short axis section, and the apical four-chamber section. Among them, regarding the parasternal left margin left ventricular long axis section, an algorithm for automatically specifying the position information and orientation information of the ultrasonic probe capable of acquiring a desired ultrasonic image has already been proposed by the present inventors (see Patent Document 2).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] According to further research by the present inventors, regarding the ultrasonic image of the apical four-chamber section among the basic sections, using the information specified when acquiring the ultrasonic image of the left ventricular long axis section by the algorithm of Patent Document 2, an algorithm for efficiently specifying the position and orientation of the ultrasonic probe at which an ultrasonic image of the apical four-chamber section appropriate for diagnosis can be obtained by an efficient operation of the ultrasonic probe was created.

[0006] The present invention was devised based on the research of the present inventors, and its object is to provide an ultrasonic image search device and its program that can easily acquire an ultrasonic image of the apical four-chamber section appropriate for diagnosis by an efficient operation of the ultrasonic probe.

Means for Solving the Problems

[0007] To achieve the above object, the present invention mainly searches for a target image in which a desired inspection cross-section for an echocardiogram is depicted from a plurality of ultrasonic images of the heart obtained while changing the position and orientation of an ultrasonic probe that contacts the chest surface of a subject. The apparatus includes an image analysis unit that estimates an appropriate arrangement consisting of a desired position and a desired orientation of the ultrasonic probe at which the target image is obtained by image analysis of each of the ultrasonic images, and extracts the ultrasonic image obtained at the time of the appropriate arrangement as the target image. The image analysis unit performs image analysis to obtain detection information including an inference value representing the probability of the presence of a predetermined site in the heart that needs to be depicted in the target image for each of the ultrasonic images, and includes a probe arrangement specifying unit that specifies the appropriate arrangement of the ultrasonic probe on the surface of the apex of the subject.

[0008] In the claims and the present specification, unless otherwise specified, the directions of the position and orientation of the ultrasonic probe are the directions shown in FIG. 1. That is, in the coordinates of the orthogonal three axes representing the position of the ultrasonic probe P, the "x-axis direction" is the cranio-caudal direction of the subject, the "y-axis direction" is the lateral (left-right) direction of the subject, and the "z-axis direction" is the normal direction of the body surface of the subject. Further, when the central portion at the lower end of the ultrasonic probe P in the direction shown in the figure is used as the rotation center, the "roll direction" representing the orientation of the ultrasonic probe P is the rotation direction φ around the axis (the x-axis) orthogonal to the beam scanning plane F, the "pitch direction" is the rotation direction θ in the direction of swinging the beam scanning plane F back and forth (around the y-axis), and the "yaw direction" is the rotation direction ψ around the z-axis.

Advantages of the Invention

[0009] In the present invention, when searching for a target image in which an apical four-chamber cross-section for echocardiography is drawn from a plurality of ultrasonic images of the heart obtained by the operation of a predetermined ultrasonic probe, the proper placement of the ultrasonic probe on the apical surface for obtaining the target image is automatically specified. That is, from the information on the parasternal left ventricular long-axis cross-section obtained separately and automatically, by analyzing the ultrasonic images respectively obtained by the minimum necessary operation of the ultrasonic probe, the proper placement is automatically specified and the target image is extracted. Therefore, by using an ultrasonic examination robot in combination, an efficient operation of the ultrasonic probe for automatically acquiring the target image becomes possible, significantly shortening the time required for echocardiography compared to the past, reducing the burden on the subject, and also expected to solve the social problem of insufficient number of echocardiography examinations due to a shortage of examiners.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Best Mode for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0012] FIG. 1 shows a block diagram representing a schematic configuration of an echocardiogram inspection system including an ultrasonic image search device according to this embodiment. In this figure, the echocardiogram inspection system 10 is a system that enables an ultrasonic examination of the heart (echocardiogram inspection) of a subject who is at a location away from the doctor. Specifically, this echocardiogram inspection system 10 includes an ultrasonic image imaging device 11 that captures an ultrasonic image depicting an inspection cross-section of the heart by bringing an ultrasonic probe P into contact with the chest surface of the subject, a probe operation device 12 that operates the ultrasonic probe P while being able to change the position and posture of the subject, and a search device 13 that searches for an ultrasonic image (hereinafter referred to as the "target image") in which an apical four-chamber cross-section, which is one of the basic cross-sections for echocardiogram inspection, is depicted from a plurality of ultrasonic images of the heart obtained while changing the position and posture of the ultrasonic probe P in contact with the chest surface of the subject.

[0013] As the ultrasonic image imaging device 11, a known ultrasonic diagnostic device capable of acquiring ultrasonic images for performing various ultrasonic examinations is applied. That is, in this ultrasonic image imaging device 11, a two-dimensional tomographic image in the same cross-section as the beam scanning surface F is acquired as an ultrasonic image by beam scanning with the ultrasonic probe P.

[0014] The probe operating device 12 is composed of a robot that enables six-degree-of-freedom motion control of the ultrasonic probe P, including translational motion in three orthogonal axes and rotational motion around three orthogonal axes, by operating a holding unit (not shown) that holds the ultrasonic probe P. Although not particularly limited, in this embodiment, an ultrasonic inspection robot already proposed by the present inventors is applied (see Japanese Patent Application Laid-Open No. 2023-70607). This ultrasonic inspection robot is configured to be able to change the position and orientation of the ultrasonic probe P relatively arranged on the chest of the subject while rotating the subject in the sitting position in the body side direction and the front-rear direction. Further, the ultrasonic inspection robot is provided with a mechanism for bringing the ultrasonic probe P into contact with the chest surface with a predetermined pressing force in accordance with the unevenness of the chest surface of the subject. Note that since the configuration and the like of the ultrasonic inspection robot are not essential elements of the present invention, detailed description thereof is omitted. The motion control in the above probe operating device 12 is performed by a simple remote operation by an examiner such as a doctor, and also by an operation command from the search device 13 that is required in the process of image search described later.

[0015] The ultrasonic probe P is provided with a known sensor (not shown) capable of detecting its position and orientation. This sensor can acquire the position of each coordinate of three orthogonal axes with a predetermined point set in advance as the origin, and the orientation which is the rotation angle around the three axes. Although not particularly limited, as the coordinate system of this embodiment, the probe coordinate system in the direction of the ultrasonic probe P shown in FIG. 1 is applied. Note that as the sensor here, various devices and systems can be adopted as long as the position and orientation of the ultrasonic probe P can be acquired, such as an acceleration sensor, a magnetic sensor, and an optical sensor.

[0016] The search device 13 is composed of an arithmetic processing device such as a CPU and a storage device such as a memory and a hard disk, and is configured by a computer that functions as the following respective parts. In this search device 13, the target image is searched by estimating an appropriate arrangement consisting of the desired position and the desired orientation of the ultrasonic probe P from which a target image in which a desired apical four-chamber cross-section is drawn can be obtained.

[0017] This search device 13 includes a storage unit 15 that stores image data including ultrasonic images acquired by the ultrasonic imaging device 11, etc., and analyzes a plurality of ultrasonic images of the heart obtained while changing the position and orientation of the ultrasonic probe P, thereby estimating the proper arrangement of the ultrasonic probe P from which a target image can be obtained, and an image analysis unit 16 that extracts the ultrasonic image obtained at the proper arrangement as the target image.

[0018] The image data is data corresponding to the ultrasonic image, including the detection information obtained by image analysis described later together with the acquisition time regarding the position and orientation of the ultrasonic probe P when the ultrasonic image was acquired, from the detection results of the sensor attached to the ultrasonic probe P. In the storage unit 15, image data is sequentially stored at predetermined timings during the movement or rotation of the ultrasonic probe P.

[0019] In the image analysis unit 16, the position and orientation of the ultrasonic probe P from which an ultrasonic image of the parasternal left ventricular long-axis section among the inspection sections used for diagnosing the disease state in the echocardiogram examination can be obtained are set as the starting point of the search operation, and then image analysis processing of the ultrasonic images obtained with the movement of the ultrasonic probe P is performed, and the target image is specified from among the plurality of acquired ultrasonic images. That is, here, by the operation of the probe operation device 12, while the ultrasonic probe P is translated, a desired position estimated to be the chest surface of the subject directly above the apex (hereinafter simply referred to as the "apex surface") is specified. Then, the translation of the ultrasonic probe P is stopped at the apex surface, and by the operation of the probe operation device 12, while changing the orientation of the ultrasonic probe P, a desired orientation from which the target image can be obtained is specified. These desired position and desired orientation are the proper arrangement of the ultrasonic probe P from which the target image can be obtained.

[0020] This image analysis unit 16 includes a starting point specifying unit 18 that specifies the position and orientation of the ultrasonic probe P at the starting point of the search operation, and a probe arrangement specifying unit 19 that specifies the proper arrangement of the ultrasonic probe P while changing the position and orientation of the ultrasonic probe P from the starting point.

[0021] In the starting point specifying unit 18, using the method disclosed in Japanese Patent Application Laid-Open No. 2023-70601 already proposed by the inventors, based on the ultrasonic image obtained from the ultrasonic probe P scanned along a predetermined range on the chest surface of the subject by the operation of the probe operating device 12, the position and orientation of the ultrasonic probe P at which an optimal parasternal left ventricular long-axis cross-section can be obtained are specified. That is, according to this method, as shown in FIG. 2, the position and orientation of the ultrasonic probe P that captures the mitral valve M within the cross-section of the heart H at the image center are obtained, this part is set as the starting point S, and the position of the left ventricular long axis LL, which is a straight line connecting the apex A at the tip of the left ventricle H1 and the center of the mitral valve M, is obtained.

[0022] In the probe placement specifying unit 19, by analyzing the ultrasonic images sequentially obtained when the translational movement and orientation change of the ultrasonic probe P are performed by the operation control of the probe operating device 12, the proper placement of the ultrasonic probe P in a desired orientation at a desired position on the apex surface of the subject is specified.

[0023] In the analysis of this ultrasonic image, detection information within the images of predetermined parts (mitral valve M, tricuspid valve T, ventricular septum VS) of the heart shown in FIG. 2 is acquired. That is, as the detection information here, inference values representing the probability of existence (confidence level) of the predetermined part, positions within the image, etc. are appropriately acquired. For this image analysis, a known object detection model such as YOLO that utilizes deep learning for recognizing a predetermined object based on learned image data is used. Note that for this image analysis, as long as the detection information can be acquired based on the obtained ultrasonic image, various other method systems and devices can be adopted.

[0024] This probe placement specifying unit 19 includes a desired position estimating unit 21 that estimates the desired position of the ultrasonic probe P in a state of contacting the apex surface, and a desired orientation estimating unit 22 that estimates the desired orientation of the ultrasonic probe P at which a target image can be obtained at the desired position.

[0025] In the desired position estimation unit 21, according to the procedure described later, the ultrasonic probe P moves while contacting the chest surface along the left ventricular long axis L from the starting point S in the posture specified by the starting point specifying unit 18, and by analyzing the ultrasonic images sequentially acquired during the movement, the positions in the longitudinal and transverse directions (x-y axes) of the surface of the apical part, which is the target position, are estimated. In the image analysis here, detection information such as the inference value of the mitral valve M in each acquired ultrasonic image is obtained by the object detection model, and based on the detection information, the positions in the longitudinal and transverse directions of the surface of the apical part are specified. Note that the position in the in-vivo and in-vitro direction (z-axis) of the surface of the apical part is automatically specified because the probe operating device 12 is provided with a mechanism for bringing the ultrasonic probe P into contact while following the unevenness of the subject. With the ultrasonic probe P positioned above the surface of the apical part, an ultrasonic image in which a cross-section of the left ventricular long axis of the apical part is depicted is acquired.

[0026] In the desired posture estimation unit 22, by analyzing the ultrasonic images sequentially acquired while changing the posture of the ultrasonic probe P at the desired position on the surface of the apical part, the mitral valve M, the ventricular septum VS, and the tricuspid valve T, which are the drawing conditions of the apical four-chamber cross-section, are on the same plane, and the desired posture of the ultrasonic probe P with the best balance of the four chambers H1 to H4 of the heart H centered on the ventricular septum VS is estimated.

[0027] This desired posture estimation unit 22 includes a yawing information acquisition unit 24 that specifies the desired posture in the yaw direction ψ by analyzing each of the ultrasonic images sequentially obtained along with the rotational movement of the ultrasonic probe P in the yaw direction ψ in contact with the surface of the apical part, a rolling information acquisition unit 25 that specifies the desired posture in the roll direction φ by analyzing each of the ultrasonic images sequentially obtained along with the rotational movement in the roll direction φ, and a pitching information acquisition unit 26 that specifies the desired posture in the pitch direction θ by analyzing each of the ultrasonic images sequentially obtained along with the rotational movement in the pitch direction θ.

[0028] In the yawing information acquisition unit 24, the ultrasonic images sequentially obtained when the ultrasonic probe P is rotated in the yaw direction ψ from the initial posture of the ultrasonic probe P on the apex A shown in Fig. 3(A) are analyzed. That is, here, according to the object detection model, from each acquired ultrasonic image, as shown in Fig. 3(B), the desired posture of the ultrasonic probe P in the yaw direction ψ in which the mitral valve M and the ventricular septum VS are more included in the beam scanning plane F is specified.

[0029] In the rolling information acquisition unit 25, the ultrasonic images sequentially obtained when the ultrasonic probe P is rotated in the roll direction φ from the state of Fig. 3(B) in which the posture in the yaw direction ψ specified by the yawing information acquisition unit 24 is fixed are analyzed. That is, here, for each acquired ultrasonic image, as shown in Fig. 3(C), in order to depict more of the tricuspid valve T that appears symmetrically with the mitral valve M around the ventricular septum VS, the desired posture of the ultrasonic probe P in the roll direction φ of the beam scanning plane F in which the position of the ventricular septum VS is depicted near the image center is specified.

[0030] In the pitching information acquisition unit 26, the ultrasonic images sequentially obtained when the ultrasonic probe P on the apex A is rotated in the pitch direction θ in the desired postures in the yaw direction ψ and the roll direction φ specified by the yawing information acquisition unit 24 and the rolling information acquisition unit 25 are analyzed. That is, here, for each acquired ultrasonic image, from the state of Fig. 4(A), as shown in Fig. 4(B), the desired posture of the ultrasonic probe P in the pitch direction θ of the beam scanning plane F in which the four chambers H1 to H4 of the heart H can be depicted in the best balanced state is specified.

[0031] In the above probe arrangement specifying unit 19, the ultrasonic image obtained with the ultrasonic probe P properly arranged is extracted as the target image.

[0032] Next, the procedure for searching for the target image while operating the ultrasonic probe P will be described with reference to the flowcharts of Figs. 5 to 8.

[0033] In the following search procedure, a first step of specifying the desired position of the ultrasonic probe P by image analysis processing in the desired position estimation unit 21, a second step of specifying the desired postures in the yaw direction ψ and roll direction φ of the ultrasonic probe P by image analysis processing in the yawing information acquisition unit 24 and the rolling information acquisition unit 25, and a third step of specifying the desired posture in the pitch direction θ of the ultrasonic probe P by image analysis processing in the pitching information acquisition unit 26 are executed in this order.

[0034] In the first step, the desired position on the apex surface is specified according to the procedure shown in FIG. 5.

[0035] First, at the starting point specifying unit 18, a starting point S that is the position of the ultrasonic probe P in a posture where an appropriate parasternal left ventricular long-axis cross-section can be obtained is specified (step S101). The ultrasonic probe P moves while being in contact with the chest surface of the subject along the left ventricular long axis LL (see FIG. 2) obtained at the starting point specifying unit 18 from the starting point S while maintaining the same posture by the operation of the probe operating device 12 (step S102). At this time, ultrasonic images are acquired every predetermined time (step S103). For each acquired ultrasonic image, an inference value of the mitral valve M is calculated by the object detection model, and the presence or absence of detection of the mitral valve M is determined (step S104). Here, when the inference value is zero or more, or a predetermined value or more, it is determined that the mitral valve M exists in the acquired ultrasonic image. At this time, the in-image position of the mitral valve M is obtained, and it is determined whether or not the in-image position exists in the vicinity of the image center, that is, within a predetermined allowable range (hereinafter referred to as the "image center range") from the image center (step S105). Therefore, when the in-image position of the mitral valve M does not exist within the image center range, the ultrasonic probe P is rotated in the roll direction φ and the posture is adjusted by controlling the operation of the probe operating device 12 so as to eliminate the positional deviation according to how much and in which direction (left or right) the mitral valve M is displaced from the image center range in the image (step S106). The ultrasonic image acquired after this posture adjustment is stored in the storage unit 15 as image data corresponding to the position and posture of the ultrasonic probe P at that time (step S107). On the other hand, when the in-image position of the mitral valve M exists within the image center range, it is stored in the storage unit 15 as image data corresponding to the position and posture of the ultrasonic probe P in the ultrasonic image at that time (step S107). Then, it is determined whether or not the moving distance of the ultrasonic probe P from the starting point S has reached a preset default distance (step S108). When the default distance is reached, among the image data stored in the storage unit 15, the position corresponding to the ultrasonic image at the last timing when the inference value of the mitral valve M is a predetermined value, that is, a preset threshold value or more, and the mitral valve M has been detected continuously for a plurality of times, is specified as the desired position on the apical surface, and the ultrasonic probe P is moved to the apical surface in this posture by controlling the operation of the probe operating device 12 (step S109).Here, when the mitral valve M is not detected continuously for a plurality of times, the inferred value of the mitral valve M is equal to or greater than the predetermined value, and the position corresponding to the ultrasonic image in which the mitral valve M was last detected is specified as the desired position. Note that the threshold value here is not particularly limited, and in addition to a value arbitrarily set in advance, it can also be the average value of the inferred values in each ultrasonic image.

[0036] Note that as the first step, instead of the procedure in FIG. 5, the procedure shown in FIG. 6 can be used to specify the desired position on the apical surface.

[0037] That is, here, the same processing as steps S101 to S107 in FIG. 5 described above is performed, but instead of determining based on the moving distance of the ultrasonic probe P in step S108, it is determined whether or not the non-detection state of the mitral valve M in step S104 continues for a predetermined number of consecutive times (step S110). Then, when the non-detection state of the mitral valve M continues, the same processing as in step S109 is performed.

[0038] In the above first step, based on the property that the entire ultrasonic image suddenly changes to black due to the intervention of an air layer between the ultrasonic probe P and the heart H immediately after the ultrasonic probe P moving along the left ventricular long axis LL starting from the mitral valve M passes over the apex A. That is, based on the finding that the position of the ultrasonic probe P on the apex A is immediately before the moment when the ultrasonic image changes to black according to the research results of the present inventors, in the movement of the ultrasonic probe P along the left ventricular long axis LL from the mitral valve M, the position where the mitral valve M was last detected in the ultrasonic image is estimated as the position on the apical surface.

[0039] In the second step shown in FIG. 7, while appropriately changing the posture of the ultrasonic probe P existing on the apical surface, the desired postures in the yaw direction ψ and roll direction φ of the ultrasonic probe P in which the mitral valve M, tricuspid valve T, and ventricular septum VS are depicted on the same plane of the ultrasonic image are specified by the processing in the yaw information acquisition unit 24 and the rolling information acquisition unit 25.

[0040] First, regarding the ultrasonic probe P that has moved to the apex surface in the first step, it is rotated in the yaw direction ψ by the operation of the probe operating device 12 (step S201), and ultrasonic images are acquired every predetermined time (step S202). Then, the following processing is performed on the acquired ultrasonic images by the yawing information acquisition unit 24. That is, inference values including the presence or absence of detection of the ventricular septum VS and the mitral valve M are calculated as detection information by the object detection model respectively (steps S203, S204). Here, when the inference value is zero or more, or a predetermined value or more, it is determined that the ventricular septum VS and the mitral valve M exist in the acquired ultrasonic image. Thereafter, a comprehensive inference value obtained by combining the respective inference values for the ventricular septum VS and the mitral valve M is calculated, and the image data at that time with the detection information including the comprehensive inference value added is stored in the storage unit 15 (step S205). This comprehensive inference value is calculated by summing the values obtained by multiplying the respective inference values of the ventricular septum VS and the mitral valve M by preset weighting constants. And when the rotation of the ultrasonic probe P in the yaw direction ψ exceeds a predetermined angle (for example, 180 degrees) (step S206), the rotation angle in the yaw direction ψ with a higher comprehensive inference value is specified as the desired posture (step S207). Note that by comparing ultrasonic images within a preset angle range (predetermined range), the rotation angle in the yaw direction ψ with a higher comprehensive inference value of the ventricular septum VS and the mitral valve M can be specified as the desired posture, and further, the highest rotation angle among the higher rotation angles within each range may be used as the desired posture in the yaw direction ψ. Then, the probe operating device 12 rotates the ultrasonic probe P so as to be in the desired posture in the yaw direction ψ.

[0041] Next, the ultrasonic probe P is rotated in the roll direction φ by the operation of the probe operating device 12 (step S208), and ultrasonic images are acquired at predetermined time intervals (step S209). Then, the following processing is performed on the acquired ultrasonic images by the rolling information acquisition unit 24. That is, based on the detection information of the ventricular septum VS in the acquired ultrasonic images by the object detection model, the presence or absence of the detection is determined (step S210). Also here, when the inference value is zero or more, or a predetermined value or more, it is determined that the ventricular septum VS exists in the acquired ultrasonic image, and it is determined whether or not the in-image position of the ventricular septum VS exists within the image center range (step S211). Therefore, when the in-image position of the ventricular septum VS does not exist within the image center range, attitude adjustment is performed to rotate the ultrasonic probe P in the roll direction φ by the operation control of the probe operating device 12 so that the ventricular septum VS enters the image center range (step S212). Then, in the process of rotating the ultrasonic probe P in the roll direction φ with the attitude in the yaw direction ψ fixed at the position of the apex surface, the ultrasonic probe P at the rotation angle in the roll direction φ when the ventricular septum VS first exists within the image center range is specified as the desired attitude in the roll direction φ (step S213). The probe operating device 12 rotates the ultrasonic probe P so as to obtain the desired attitude in the roll direction φ.

[0042] In the desired attitudes in the yaw direction ψ and the roll direction φ specified in the above second step, the mitral valve M is depicted in a state where the ventricular septum VS exists at the center of the ultrasonic image, and the tricuspid valve T, which is symmetric to the mitral valve M with the ventricular septum VS in between, is also depicted on the same plane of the ultrasonic image. Note that as long as the position and attitude of the ultrasonic probe P at the moment when both the mitral valve M and the ventricular septum VS can be depicted when the ultrasonic probe P is rotated a full circle in the yaw direction ψ are known, the processing in the first step may be omitted, and the second step may be started from the position and attitude.

[0043] In the third step shown in FIG. 8, for the ultrasonic probe P present on the surface of the apex, while maintaining the desired postures in the yaw direction ψ and roll direction φ specified in the second step, the posture in the pitch direction θ is appropriately changed, and when an image in which the balance of the four chambers H1 to H4 of the heart H is depicted in a better state is obtained by the processing in the pitching information acquisition unit 26, the desired posture in the pitch direction θ of the ultrasonic probe P is specified.

[0044] First, for the ultrasonic probe P, while maintaining the postures in the yaw direction ψ and roll direction φ specified in the second step at the position on the surface of the apex, it is rotated in the pitch direction θ by the operation of the probe operating device 12 (step S301), and ultrasonic images are acquired at predetermined time intervals (step S302). For the acquired ultrasonic images, the following processing is performed by the pitching information acquisition unit 26. That is, inference values including the presence or absence of detection of the mitral valve M and tricuspid valve T are calculated as detection information by the object detection model (steps S303, S304). Here, when the inference value is zero or more, or a predetermined value or more, it is determined that the mitral valve M and tricuspid valve T are present in the acquired ultrasonic image. Then, a comprehensive inference value obtained by combining the respective inference values for the mitral valve M and tricuspid valve T is calculated, and the image data at that time with the detection information including the comprehensive inference value is stored in the storage unit 15 (step S305). This comprehensive inference value is calculated by summing the values obtained by multiplying the respective inference values of the mitral valve M and tricuspid valve T by preset weighting constants. Then, it is determined whether or not the rotation of the ultrasonic probe P in the pitch direction θ is equal to or greater than a predetermined angle range (for example, a range of plus or minus 40 degrees based on the posture at the time of determining the desired position in the first step) (step S306), and when the rotation within the predetermined angle range is completed, the rotation angle in the pitch direction θ with the highest comprehensive inference value is specified as the desired posture (step S307). Note that by comparing the ultrasonic images within the preset angle range (predetermined range), the rotation angle in the pitch direction θ with a higher comprehensive inference value for the mitral valve M and tricuspid valve T can be specified as the desired posture. Further, the highest rotation angle among the higher rotation angles within each range may be used as the desired posture in the pitch direction θ.

[0045] Through the above first to third steps, the ultrasonic image corresponding to the desired position and desired orientation of the ultrasonic probe P identified is extracted as a target image, which is an ultrasonic image in which an apical four-chamber cross-section appropriate for diagnosis is depicted (step S308).

[0046] In the above embodiment, the movement and rotation of the ultrasonic probe P are performed by the operation of the probe operating device 12. However, the present invention is not limited to this. Under the guidance of the search device 13, while manually moving and rotating the ultrasonic probe P by the operator, the desired position and desired orientation of the ultrasonic probe P appropriate for obtaining a target image can also be identified by the processing in the search device 13.

[0047] In addition, the configuration and processing procedure of each part of the device in the present invention are not limited to the above description, and various modifications are possible as long as they exhibit substantially the same function.

Explanation of Reference Numerals

[0048] 13 Search device 16 Image analysis unit 19 Probe placement specifying unit 21 Desired position estimation unit 22 Desired orientation estimation unit 24 Yawing information acquisition unit 25 Rolling information acquisition unit 26 Pitching information acquisition unit A Apex H1 - H4 Four chambers LL Left ventricular long axis M Mitral valve P Ultrasonic probe T Tricuspid valve VS Ventricular septum

Claims

1. An apparatus for searching for a target image in which a desired inspection cross-section for an echocardiogram is depicted from a plurality of ultrasonic images of the heart obtained while changing the position and orientation of an ultrasonic probe that contacts the chest surface of a subject, comprising an image analysis unit that estimates an appropriate arrangement consisting of a desired position and a desired orientation of the ultrasonic probe at which the target image is obtained by image analysis of each of the ultrasonic images, and extracts the ultrasonic image obtained at the appropriate arrangement as the target image, The image analysis unit performs image analysis to obtain detection information including an inference value representing the probability of existence of a predetermined part in the heart that needs to be depicted in the target image for each of the ultrasonic images, and comprises a probe arrangement specifying unit that specifies the appropriate arrangement of the ultrasonic probe on the apex surface of the subject, The probe arrangement specifying unit comprises a desired position estimating unit that estimates the desired position at which the ultrasonic probe comes into contact with the apex surface, In the desired position estimating unit, for each of the ultrasonic images sequentially obtained while the ultrasonic probe moves along the chest surface along the long axis of the left ventricle starting from a position estimated in advance on the mitral valve, based on the detection information of the mitral valve, the position of the ultrasonic probe at the last timing when the inference value is equal to or greater than a predetermined value and the mitral valve is detected continuously for a plurality of times is set as the desired position. An ultrasonic image search apparatus characterized by this.

2. An apparatus for searching for a target image in which a desired inspection cross-section for an echocardiogram is depicted from a plurality of ultrasonic images of the heart obtained while changing the position and orientation of an ultrasonic probe that contacts the chest surface of a subject, comprising an image analysis unit that estimates an appropriate arrangement consisting of a desired position and a desired orientation of the ultrasonic probe at which the target image is obtained by image analysis of each of the ultrasonic images, and extracts the ultrasonic image obtained at the appropriate arrangement as the target image, The image analysis unit performs image analysis to obtain detection information including an inference value representing the probability of the existence of a predetermined site in the heart that needs to be depicted in the target image for each of the ultrasonic images, and includes a probe arrangement specifying unit that specifies the proper arrangement of the ultrasonic probe on the surface of the apex of the subject. The probe arrangement specifying unit includes a desired posture estimating unit that estimates the desired posture of the ultrasonic probe existing at a position in contact with the surface of the apex. The desired posture estimating unit includes a yawing information acquisition unit that specifies the desired posture in the yaw direction by analyzing each of the ultrasonic images sequentially obtained along with the rotational movement of the ultrasonic probe in the yaw direction. In the yawing information acquisition unit, for each of the ultrasonic images, based on the detection information of the mitral valve and the ventricular septum, the posture in the yaw direction with a higher comprehensive inference value obtained by comprehensively combining the respective inference values of the mitral valve and the ventricular septum is set as the desired posture. An ultrasonic image search device characterized by this.

3. The desired posture estimating unit includes a rolling information acquisition unit that specifies the desired posture in the roll direction by analyzing each of the ultrasonic images sequentially obtained along with the rotational movement of the ultrasonic probe in the roll direction with the desired posture in the yaw direction of the ultrasonic probe specified by the yawing information acquisition unit fixed. In the rolling information acquisition unit, for each of the ultrasonic images, based on the detection information of the ventricular septum, the posture in the roll direction in which the ventricular septum exists near the image center is set as the desired posture. The ultrasonic image search device according to claim 2, characterized by this.

4. The desired posture estimating unit includes a pitching information acquisition unit that specifies the desired posture in the pitch direction by analyzing each of the ultrasonic images sequentially obtained along with the rotational movement of the ultrasonic probe in the pitch direction with the desired postures in the yaw direction and roll direction of the ultrasonic probe specified by the yawing information acquisition unit and the rolling information acquisition unit fixed. In the pitching information acquisition unit, for each of the ultrasonic images, based on the detection information of the mitral valve and the tricuspid valve, the posture in the pitch direction with a higher comprehensive inference value obtained by comprehensively inferring values of the mitral valve and the tricuspid valve is set as the desired posture. The ultrasonic image search device according to claim 3.

5. A device for searching for a target image in which a desired inspection cross-section for an echocardiogram is depicted from a plurality of ultrasonic images of the heart obtained while changing the position and posture of an ultrasonic probe that contacts the chest surface of a subject, comprising an image analysis unit that estimates an appropriate arrangement consisting of a desired position and a desired posture of the ultrasonic probe at which the target image is obtained by analyzing each of the ultrasonic images, and extracts the ultrasonic image obtained at the time of the appropriate arrangement as the target image. The image analysis unit performs image analysis to acquire detection information including an inference value representing the probability of the presence of a predetermined site in the heart that needs to be depicted in the target image for each of the ultrasonic images, and includes a probe arrangement specifying unit that specifies the appropriate arrangement of the ultrasonic probe on the surface of the apex of the subject's heart. The probe arrangement specifying unit includes a desired posture estimation unit that estimates the desired posture of the ultrasonic probe existing at a position in contact with the surface of the apex so that a four-chamber cross-section of the apex is depicted as the target image. In the desired posture estimation unit, based on the detection information of the mitral valve and the ventricular septum obtained by analyzing each of the ultrasonic images sequentially obtained with the rotational movements of the ultrasonic probe in the yaw direction and the roll direction, the mitral valve, the tricuspid valve, and the ventricular septum can be depicted on the same plane of the ultrasonic image, and a four-chamber can be depicted with the ventricular septum as the center. The desired postures in the yaw direction and the roll direction are specified, and from the detection information of the mitral valve and the tricuspid valve obtained by analyzing each of the ultrasonic images sequentially obtained with the rotational movement of the ultrasonic probe in the pitch direction in the state of the desired postures in the yaw direction and the roll direction, the desired posture in the pitch direction in which a four-chamber can be depicted in a state with better balance is specified. The ultrasonic image search device is characterized by this.

6. A program for a device that searches for a target image in which a desired inspection cross-section for an echocardiogram is drawn from a plurality of ultrasonic images of the heart obtained while changing the position and orientation of an ultrasonic probe that contacts the chest surface of a subject, causing a computer to function as an image analysis unit that estimates an appropriate arrangement consisting of a desired position and a desired orientation of the ultrasonic probe from which the target image is obtained by image analysis of each of the ultrasonic images, and extracts the ultrasonic image obtained at the time of the appropriate arrangement as the target image, The image analysis unit performs image analysis to obtain detection information including an inference value representing the probability of the presence of a predetermined site in the heart that needs to be drawn in the target image for each of the ultrasonic images, and includes a probe arrangement specifying unit that specifies the appropriate arrangement of the ultrasonic probe on the surface of the apex of the subject's heart. The probe arrangement specifying unit includes a desired position estimating unit that estimates the desired position in a state where the ultrasonic probe abuts on the surface of the apex of the heart. In the desired position estimating unit, for each of the ultrasonic images sequentially obtained while the ultrasonic probe moves along the chest surface along the long axis of the left ventricle starting from a position estimated in advance above the mitral valve, based on the detection information of the mitral valve, the position of the ultrasonic probe at the last timing when the inference value is equal to or greater than a predetermined value and the mitral valve has been detected continuously for a plurality of times is set as the desired position. A program for an ultrasonic image search device characterized by this.

7. A program for a device that searches for a target image in which a desired inspection cross-section for an echocardiogram is drawn from a plurality of ultrasonic images of the heart obtained while changing the position and orientation of an ultrasonic probe that contacts the chest surface of a subject, causing a computer to function as an image analysis unit that estimates an appropriate arrangement consisting of a desired position and a desired orientation of the ultrasonic probe from which the target image is obtained by image analysis of each of the ultrasonic images, and extracts the ultrasonic image obtained at the time of the appropriate arrangement as the target image, The image analysis unit performs image analysis to obtain detection information including an inference value representing the probability of the existence of a predetermined site in the heart that needs to be depicted in the target image for each of the ultrasonic images, and includes a probe arrangement specifying unit that specifies the proper arrangement of the ultrasonic probe on the apex surface of the subject. The probe arrangement specifying unit includes a desired posture estimating unit that estimates the desired posture of the ultrasonic probe existing at a position in contact with the apex surface so that an apical four-chamber cross-section is depicted as the target image. In the desired posture estimating unit, based on the detection information of the mitral valve and the ventricular septum obtained by analyzing each of the ultrasonic images sequentially obtained with the rotational movements of the ultrasonic probe in the yaw direction and the roll direction, the mitral valve, the tricuspid valve, and the ventricular septum can be depicted on the same plane of the ultrasonic image, and the desired postures in the yaw direction and the roll direction that enable the depiction of four chambers centered on the ventricular septum are specified. At the same time, from the detection information of the mitral valve and the tricuspid valve obtained by analyzing each of the ultrasonic images sequentially obtained with the rotational movement of the ultrasonic probe in the pitch direction in the state of the desired postures in the yaw direction and the roll direction, the desired posture in the pitch direction that enables the depiction in a state with better balance of the four chambers is specified. A program for an ultrasonic image search device characterized by this.

Images