Ultrasound diagnostic system, method of operating an aid, and program product

Abstract

The present disclosure relates to an ultrasonic diagnostic system and an operation assisting method. A current cross section search section (38) sets a temporary cross section group with respect to a tissue model corresponding to a subject tissue, calculates a similarity between each temporary cross section data and tomographic image data, thereby determining a current cross section corresponding to a scanning plane. An operation assisting information generation section (44) generates operation assisting information based on difference information indicating a spatial relationship between the current cross section and a target cross section (a cross section corresponding to an observation cross section). Based on a camera image, the temporary cross section group set with respect to the tissue model is screened.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to an ultrasonic diagnostic system and an operation assistance method, and particularly relates to a technique for assisting probe operation. BACKGROUND

[0002] An ultrasonic diagnostic system is a system that transmits and receives ultrasonic waves to and from an examinee and forms an ultrasonic image based on a reception signal thereby obtained. The ultrasonic diagnostic system is constituted by an ultrasonic diagnostic apparatus, or by an ultrasonic diagnostic apparatus and various devices connected thereto.

[0003] At the time of ultrasonic examination, a probe (ultrasonic probe) is abutted against the surface of an examinee (living body), and in this state, ultrasonic waves are transmitted from the probe into the examinee, and a reflected wave from the living body is received by the probe. The probe (more accurately, a transducer) is held by an examiner (for example, a doctor or an examination technician). A scan plane (beam scan plane) as a two-dimensional data taking-in region is formed by the probe.

[0004] It is not easy to accurately align the scan plane with an observation cross section (a cross section to be diagnosed) in a subject tissue, that is, to accurately and quickly perform probe operation, skill is required. It is not easy to accurately align the current scan plane with a past observation cross section. Therefore, several techniques for assisting probe operation have been proposed. For example, an ultrasonic diagnostic apparatus provided with a function of displaying navigation information for assisting probe operation is disclosed in Document 1 (Japanese Patent Application Publication No. 2004-16268).

[0005] In addition, an ultrasonic diagnostic apparatus provided with a camera is disclosed in Document 2 (Japanese Patent Application Publication No. 2013-255658). In this ultrasonic diagnostic apparatus, a body surface position and a probe position are determined based on a camera image. In Document 1 and Document 2, there is no description of the use of a tissue model for probe navigation. SUMMARY

[0006] An object of the present disclosure is to provide probe operation assistance information for aligning a scan plane with an observation cross section to an examiner. Alternatively, an object of the present disclosure is to generate probe operation assistance information with a simple structure.

[0007] An ultrasonic diagnostic system according to the present disclosure is characterized by including: a former that forms a tomographic image based on information obtained from a scan plane formed by a probe; a searcher that determines a current model cross section corresponding to the scan plane by comparing the tomographic image with a temporary model cross section group set for a tissue model reflecting a three-dimensional configuration of a subject tissue; and a generator that generates probe operation assistance information for bringing the scan plane close to an observation cross section of the subject tissue based on the current model cross section.

[0008] The operation assisting method according to the present disclosure is characterized by including a process of determining a current model section corresponding to a scan surface formed by a probe, by comparing a tomographic image formed based on information obtained from the scan surface with a provisional model section group set for a tissue model reflecting a three-dimensional configuration of a subject tissue, and a process of generating probe operation assisting information based on a spatial relationship between the current model section and a target model section in the tissue model. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 is a block diagram showing an ultrasound diagnostic system according to an embodiment.

[0010] Figure 2 is a diagram for explaining management of a tissue model and an observation section.

[0011] Figure 3 is a diagram for explaining a method of estimating a subject posture.

[0012] Figure 4 is a diagram for explaining a method of matching coordinate systems.

[0013] Figure 5 is a diagram for explaining selection of a provisional section group.

[0014] Figure 6 is a diagram showing an example of a provisional section set in a tissue model.

[0015] Figure 7 is a diagram showing a current section and a target section that are separate from each other.

[0016] Figure 8 is a diagram showing a current section and a target section that are consistent with each other.

[0017] Figure 9 is a diagram showing another current section.

[0018] Figure 10 is a diagram showing steps in navigation.

[0019] Figure 11 is a diagram showing an example of a navigation image.

[0020] Figure 12 is a block diagram showing a first modified example.

[0021] Figure 13 is a diagram for explaining camera image processing in the first modified example. DETAILED DESCRIPTION

[0022] Hereinafter, an embodiment will be described based on the drawings.

[0023] (1) Overview of the implementation method

[0024] The ultrasound diagnostic system described in this embodiment includes a tomographic image forming unit, a search unit, and an auxiliary information generation unit. The tomographic image forming unit forms a tomographic image based on information obtained from a scanning plane formed by a probe. The search unit compares a set of temporary model cross-sections defined for a tissue model reflecting the three-dimensional structure of the object tissue with the tomographic image, thereby determining the current model cross-section corresponding to the scanning plane. Based on the current model cross-section, the auxiliary information generation unit generates probe operation auxiliary information to bring the scanning plane closer to the observation cross-section in the object tissue. The tomographic image forming unit can also be called a formulator. The search unit can also be called a searcher. The auxiliary information generation unit can also be called a generator.

[0025] Based on the above structure, the position of the current model section, i.e., the position of the current scanning plane, can be easily determined using the tissue model. If the probe is moved according to probe operation assistance information, the scanning plane is brought closer to the observation section in the object tissue. This assists the examiner in probe operation.

[0026] Similarity can also be calculated between each temporary model section and the tomographic image, and the temporary model section most similar to the tomographic image in the temporary model section group can be determined as the current model section (the section corresponding to the scan plane). When the spatial relationship between the probe and / or object tissue and the tissue model is unclear, a large number of temporary model sections need to be set for the tissue model. In this case, the computational load increases when searching for the current model section. Therefore, the spatial relationship between the probe and / or object tissue and the tissue model can also be determined, and the group of temporary model sections set for the tissue model can be filtered based on this spatial relationship. In this case, the above spatial relationship can also be determined based on camera images.

[0027] The ultrasound diagnostic system described in this embodiment includes a model storage unit that stores multiple tissue models. The model storage unit is composed of a memory. A specific tissue model corresponding to the target tissue is selected from the multiple tissue models. A temporary model section group is set for the specific tissue model. Tissue models can be prepared for each tissue individually, or for each examination site within the tissue. For example, a tissue model is volume data acquired through the transmission and reception of ultrasound waves. Alternatively, 3D data obtained manually, or 3D data acquired by other medical devices such as CT scans, can also be used as tissue models. Each tissue model is data reflecting the three-dimensional structure (or three-dimensional manner) of the target tissue and is pre-prepared data. Data obtained from a subject different from the subject being examined can also be used as a tissue model.

[0028] In the ultrasound diagnostic system described in this embodiment, a target model section corresponding to the observation section is determined in the tissue model. The auxiliary information generation unit generates probe operation auxiliary information based on the spatial relationship between the current model section and the target model section in the tissue model. In this embodiment, in the tissue model coordinate system, the difference between the position of the current model section and the position of the target model section is calculated as differential information, and probe operation auxiliary information is generated based on the differential information. The observation section is the section in the target tissue that should be inspected or interpreted. Typically, multiple observation sections are specified in the inspection protocol. Sections that have been observed in the past can also be registered as observation sections. According to the above structure, differential information can be generated based on the simulated spatial relationship between the current model section and the target model section in the tissue model coordinate system, which replaces the real spatial relationship between the scanning plane and the observation section. A positioning system for the position and orientation of the detection probe can also be set (e.g., a positioning system using a three-dimensional magnetic field generator and a magnetic sensor), but according to the above structure, the above differential information can be easily generated even without using a complex structure such as a positioning system.

[0029] The ultrasound diagnostic system according to the embodiment includes: a camera that captures images of a subject in contact with a probe to obtain camera images; and a subject posture recognition unit that identifies the subject's posture based on the camera images. The subject posture recognition unit may also be referred to as a first recognizer. The search unit sets a series of temporary model cross-sections for the tissue model based on the subject's posture. According to this structure, it is possible to filter the series of temporary model cross-sections set for the tissue model based on the subject's posture. It is also possible to establish a correspondence between the tissue model coordinate system and the subject coordinate system based on the subject's posture. For example, if the subject is in a lateral decubitus position, the tissue model can be tilted to match this posture. In practice, the coordinate system can be rotated, or logically rotated. The setting range of the temporary model cross-section groups varies according to the subject's posture.

[0030] The ultrasound diagnostic system according to the embodiment includes: a camera for capturing images of a subject with a probe in contact with it; and a probe position recognition unit for recognizing the position of the probe based on the camera image. The probe position recognition unit may also be referred to as a second recognizer. The search unit sets a temporary model cross-section group for the tissue model based on the probe position. According to this structure, the temporary model cross-section group set for the tissue model can be filtered based on the probe position.

[0031] The ultrasound diagnostic system described in this embodiment includes: a camera that captures images of a subject in contact with a probe to obtain camera images; and a model selection unit that selects a specific tissue model corresponding to the target tissue from multiple tissue models based on the camera images. The model selection unit may also be called a selector. According to this structure, a tissue model corresponding to the tissue to be diagnosed can be automatically selected from multiple tissue models, thus reducing the user's workload. Alternatively, the target tissue can be determined based on the subject's posture and the probe's position, thereby selecting a specific tissue model corresponding to the target tissue.

[0032] In this implementation, the auxiliary information generation unit generates multiple operation instruction messages in stages to serve as auxiliary information for probe operation. Examples of probe operations include parallel movements in the forward-backward direction, parallel movements in the left-right direction, rotational movements around the probe's central axis, and rotational movements around an axis along the electronic scanning direction. Providing multiple operation instruction messages simultaneously can easily confuse the operator, but providing them in stages reduces the likelihood of confusion.

[0033] The operation assistance method involved in the implementation includes: a step of comparing a temporary model section set for a tissue model reflecting the three-dimensional structure of an object tissue with a tomographic image formed based on information obtained from a scan surface formed by a probe, thereby determining a current model section corresponding to the scan surface; and a step of generating probe operation assistance information based on the spatial relationship between the current model section and the target model section in the tissue model.

[0034] The aforementioned operation assistance method can be implemented as a software function. In this case, the program for executing the aforementioned operation assistance method is installed in the ultrasound diagnostic system, which serves as an information processing device, via a removable storage medium or a network. The information processing device has a non-transitory storage medium for storing the aforementioned program. The concept of an information processing device includes computers, ultrasound diagnostic devices, ultrasound diagnostic systems, etc.

[0035] (2) Detailed implementation method

[0036] exist Figure 1 The diagram illustrates a structural example of an ultrasound diagnostic system according to an embodiment. The ultrasound diagnostic system is a medical system installed in medical institutions such as hospitals. Through the ultrasound diagnostic system, ultrasound waves are sent to and received from the patient, and an ultrasound image representing the tissues within the patient is formed based on the information obtained. Tissues targeted for diagnosis include, for example, the heart, liver, kidneys, fetus, and breast tissue.

[0037] The ultrasound diagnostic system has a main body (ultrasound diagnostic device main body) 10. The main body 10 includes electronic circuitry, a processor, and a storage unit. The processor is, for example, a CPU that executes programs. Figure 1 In the main body 10, multiple blocks (refer to symbols 24, 30, 32, 34, 38, 42, 44) represent the various functions performed by the processor. The probe (ultrasonic probe) 12, operation panel 14, display 16, camera 18, etc. are connected to the main body 10.

[0038] The operation panel 14 is an input device with multiple switches, multiple buttons, a trackball, a keyboard, etc. The display 16 is composed of an LCD, an organic EL display, etc. The camera 18 is a device for taking pictures of the subject in contact with the probe; it is composed of a black-and-white camera or a color camera.

[0039] The probe 12 consists of a probe head, a cable, and a connector. The probe head contains an array of vibrating elements, including multiple vibrating elements arranged in a straight line or arc. Ultrasonic waves are transmitted from the vibrating element array into the subject, and reflected waves from the biological body are received by the vibrating element array. More specifically, an ultrasonic beam (transmitting beam and receiving beam) is formed by the vibrating element array, and a scanning surface (beam scanning surface) is formed by electronic scanning. Known electronic scanning methods include electronic sector scanning and electronic linear scanning. The probe head is held by the user as an examiner, and the transmitting and receiving surfaces of the probe head abut against the surface of the subject. The probe head is the main part; hereinafter, the probe head will be simply referred to as probe 12. Additionally, the connector is detachably connected to the main body 10. A two-dimensional vibrating element array may be used instead of a one-dimensional vibrating element array.

[0040] The transmitting / receiving unit 20 is an electronic circuit that functions as both a transmitting beamformer and a receiving beamformer. During transmission, multiple transmitting signals are supplied in parallel to the transducer element array from the transmitting / receiving unit 20. This forms a transmitting beam. During reception, if reflected waves from a biological body are received in the transducer element array, multiple receiving signals are output from the transducer element array to the transmitting / receiving unit 20. In the transmitting / receiving unit 20, receiving beam data is generated by phase alignment and summing (delay and summing) of the multiple received signals. Typically, one receiving frame data is constituted by one scan of the ultrasonic beam. One receiving frame data consists of multiple receiving beam data arranged in the electronic scanning direction. One receiving beam data consists of multiple echo data arranged in the depth direction. Repeated electronic scanning of the ultrasonic beam generates multiple receiving frame data repeatedly. These receiving frame data constitute a receiving frame data string.

[0041] The tomographic image forming unit 22 is a generator that produces tomographic image data strings based on received frame data strings. Specifically, the tomographic image forming unit 22 has a digital scan converter (DSC). The DSC is a dedicated processor with coordinate transformation, pixel interpolation, frame rate conversion, and other functions. The tomographic image data strings are sent to the display processing unit 24 and the tomographic image data storage unit 26. The tomographic image data storage unit 26 temporarily stores each tomographic image data for the matching process described later. Each tomographic image is a B-mode tomographic image.

[0042] The display processing unit 24 has image synthesis and color processing functions, among others. The display processing unit 24 generates images for display on the monitor 16. The images to be displayed include tomographic images, which are motion images. Furthermore, in this embodiment, the monitor 16 displays operational assistance information for assisting probe operation or for probe navigation.

[0043] Camera 18 is a device for acquiring camera images to determine the posture, etc., of the subject. Camera 18 is installed at a location where such camera images can be acquired. For example, if the examiner is located on one side of a bed on which the subject is placed, camera 18 is installed diagonally downwards above the opposite side of the bed. Camera 18 may also be installed downwards from the ceiling. Installing multiple cameras is also possible.

[0044] The subject posture recognition unit 32 is a first recognizer that identifies the subject's posture based on camera images. For example, it can recognize supine postures, lateral recumbent postures, etc. As the subject posture recognition unit 32, a recognizer or estimator that has been machine-learned can also be used.

[0045] The tissue model group storage unit 28 stores multiple tissue models corresponding to multiple tissues. One tissue model can be prepared for a single tissue (e.g., the heart), or multiple tissue models can be prepared for a single tissue. Tissue models can also be prepared for each posture of the subject. Each tissue model can be ultrasound data obtained through ultrasound transmission and reception, or 3D data obtained from a CT or MRI device, or artificially generated 3D data. Each tissue model is three-dimensional data within a three-dimensional storage space, reflecting the three-dimensional structure of the tissue (representing or simulating the three-dimensional structure of the tissue).

[0046] The tissue model selection unit 30 is a selector that selects a specific tissue model from multiple tissue models stored in the tissue model group storage unit 28 based on the target tissue. The target tissue is specified by the user or determined by electronic medical record information, etc. As described later, the target tissue can also be determined based on camera images. For example, the target tissue can also be determined based on the subject's posture and probe position, thereby selecting a specific tissue model.

[0047] In the illustrated structural example, data representing the selected specific tissue model is sent to the coordinate system matching unit 34. The coordinate system matching unit 34 is a unit that aligns the subject's coordinate system with the tissue model's coordinate system. At this time, operations such as rotation can be applied to the data representing the tissue model, or the two coordinate systems can be logically combined.

[0048] The organization model storage unit 36 ​​stores data representing the organization model after coordinate system matching. Based on the stored organization model data, multiple temporary cross-sectional data (multiple temporary model cross-sectional data) are read sequentially. Alternatively, the organization model storage unit 36 ​​can not be set up independently, and multiple temporary cross-sectional data can be read sequentially from the organization model group storage unit 28.

[0049] The current section search unit 38 is a search engine that sequentially compares a set of temporary sections defined for the tissue model in the tissue model storage unit 36 ​​with tomographic images to calculate the similarity. The temporary section that produces the best similarity is determined as the current section (current model section) corresponding to the scan plane. Specifically, correlation operations are performed between each temporary section data and the tomographic image data, thereby calculating the similarity as a correlation value. Based on the calculated similarities, the current section is determined. This series of processes for calculating correlation values ​​can also be called pattern matching processing. Parameters such as the spacing and number of sections in the temporary section set are pre-specified by the user or automatically specified according to the set conditions.

[0050] Theoretically, multiple temporary cross-sections can be set for a tissue model. However, in this implementation, the set of temporary cross-sections is filtered based on information such as the subject's posture and probe position. That is, only a series of temporary cross-sections with matching potential are set. This reduces computational load and improves matching accuracy. The temporary cross-section that produces the best similarity is identified as the current cross-section corresponding to the scanning surface in three-dimensional space within the biological body. The position of the current cross-section in the tissue model corresponds to the position of the scanning surface in actual three-dimensional space. As shown by symbol 39, the current cross-section search unit 38 can provide information such as probe position and proximity type, in addition to information indicating the subject's posture. Based on this information, the set of temporary cross-sections set for the tissue model is filtered.

[0051] The target section management table storage unit 40 stores information defining multiple target sections. These multiple target sections are sections defined for the tissue model and correspond to multiple actual observation sections. For example, the positions of each target section within the tissue model are managed. In this case, the coordinates of the three points included in each target section can be managed, as well as the representative coordinate information of each target section. As the inspection protocol proceeds, multiple observation sections are specified. Simultaneously, multiple target sections are selected. During the difference operation, the target section management table storage unit 40 is assigned an identifier for the determined observation section.

[0052] The difference calculation unit 42 is a processor that calculates difference information based on the spatial relationship between the current cross-section and the target cross-section in the tissue model coordinate system. Specifically, it calculates the difference between the position of the current cross-section and the position of the target cross-section as difference information. For example, using the tissue model coordinate system as a premise, it determines 6-dimensional coordinate values ​​(Δx, Δy, Δz, Δθx, Δθy, Δθz) as difference information. Then, the difference information in the tissue model coordinate system can be transformed into difference information in the subject's coordinate system. In short, it generates the coordinate information required for generating operational assistance information. It can also calculate difference information based on a coordinate system with the probe as a reference.

[0053] The operation assistance information generation unit 44 is a generator that generates operation assistance information based on differential information. For example, the operation assistance information may include operation guidance images, operation guidance sounds, etc. The user may also select the type of information generated. In the display processing unit 24, the operation guidance image is assembled into the display image. The operation guidance image is displayed on the screen of the monitor 16. Operation guidance sounds may also be output from a speaker (not shown).

[0054] Providing multiple operation instructions simultaneously can easily confuse inspectors. Therefore, it is desirable to provide multiple operation instructions in stages according to a given sequence. For example, one could output in stages a parallel movement instruction in a direction orthogonal to the electronic scanning direction (front-back direction), a parallel movement instruction in a direction along the electronic scanning direction (left-right direction), a rotational movement instruction about the probe's central axis, a tilting instruction that tilts the probe at an angle to the scanning plane normal, and a tilting instruction that maintains the orientation of the scanning plane normal while tilting the probe. These series of instructions can also be repeated.

[0055] Figure 2 Table 46 shows an example of organizational model and management of the observation section. Figure 2In the table, the portion indicated by symbol 28A corresponds to the already described tissue model group storage section, and the portion indicated by symbol 40A corresponds to the already described target section management table. In Table 46, along the horizontal direction, the object tissue 48, the subject posture 50, the proximity type 52, the tissue model 54, the observation section 56, and the target section position 58 are shown.

[0056] As the target tissue 48, multiple tissues that may be the subject of examination are registered. As the subject posture 50, multiple postures are registered (supine, lateral, etc.). The proximity type 52 determines the position and posture when the probe is placed. If it is the heart, there are known types such as placing the probe on the chest surface and receiving and transmitting ultrasound through the intercostal spaces, and placing the probe obliquely on the upper abdomen and transmitting and receiving ultrasound towards the apex of the heart. As the tissue model 54, one or more tissue models are prepared for each tissue. As the observation section 56, multiple observation sections that can be selected sequentially are registered in the examination protocol. The user can also select each observation section as the navigation object. As the target section position 58, the position of the target section corresponding to each observation section is managed. This position is, for example, the position in the tissue model coordinate system.

[0057] exist Figure 3 The diagram schematically illustrates a method for recognizing a subject's pose. In the illustrated example, the camera image 60 includes a subject image 62, an examiner image 64, and a probe image 66, among others. The subject pose recognition unit 32 includes a cropper 70 and a machine learning-based subject pose estimator 72. The cropper 70 extracts a portion 68 of the image transmitted by the subject pose estimator 72 from the camera image 60. For example, the subject image 62 can also be used as a reference to determine the image portion 68.

[0058] The subject pose estimator 72 is constructed, for example, by a CNN (Convolutional Neural Network). The subject pose estimator 72 can be constructed by pre-feeding the CNN with multiple training data and allowing the CNN to learn. If the subject pose estimator 72 is input to the image portion 68, a specific pose, such as a supine position with the head up, is estimated. As a result of the pose estimation, the subject coordinate system is determined. For example, the head direction H, the left direction L, and the forward direction F are determined.

[0059] exist Figure 4The coordinate system matching method is schematically illustrated. As shown in (A), in the subject image 62, the subject coordinate system 63 can be considered as described above. On the other hand, as shown in (B) and (C), an organization model 74 corresponding to the object organization is selected from the organization model group storage unit 28. The organization model 74 has an organization model coordinate system 78. In the illustrated example, the organization model coordinate system 78 has an x-direction, a y-direction, and a z-direction. In addition, the symbol 76 represents a structure in the organization model 74 (e.g., the four chambers in the heart).

[0060] As shown in (C), the coordinate system matching unit 34 performs the process of matching the subject coordinate system 63 with the tissue model coordinate system 78. For example, the tissue model coordinate system 78 is rotated to fit the subject coordinate system 63. The symbol 74A represents the tissue model after coordinate system matching. In the illustrated example, the x-direction corresponds to the L-direction, the F-direction corresponds to the -y-direction, and the z-direction corresponds to the H-direction. Coordinate system matching can be performed with the required precision, and can also be performed coarsely if the scanning plane and the observation section are ultimately aligned by the examiner's visual judgment. Alternatively, a tissue model suitable for the subject's posture (without requiring rotation, etc.) can be selected. In this case, coordinate system matching is not required.

[0061] After coordinate system matching, the range within which the transmit / receive origin can be set (equivalent to the probe contact range) can be defined based on its relationship with the tissue model 74A. For example, the two surfaces or portions shown by symbols 200 and 202 are areas where the transmit / receive origin can be set. If the proximity type can be determined, the range within which the transmit / receive origin can be further filtered. By filtering the range within which the transmit / receive origin can be set, the number of temporary cross-section groups that should be set for the tissue model can be reduced.

[0062] exist Figure 5 The diagram shows a temporary section list 204 consisting of all temporary sections that can theoretically be set relative to the organization model. The range 206 of temporary section groups actually set in the organization model is filtered based on the range where the transmission and reception origin can be set. This significantly reduces the computational load when determining the position of the current section.

[0063] exist Figure 6The diagram shows temporary sections 82a and 82b representing a portion of the organization model 74B after coordinate system matching. Symbol 80 denotes a region corresponding to the scan plane, and symbol 81 denotes a position corresponding to the transmit / receive origin. Similarity is calculated between data on temporary section 82a and tomographic image data; subsequently, similarity is calculated between data on temporary section 82b and tomographic image data. This process is repeated. The temporary section that produces the optimal similarity is determined as the current section corresponding to the scan plane. Specifically, the temporary section encompassing region 80 is determined as the current section. In this case, for example, the surface or portion indicated by symbol 204 is determined as the range within which the transmit / receive origin can be set.

[0064] exist Figure 7 The diagram illustrates the spatial relationship between the current section 84 and the target section 86 within the organization model 74C. Differential information D is generated between the current section 84 and the target section 86, and operational auxiliary information is generated based on this differential information D.

[0065] exist Figure 8 The diagram shows other spatial relationships between the current section 84 and the target section 86 within the organization model 74C. The current section 84 and the target section 86 are almost identical. The difference information D represents this difference information.

[0066] exist Figure 9 In the organization model 74C, other regions 88 corresponding to the scanning plane are shown. Symbol 90 indicates the position corresponding to the transmit / receive origin. The result of the matching process determines the temporary cross-section of the package region 88 as the current cross-section.

[0067] exist Figure 10 The diagram schematically illustrates the operation assistance process. During this process, as shown in (A) to (D), multiple operational instructions are provided to the examiner in stages. In the illustrated example, initially, as shown in (A), the movement of probe 92 in the forward-backward direction (movement in a direction orthogonal to the scanning plane) is indicated (refer to symbol 100). Movement of probe 92 in the left-right direction (movement along the direction of the electronic scan) can also be indicated before or after this. Symbol 94 represents the three-dimensional space within the subject. Symbol 96A represents the scanning plane before movement, and symbol 98A represents the scanning plane after movement.

[0068] Next, as shown in (B), the rotational movement of probe 92 (movement about the probe's central axis 102) is indicated. Symbol 96B represents the scanning surface before the movement, and symbol 98B represents the scanning surface after the movement. Next, as shown in (C), the first tilting movement of probe 92 (movement about the horizontal axis 104 parallel to the electronic scanning direction) is indicated. Symbol 96C represents the scanning surface before the movement, and symbol 98C represents the scanning surface after the movement. Finally, as shown in (D), the second tilting movement of probe 92 (movement that moves the scanning surface parallel to the normal 106, with the origin of transmission and reception as the center of rotation) is indicated. Symbol 96D represents the scanning surface before the movement, and symbol 98D represents the scanning surface after the movement. Even after the above auxiliary operation process, each step can be repeatedly performed if the current cross-section is not consistent with or close to the target cross-section.

[0069] exist Figure 11 The following is an example of a manipulation guidance image. On screen 108 of the display, a manipulation guidance image 112 is displayed together with the ultrasound image 110. The manipulation guidance image 112 includes a probe image 114 and indicator marks 116. Multiple indicator marks corresponding to multiple types of indications are prepared and selectively used. A model simulating the subject may also be displayed when showing the manipulation guidance image. The direction of movement (e.g., towards the head, towards the left arm) may also be indicated with text or sound, based on the subject, along with the manipulation guidance image.

[0070] exist Figure 12 The main components of the structure involved in the modified example are shown in the diagram. Figure 12 In the middle, to and Figure 1 Elements that are identical to those shown are labeled with the same symbol, and their descriptions are omitted.

[0071] exist Figure 12 In the illustrated variation, a probe position recognition unit 132 is provided. This probe position recognition unit 132 is a second recognizer that identifies the probe position based on a camera image and its relationship to the subject. For example, it identifies the position of the probe on the front or side of the chest of a subject in a supine position. This allows for the automatic determination of the proximity type, the spatial relationship between the tissue model and the probe, or the identification of the target tissue. Consequently, it enables the filtering of areas where the current cross-section might occur, or the automatic identification of the target tissue.

[0072] The probe position recognition unit 132 provides information indicating the probe position to the interception unit 134 and the model selection unit 30A. The model selection unit 30A automatically determines the target tissue based on the subject's posture and the probe position, and selects the specific tissue model to be used based on the determined target tissue.

[0073] The current section search unit, in the illustrated shape example, consists of an extraction unit 134 and a matching unit 136. The extraction unit 134 sequentially extracts multiple temporary section data from the tissue model. At this time, the extraction unit 134 adaptively or restrictively determines the range of the temporary section group based on the subject's posture and probe position. That is, the temporary section group is filtered based on the subject's posture and probe position. The matching unit 136 performs correlation operations between each temporary section data and the tomographic image data, thereby calculating the similarity for each temporary section data. The current section is thus determined.

[0074] exist Figure 13 The diagram illustrates a structural example of the subject pose recognition unit 32 and the probe position recognition unit 132 involved in a modified example. An image portion 68 is cropped from the camera image 60 by the cropper 70. The machine learning-based subject pose estimator 72 estimates the subject pose based on the image portion 68. The probe position recognition unit 132 is composed of a machine learning-based probe position estimator 138. It is composed of a learned CNN. The machine learning-based probe position estimator 138 estimates the probe position based on the image portion 68. The probe pose can also be estimated at this time. Both the subject pose and the probe position can be estimated simultaneously using a single estimator.

[0075] According to the above implementation method, without using a positioning system with a complex structure, the position of the scanning surface can be easily determined in relation to the observation section. Based on this, operational assistance information can be generated based on the spatial relationship between the position of the scanning surface and the observation section. When determining the position of the scanning surface, i.e., the position of the current section, by analyzing the camera image, temporary section groups used in the matching operation can be filtered, thus reducing the computational load and improving computational accuracy.

Claims

1. An ultrasonic diagnostic system, characterized in that, include: The formor generates a tomographic image based on information obtained from the scanning surface formed by the probe; The searcher determines the current model section corresponding to the scan plane by comparing the tomographic image with a set of temporary model sections set for an organization model that reflects the three-dimensional structure of the object's tissue. as well as The generator, based on the current model cross-section, generates probe operation assistance information to bring the scanning surface close to the observation cross-section of the object tissue. Also includes: A camera is used to capture images of the subject who is in contact with the probe. as well as The first recognizer identifies the subject's posture based on the camera images. The searcher sets the temporary model section group for the tissue model based on the subject's posture.

2. The ultrasonic diagnostic system according to claim 1, characterized in that, This includes a memory that stores multiple organizational models. As the organizational model, a specific organizational model corresponding to the object organization is selected from the plurality of organizational models. The temporary model section group is set for the specific organization model.

3. The ultrasonic diagnostic system according to claim 1, characterized in that, In the tissue model, determine the target model section corresponding to the observed section. The generator generates the probe operation assistance information based on the spatial relationship between the current model section and the target model section in the tissue model.

4. The ultrasonic diagnostic system according to claim 1, characterized in that, Also includes: The second identifier identifies the position of the probe based on the camera image. The searcher sets the temporary model section group for the tissue model based on the subject's posture and the probe's position.

5. The ultrasonic diagnostic system according to claim 2, characterized in that, Also includes: The selector, based on the camera image, selects the specific organization model corresponding to the object organization from the plurality of organization models.

6. The ultrasonic diagnostic system according to claim 1, characterized in that, The generator periodically generates multiple operation instruction messages to serve as auxiliary information for the probe operation.

7. The ultrasonic diagnostic system according to claim 1, characterized in that, The tissue model is based on volumetric data obtained through the transmission and reception of ultrasound waves.

8. An operation assistance method, characterized in that, include: The process of determining the current model section corresponding to the scan surface by comparing a tomographic image formed based on information obtained from the scan surface formed by the probe with a set of temporary model sections set for a tissue model that reflects the three-dimensional structure of the object's tissue. as well as The process of generating probe operation auxiliary information based on the spatial relationship between the current model cross section and the target model cross section in the tissue model. Also includes: The process of taking a picture of the subject who comes into contact with the probe to obtain a camera image; as well as The process of recognizing the subject's posture based on the camera images. Based on the subject's posture, the temporary model section group is set for the tissue model.

9. A program product comprising a program for performing an operation-aiding method in an ultrasound diagnostic system, characterized in that, include: The function of the current model section corresponding to the scan surface is determined by comparing a tomographic image formed based on information obtained from the scan surface formed by the probe with a series of temporary model sections set for a tissue model that reflects the three-dimensional structure of the object's tissue. as well as The function of generating probe operation auxiliary information based on the spatial relationship between the current model cross section and the target model cross section in the tissue model. Also includes: The function of taking pictures of the subject who comes into contact with the probe to obtain camera images; as well as The function of recognizing the subject's posture based on the camera images. Based on the subject's posture, the temporary model section group is set for the tissue model.

Images