Ultrasound diagnostic system

Abstract

To enable efficient searching of past ultrasound images that are highly relevant to the current examination. [Solution] The system generates an ultrasound image from the echo signal obtained by the probe (S10), analyzes the ultrasound image (S12), generates linking information including the results of the analysis (S14), and registers the ultrasound image and linking information in the database (S15). The system searches the database for past ultrasound images with linking information similar to the current linking information (S16). Furthermore, for each of the retrieved past images, the system calculates the image similarity with the current ultrasound image (S18), calculates a relevance score from the image similarity and the similarity of the linking information, and extracts several past images in descending order of relevance score (S20). The extracted images are displayed in a thumbnail display area (S22).

Description

Technical Field

[0001] The present invention relates to an ultrasonic diagnostic system.

Background Art

[0002] In ultrasonic diagnosis, ultrasonic images obtained from past examinations of a subject in the current examination are displayed for reference.

[0003] The ultrasonic diagnostic apparatus disclosed in Patent Document 1 stores an ultrasonic image obtained at the time of execution of a process in a protocol as a reference image. Then, the ultrasonic diagnostic apparatus displays, side by side with the ultrasonic image obtained and displayed in the currently executing process, a past reference image stored for the same process for the same patient (see, for example, paragraphs 0037, 0055 - 0056 of the same document).

[0004] The ultrasonic diagnostic system disclosed in Patent Document 2 provides a first display screen for displaying a list of thumbnails of a plurality of ultrasonic images of a subject diagnosed in the past (see paragraphs 0046, 0050 - 0053 of the same document). Further, the ultrasonic diagnostic system provides a second display screen for displaying side by side one of the plurality of past ultrasonic images and an ultrasonic image obtained in the current examination (see paragraphs 0046, 0054 - 0055 of the same document). The user performs ultrasonic diagnosis on a part corresponding to the past image while viewing the past image on the second display screen. The image obtained by this ultrasonic diagnosis is displayed side by side with the past image.

[0005] The medical image inspection apparatus disclosed in Patent Document 3 searches for a past image of the same patient taken under the same imaging conditions as the imaging conditions in the current examination, and displays the searched past image side by side with the image obtained in the current examination (see paragraphs 0026, 0029 - 0033 of the same document).

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

[0007] The present invention aims to enable efficient retrieval of past ultrasound images that are highly relevant to the current examination, not limited to past ultrasound images taken using the same procedure as the current examination. [Means for solving the problem]

[0008] The ultrasound diagnostic system according to the present invention includes a processor, which generates a first ultrasound image from a signal received by an ultrasound probe, performs processing for displaying the first ultrasound image, performs analysis of the first ultrasound image to obtain analysis results, performs a search for past ultrasound images having the same or similar analysis results as the analysis results for the first ultrasound image, obtains a score representing the degree of similarity with the first ultrasound image for each of the past ultrasound images obtained as a result of the search, and performs display control to display one or more images selected from the past ultrasound images obtained as a result of the search based on the score together with the first ultrasound image.

[0009] The analysis performed here is, for example, a process to obtain a cross-sectional recognition result as the analysis result, which indicates which of a predetermined set of cross-sections the first ultrasound image is an image of.

[0010] In one embodiment, the first ultrasound image is an image of a blood flow velocity waveform obtained by the Doppler method, and the analysis is a process of determining the maximum flow velocity of the blood flow velocity waveform as the analysis result.

[0011] In another embodiment, the processor performs a process to detect tumor portions from the first ultrasound image, and in calculating the score, calculates the score between the past ultrasound image and the first ultrasound image based on a first similarity between the past ultrasound image and the first ultrasound image as a whole and a second similarity between the tumor portions.

[0012] In yet another embodiment, when the processor saves the first ultrasound image, it saves the analysis result obtained by the analysis as an item of attribute information of the first ultrasound image.

[0013] In yet another embodiment, if the first ultrasound image is a moving image, the processor stores the moving image, a representative still image from the moving image, and the analysis results obtained from the analysis of the still image.

[0014] In yet another embodiment, the processor, in the search, obtains memo information entered by the user for the displayed first ultrasound image, and searches for past ultrasound image data that has the same or similar analysis results as the analysis result and has the same or similar memo information as the obtained memo information.

[0015] In yet another embodiment, the processor performs the process of displaying, in the display control, a plurality of thumbnails of images selected from the past ultrasound images obtained as a result of the search, based on the score, in a thumbnail display field provided near the first ultrasound image, arranged in descending order of the score.

[0016] In yet another embodiment, if the processor, in the display control, has selected a plurality of images based on the score from the past ultrasound images obtained as a result of the search, and one of these images is designated as a key image, it displays a thumbnail of the image designated as the key image at the top of the thumbnail display area, regardless of the score for that image.

[0017] In yet another embodiment, if the processor is using a protocol assistant function that assists in the execution of a protocol consisting of one or more inspection steps, the display control displays a thumbnail of a past ultrasound image obtained in the same inspection step of the same protocol as the currently executed protocol at the top of the thumbnail display field, and then displays a thumbnail of the image designated as the key image in the next position.

[0018] When the processor is using a protocol assistant function that assists in the execution of a protocol consisting of one or more inspection steps, in the display control, it displays a thumbnail of a past ultrasound image obtained in the same inspection step of the same protocol as the currently executed protocol at the top of the thumbnail display area.

[0019] In yet another embodiment, the processor displays the past ultrasound image with the highest score alongside the first ultrasound image as a reference image for the first ultrasound image in the display control.

[0020] In yet another embodiment, if the processor obtains information indicating that a measurement has been performed on the past ultrasound image displayed as the reference image, it executes a process for performing the measurement on the displayed first ultrasound image.

[0021] In yet another embodiment, the processor selects a layout for displaying the first ultrasound image and a reference image selected from past ultrasound images, based on the cross-sectional recognition result.

[0022] In yet another embodiment, the processor selects a dual layout in which the first ultrasonic image and the reference image are placed side by side at the same size when the cross-sectional recognition result corresponds to a predetermined first type of cross-section.

[0023] In yet another aspect, when findings are generated by the analysis process, the processor selects a dual layout in which the first ultrasonic image and a reference image selected from the past ultrasonic images are arranged side by side in the same size as the layout when displaying the first ultrasonic image and the reference image.

[0024] In yet another aspect, when the mode of the ultrasonic diagnostic system when generating the first ultrasonic image is a specific mode, the processor selects a dual layout in which the first ultrasonic image and a reference image selected from the past ultrasonic images are arranged side by side in the same size as the layout when displaying the first ultrasonic image and the reference image.

Advantages of the Invention

[0025] According to the present invention, a past ultrasonic image strongly related to the current examination can be efficiently retrieved.

Brief Description of the Drawings

[0026] [Figure 1] It is a diagram showing an example of the functional configuration of the ultrasonic diagnostic system of the embodiment. [Figure 2] It is a diagram schematically showing an example of the screen configuration of the display screen provided by the ultrasonic diagnostic system. [Figure 3] It is a diagram schematically showing another example of the screen configuration of the display screen provided by the ultrasonic diagnostic system. [Figure 4] It is a diagram schematically showing yet another example of the screen configuration of the display screen provided by the ultrasonic diagnostic system. [Figure 5] It is a diagram illustrating a group of items included in the association information. [Figure 6] It is a diagram illustrating a group of items belonging to the imaging conditions among the association information. [Figure 7] It is a diagram illustrating a group of items belonging to the analysis results among the association information. [Figure 8] It is a diagram illustrating a group of items belonging to the user input data among the association information. [Figure 9]This diagram illustrates the procedure for searching for past images that are strongly related to the current image. [Figure 10] This figure schematically shows the image of the past image region generated by the procedure in Figure 9. [Figure 11] This figure illustrates the changes to the procedure in Figure 9 when a key image is taken into consideration. [Figure 12] This figure schematically shows the image of the past image region generated by the procedure in Figure 11. [Figure 13] This diagram illustrates the changes to the procedure in Figure 9 when considering the protocol assistant function. [Figure 14] This figure illustrates the changes to the procedure in Figure 9 when considering the key image and protocol assistant functions. [Figure 15] This figure schematically shows the image of the past image region generated by the procedure in Figure 14. [Figure 16] This figure illustrates the changes to the procedure shown in Figure 9 when considering the similarity of the tumor portions. [Figure 17] This diagram illustrates the processing steps involved in registering video images into a database. [Figure 18] This diagram illustrates a processing procedure that controls measurement in conjunction with the display of a reference image. [Figure 19] This diagram shows an example of the procedure for automatically selecting the display screen layout. [Modes for carrying out the invention]

[0027] Embodiments of this disclosure will be described below with reference to the drawings.

[0028] <Example of a functional configuration of an ultrasound diagnostic system> Figure 1 shows an ultrasound diagnostic system 1 according to an embodiment. This ultrasound diagnostic system 1 is a medical system installed in a medical institution and used during ultrasound examinations. The ultrasound diagnostic system 1 comprises, as functional elements, a probe 10, a transmitting / receiving unit 12, an image forming unit 14, a display processing unit 16, a display device 18, and an information processing system 20.

[0029] The probe 10 transmits and receives ultrasonic waves. Inside the probe 10 is an array of vibrating elements consisting of multiple vibrating elements. An ultrasonic beam is formed by the array of vibrating elements, and this beam is scanned electronically. This forms a beam scanning surface.

[0030] The transmitting / receiving unit 12 supplies multiple transmission signals in parallel to the vibrating element array during transmission, and applies phase-correcting summation (delay summation) to multiple received signals from the vibrating element array during reception. This constitutes beam data. One received frame data is generated by one scan of the ultrasonic beam. One received frame data consists of multiple beam data arranged in the electronic scanning direction. Each beam data consists of multiple echo data arranged in the depth direction.

[0031] The received frame data output sequentially from the transmitting / receiving unit 12 is sequentially input to the image forming unit 14. The image forming unit 14 includes a DSC (Digital Scan Converter). The DSC generates a display frame data sequence from the received frame data sequence. The DSC has a coordinate transformation function, a pixel interpolation function, etc. One display frame data constitutes one tomographic image data. Figure 1 shows an image forming unit that forms various types of ultrasound images, although it is not shown. These various ultrasound images include, for example, two-dimensional tomographic images such as B-mode tomographic images, color Doppler images (also called color flow images), and power Doppler images. The formed ultrasound images may also include images composed of a combination of multiple types of two-dimensional tomographic images, such as an image in which a color Doppler image or a power Doppler image is superimposed on a B-mode tomographic image. Images other than two-dimensional tomographic images, such as pulsed Doppler images (for example, temporal changes in signal waveforms and intensity), are also examples of ultrasound images.

[0032] The display processing unit 16 has image synthesis functions, color processing functions, etc. The display processing unit 16 forms the display screen that is displayed on the display device 18. The display screen first includes an ultrasonic image. The ultrasonic image is a moving image during real-time operation and a still image in the frozen state. The display device 18 is composed of an LCD, an organic EL display, etc. The display screen includes various images generated by the information processing system 20. These may include process list images, measurement list images, measurement value graphs, inspection value graphs, etc.

[0033] The information processing system 20 performs information processing such as controlling the operation of each component of the ultrasound diagnostic system 1, performing various analysis processes on ultrasound images, generating various images to be displayed on the display screen, and configuring the layout of the display screen. The information processing system 20 is equipped with hardware such as a processor (CPU, central processing unit) and memory (RAM, random access memory).

[0034] <Example of analysis process> The analysis processes performed by the information processing system 20 include, for example, the following:

[0035] (1) Cross section recognition The analysis processes performed by the information processing system 20 include, for example, cross-sectional recognition. Cross-sectional recognition is the process of recognizing which cross-section of the subject the ultrasound image (e.g., a B-mode tomographic image) represents.

[0036] The cross-section of a subject is determined, for example, by a combination of the organs within the subject and the positional relationship of the cross-section to those organs. Furthermore, guidelines for ultrasound diagnosis of specific body parts such as the heart, abdomen, and breasts specify the cross-sections that should be imaged for the diagnosis of those areas.

[0037] For cross-sectional recognition, a machine learning model can be used, for example. The machine learning model can be constructed using methods such as neural networks. To construct this model, a large number of pairs of ultrasound images and information identifying the cross-section shown in the ultrasound image (e.g., the identification name of the cross-section) are prepared as training data. The machine learning model is then trained by providing the ultrasound images as input data and the information identifying the cross-section as training data. By installing the sufficiently trained machine learning model into the information processing system 20, the information processing system 20 is given the function of cross-sectional recognition. When an ultrasound image is input to the trained machine learning model, it determines the cross-section shown in the image and outputs information identifying the determined cross-section. The machine learning model can also output the confidence level of the determined cross-section.

[0038] Furthermore, similar to cross-sectional recognition, it is possible to build and utilize machine learning models that can recognize organs displayed in ultrasound images.

[0039] (2) Flow velocity calculation Flow velocity calculation is the process of calculating the flow velocity of blood flowing through blood vessels. In this process, the blood flow velocity is calculated by analyzing ultrasound images containing flow velocity information, such as color Doppler images or pulsed Doppler mode (also called PW mode) images. The target area for flow velocity calculation may be specified by the user, or it may be automatically set by the information processing system 20. An example of user specification is when the user specifies the sample volume in pulsed Doppler mode on a B-mode tomographic image. In an example of automatic setting by the information processing system 20, for example, the area where the flow velocity is to be measured is pre-set for each cross-section. The information processing system 20 then identifies the measurement area set for the cross-section shown in the currently displayed ultrasound image and calculates the flow velocity at that area from the ultrasound image.

[0040] The calculated flow velocity may be the maximum flow velocity at the measurement site or the average flow velocity. Which flow velocity to calculate is predetermined.

[0041] In one example, the information processing system 20 analyzes the maximum flow velocity in the blood flow velocity waveform obtained in PW mode. In another example, the information processing system 20 analyzes the average flow velocity within a region of interest (which may be specified by the user, for example) in the color Doppler image.

[0042] (3) Tumor detection Tumor detection is the process of detecting areas with tumor-like characteristics from ultrasound images, such as B-mode tomography images.

[0043] For tumor detection, a machine learning model can be used, for example. The machine learning model can be constructed using methods such as neural networks. To construct this model, a large number of pairs of ultrasound images and information about tumors related to those images are prepared as training data. The machine learning model is then trained by providing the ultrasound images as input data and the tumor information as training data. By installing this sufficiently trained machine learning model into the information processing system 20, the system gains the functionality of tumor detection. The tumor information may also be a report indicating the presence or absence of a tumor. In this case, the trained machine learning model determines whether the input ultrasound image contains a portion with tumor characteristics and outputs the result of that determination along with a confidence level. Alternatively, the tumor information may represent the location and extent of the portion containing tumor characteristics within the ultrasound image. In this case, the trained machine learning model can determine the location and extent of the portion containing tumor characteristics from the input ultrasound image and output the result of that determination along with a confidence level. Such tumor detection functionality is already being implemented, for example, as CAD (Computer-Aided Detection).

[0044] (4) Others Furthermore, the information processing system 20 is not required to perform all of the analysis processes exemplified above, and may perform analysis processes other than those exemplified above. For example, as another analysis process, the information processing system 20 may identify the location of the blood vessel to be measured in a specific cross-section on the B-mode tomographic image and automatically calculate the diameter of the blood vessel at that location. Also, as another analysis process, the information processing system 20 may extract image features such as histograms, edges, and textures from B-mode images and color Doppler images. These image features can also be used in the calculation of image similarity for B-mode images and color Doppler images, as described later. Also, as another analysis process, the information processing system 20 may extract blood flow velocity waveform features from pulsed Doppler mode images. These features can also be used in the calculation of image similarity for pulsed Doppler mode images.

[0045] Another example of analysis processing is the generation of findings. Findings are sentences that describe abnormalities or signs of abnormalities detected by the examination. For example, if a suspected tumor is detected by the tumor detection function, a finding indicating a suspected tumor is generated. If no abnormalities or signs of abnormalities are detected in the ultrasound image, no findings are generated. To automate the generation of findings, a machine learning model can be constructed and used to generate findings from one or more pieces of information, such as ultrasound images, the results of various analyses of those ultrasound images, or the detection results of other detection systems such as electrocardiograms. The findings generated by the information processing system 20 are reviewed by users such as doctors, corrected as necessary, and then registered in the database.

[0046] <Protocol Assistant Function> The information processing system 20 may have a protocol assistant function. The protocol assistant function is a function that supports the execution of a series of examinations, which tend to be complex, by automatically executing protocols (i.e., examination procedures). A protocol defines the procedures for a series of examinations to be performed using the functions of the ultrasound diagnostic system 1. For example, a protocol defines the content of each examination step to be performed (e.g., the cross section to be examined, imaging conditions, etc.) and the order in which these examination steps are executed.

[0047] Once a specific protocol is selected and executed, pre-configured execution conditions are automatically set for each process. Furthermore, necessary measurements are initiated during the execution of that process. When a store operation is performed, the system automatically transitions to the next process. Each individual process constituting the inspection protocol is also referred to as a "View" from the perspective of image observation.

[0048] The information processing system 20 has standard protocols installed that are suitable for each application. Furthermore, it is possible to create patient-specific protocols by modifying the standard protocols, register them in the information processing system 20, and reuse them. For example, the mechanism disclosed in Patent Document 1 can be used to utilize the standard and specific protocols.

[0049] <Example of display screen> Figure 2 shows an example of a display screen 100 configured by the information processing system 20.

[0050] The display screen 100 shows the current image 110. The current image 110 is an ultrasound image taken at the current time by ultrasound transmission and reception by the probe 10. When a specific protocol is selected, a process list 112 indicating the contents of that protocol is displayed. The process list 112 consists of multiple process displays 114 corresponding to multiple processes. The process display 114 corresponding to the currently executing process is displayed in a form that can be distinguished from process displays corresponding to other processes (for example, a different background color). Although omitted in the figure, a measurement list may be displayed showing information (such as measurement names) that identifies one or more measurements defined for the currently executing process. A measurement value display field may be provided where acquired measurement values ​​are displayed.

[0051] A reference image 120 is displayed near the current image 110. The reference image 120 is an image that serves as a reference for interpreting the current image 110. In a typical example, the reference image 120 is selected from past images. Past images are ultrasound images that were previously acquired from the same subject as the current image 110 and registered in the database. For example, the reference image 120 is a past image of the same type as the current image 110 (for example, an ultrasound image of the same mode and the same cross-section as the current image 110). However, this is only one example.

[0052] When displaying a past image as reference image 120, the information processing system 20 searches the database for a similar image to be displayed as reference image 120, based on the current image 110 and its various attribute information (hereinafter also referred to as linking information).

[0053] For example, the information processing system 20 searches for past images registered in the database that are associated with the patient ID (identification information) of the current subject, and that are associated with the same process identification information as the current process (i.e., view). This is just one example, but the process identification information may be a combination of the identification information of the protocol that includes the process in question and information that uniquely identifies that process within that protocol (for example, the order of that process within that protocol). The information processing system 20 may display the images found in this way as reference images 120.

[0054] Furthermore, for example, the information processing system 20 searches for past images of the current patient in the database that are strongly related to the current image 110. In this search, the strength of the relationship between past images and the current image 110 is determined based on, for example, the strength of the relationship between attribute information of ultrasound images and the image similarity between ultrasound images. Image similarity is the degree of similarity between images as images. The information processing system 20 may search for the image that is most strongly related to the current image 110 and display it as the reference image 120. Alternatively, as will be described in more detail later, the user may select an image from among multiple past images that are strongly related to the current image 110 and displayed in the past image area 130 and display it as the reference image 120.

[0055] The user can adjust the position and orientation of the probe 10 while referring to the displayed reference image 120. They can also evaluate the content of the displayed current image 110 while referring to it.

[0056] In the example shown in Figure 2, the size of the displayed reference image 120 is smaller than the current image 110, but this is just one example. The information processing system 20 may also have a display mode that displays the reference image 120 at the same size as the current image 110.

[0057] The database where past images are stored may be dedicated to the ultrasound diagnostic system 1, or it may be a shared database accessible from the ultrasound diagnostic system 1. In this case, the shared database may be, for example, a patient database that records past diagnostic information for each patient.

[0058] Below the current image 110, a past image area 130 is provided. In the past image area 130, a list of thumbnails (i.e., reduced images) 132 of multiple past images that are strongly related to the current image 110 is displayed. Near each thumbnail 132 (for example, above and below the thumbnail), one or more pre-selected attribute pieces of information of the past image, such as the date the past image related to that thumbnail 132 was acquired, may be displayed. The information processing system 20 can accept a selection from the user of the past image thumbnails 132 displayed in the list that they wish to display as a reference image 120. In this embodiment, we propose a process for efficiently searching for past images that are strongly related to the current image 110. This process will be explained in detail later.

[0059] Note that thumbnail 132 is smaller in size than reference image 120.

[0060] Furthermore, the display screen 100 is provided with a current image area 140. The current image area 140 displays a list of thumbnails 142 of the current image. The current image is the ultrasound image acquired during the current examination. For example, the current image is the ultrasound image acquired and saved using the probe 10 from the start of the examination of the target patient up to the present. In the case of an examination using the protocol assistant function, the current image is the ultrasound image saved in each examination step up to the present in the currently executing protocol. Near the thumbnail 142, one or more pre-selected attribute information of the current image, such as the name of the examination step in which the thumbnail 142 related to the current image was acquired, may be displayed. The information processing system 20 can accept a selection from the user of the thumbnails 142 of the current image that they wish to display as a reference image 120.

[0061] In addition to the past and current images exemplified above, standard images may also be displayed as reference images 120. Standard images are non-personal, standard images that are useful as reference in the current examination, such as ultrasound images or schematic diagrams of other people. Standard images are not images obtained from the subject themselves. The user can select which images to display as reference images 120 from, for example, past images listed in the past image area 130, or current images and a list of standard images listed in the current image area 140.

[0062] Figure 3 schematically shows a display screen 100a with a different layout from Figure 2. In display screen 100a, the current image 110a and the reference image 120a are displayed side by side at the same size. This layout is called a dual layout. In contrast, the layout of the current image 110 and reference image 120 shown in Figure 2 is called a standard layout. In the illustrated example, the size of the current image 110a in the dual layout is smaller than the size of the current image 110 in the standard layout, and the size of the reference image 120a in the dual layout is larger than the size of the reference image 120 in the standard layout.

[0063] Figure 4 schematically shows another display screen 100b with a different layout. The display screen 100b has four 2x2 reference areas 125b. This layout is called a multi-reference layout. The size of each reference area 125b is larger than the thumbnail 132 or 142, and smaller than the current image 110b. Note that the 2x2 arrangement shown is merely an example. A multi-reference layout refers to any layout with multiple reference areas 125b.

[0064] Each reference area 125b in a multi-reference layout can display a reference image 120b or other information. Examples of "other information" to be displayed in a reference area 125b include measurement data. For example, when a past image in a certain reference area 125b is displayed as a reference image 120b, one or more measurement data points (e.g., flow velocity) corresponding to that past image can be displayed in the adjacent reference area 125b. Another example of "other information" is a graph generated from measurement data (e.g., a graph showing the time-series changes in measurement values ​​obtained from multiple past measurements).

[0065] The information items (e.g., past images, measurement data, etc.) to be displayed in each reference area 125b of the multi-reference layout can be pre-configured. Furthermore, the information processing system 20 may switch the displayed content of each reference area 125b, which is displayed according to the settings, to other items based on user instructions or other factors.

[0066] The information processing system 20 receives instructions from the user regarding which of the display layouts shown in Figures 2 to 4 to use. Furthermore, the information processing system 20 accepts instructions to change the display layout while it is being displayed.

[0067] Although not shown in the diagram, the display screen 100 may also include an area for displaying measurement results or a pop-up window. The displayed measurement results may be those of the current measurement. Information showing the trend of measurement results from the past to the present (for example, a graph showing the time-series changes in the measured values) may also be displayed. This trend information may show the changes in measurement results obtained from the same patient using the same protocol and procedure.

[0068] <Linking Information> The linking information associated with ultrasound images will be explained with reference to Figures 5 to 8. This linking information consists of attribute information of the ultrasound image and is registered, for example, in the aforementioned database.

[0069] Figure 5 illustrates the items included in the linked information. The linked information illustrated in Figure 5 includes items such as image ID, patient ID, date and time of acquisition, acquisition conditions, analysis results, and user input data. The image ID is a unique identifier assigned to the ultrasound image. The patient ID is a unique identifier for the patient corresponding to that ultrasound image. The date and time of acquisition is the date and time the ultrasound image was taken. The acquisition conditions are the various conditions (e.g., device settings) at the time the ultrasound image was taken, and include several items (see Figure 6 for further details). The analysis results are data from various analyses and measurements related to the ultrasound image, and include several items (see Figure 7 for further details). The user input data is data entered by the user in relation to the ultrasound image, and includes several items (see Figure 8 for further details). Information such as image ID, patient ID, and date and time of acquisition is obtained from the control mechanism of the ultrasound diagnostic system 1 and registered in the database.

[0070] Figure 6 illustrates the group of items belonging to the shooting conditions among the linked information. As illustrated in Figure 6, the items belonging to the shooting conditions include the probe used, application, scan mode, display field of view, depth, scan width, protocol ID, process ID, etc.

[0071] "Probe used" is information indicating the type of probe 10 used to acquire the ultrasound image. Types of probes 10 include, for example, linear, sector, and convex types. Furthermore, even within the same linear type, there may be multiple probe models with different performance specifications; therefore, "Probe used" may be further subdivided to the level of such specific models for identification.

[0072] "Application" is the identifier name of the application program used to generate, process, or analyze the ultrasound image. "Display field of view" is, for example, the display field of view of the color Doppler display within the ultrasound image. "Depth" is the display width in the depth direction of the ultrasound image. "Scan width" is the width in the scan direction of the ultrasound image. "Protocol / Process ID" is identifier information representing the protocol being executed when the ultrasound image was acquired and the inspection process within that protocol. That is, in an inspection using the protocol assistant function, for an ultrasound image acquired in a certain inspection process within a certain protocol, identifier information that uniquely represents the combination of that protocol and inspection process is recorded in "Protocol / Process ID". For images acquired in an inspection that does not use the protocol assistant function, "Protocol / Process ID" will be blank.

[0073] In addition to the examples shown in the diagram, information on flow velocity settings for continuous wave mode and pulsed Doppler mode are also examples of items in the shooting conditions.

[0074] Furthermore, whether or not measurements are taken in relation to the ultrasound image (for example, blood flow velocity or blood vessel diameter) can also be considered as one of the imaging conditions.

[0075] The values ​​for each of these imaging conditions are obtained from the control mechanism of the ultrasound diagnostic system 1 and registered in the database.

[0076] Figure 7 illustrates the group of items belonging to the analysis results among the linked information. As illustrated in Figure 7, items belonging to the analysis results include cross-sectional recognition results, blood flow velocity, blood flow area, blood flow shape, presence or absence of tumor, tumor image, tumor brightness, tumor shape, etc.

[0077] "Cross-sectional recognition result" is the recognition result of the cross-section represented by the ultrasound image. This recognition result is obtained by the cross-sectional recognition analysis process described above. "Blood flow velocity" is the flow velocity of the blood flow represented in the ultrasound image. This flow velocity is obtained by the flow velocity detection analysis process described above. "Blood flow area" and "Blood flow shape" are information indicating the area and shape of the blood flow portion in the ultrasound image. This information can be obtained by known analysis processes. "Presence or absence of tumor" is information indicating whether or not the ultrasound image contains a portion corresponding to a tumor, and is obtained by the tumor detection analysis process described above. "Tumor image" is an image of the portion corresponding to a tumor in the ultrasound image, and is obtained by the tumor detection analysis process described above. "Tumor brightness" and "Tumor shape" are information indicating the brightness and shape of the portion corresponding to the tumor, and this information can be obtained by known analysis processes.

[0078] Although not shown in the diagram, if measurements other than blood flow velocity and blood flow area are performed on the ultrasound image (for example, the size of objects within the ultrasound image, such as blood vessels), the values ​​of those measurements will become one of the items in the analysis results.

[0079] Figure 8 illustrates the group of items belonging to user input data within the linked information. As illustrated in the figure, items belonging to user input data include annotations, body marks, key image flags, etc.

[0080] Annotation is a comment entered by the user on the ultrasound image. Annotation is an example of memo information entered by the user. Body marks are marks on the subject's body where probe 10 is applied. The user selects and enters a body mark from the list of body marks provided by the ultrasound diagnostic system 1 that represents the location where probe 10 is applied during the current examination. Key image flag is binary data indicating whether or not the ultrasound image is a key image. A key image is an ultrasound image that the user has determined to be important. If the user determines that the displayed ultrasound image is important for future diagnosis, for example, they can designate that ultrasound image as a key image. In response to this operation, information indicating that the ultrasound image is a key image is registered in the database.

[0081] The above describes the set of items that may be included in the linking information for ultrasound images, with reference to Figures 5 to 8. However, the set of items shown in Figures 5 to 8 is merely an example. The linking information does not need to include all of the set of items shown in those figures, and it may also include items not shown in those figures. Furthermore, individual ultrasound images do not need to have values ​​for all the items in the linking information; it is sufficient if they have values ​​for the items that were available for that particular image.

[0082] Furthermore, Figures 5 to 8 merely exemplify the groups of items included in the linking information and do not define the data structure of the linking information. For example, the grouping of items such as shooting conditions, analysis results, and user input data shown in Figures 5 to 8 is merely for explanatory purposes. The database managing the linking information does not need to adhere to such grouping. Also, the linking information may be managed in a single database or distributed across multiple databases.

[0083] <Search for related past images> Next, referring to Figure 9, we will illustrate the procedure for searching for past images that are strongly related to the current image 110.

[0084] In the procedure shown in Figure 9, the information processing system 20 controls the transmitting / receiving unit 12 to cause the probe 10 to transmit and receive ultrasound waves, and causes the image forming unit 14 to generate an ultrasound image based on the received signal obtained by this transmission and reception (S10). The ultrasound image generated in step S10 is displayed on the display screen 100 as the current image 110.

[0085] Next, the information processing system 20 performs the analysis process exemplified above on the current image 110 (S12). The information processing system 20 also obtains the values ​​of each item in the shooting conditions of the current image 110. Furthermore, if the user has entered annotations, body marks, etc., on the display screen 100, the information processing system 20 obtains them. Next, the information processing system 20 generates association information for the current image 110 (S14). This association information includes the results of the analysis process described above, the shooting conditions, and the data entered by the user.

[0086] When the user inputs an instruction to save the current image 110, the information processing system 20 registers the current image 110 and the linking information generated in step S14 in the database (S15). The linking information registered in the database at this time also includes the analysis results obtained by the analysis process described above. Furthermore, at this time, a unique image ID is assigned to the saved current image 110, and that image ID, the date and time at that time, and the patient ID of the current subject are registered in the database as part of the linking information.

[0087] Here, we have described the case where the user instructs the current image 110 to be saved, but the information processing system 20 also executes the process in step S15 in the same way when it determines to save the current image 110 according to predetermined conditions. The timing at which the information processing system 20 determines to save the current image can also be set in advance by the user. For example, the timing at which the user instructs the current image 110, which is being displayed in real time as a moving image, to be frozen (i.e., paused) may be determined as the timing to save the current image 110.

[0088] Furthermore, registration of the linked information into the database may be performed at any time after the inspection is completed (for example, at a time instructed by the user).

[0089] Furthermore, the information processing system 20 uses the linking information obtained in S14 to search the database for past images that are strongly related to the current image 110 (S16). In other words, the information processing system 20 searches the database for past images that have linking information similar to the linking information obtained in step S14, from among the past images registered in the database in association with the patient ID of the current subject.

[0090] For example, in this search, the information processing system 20 searches for past images that have a high similarity value to the value of the item in the current image 110's linking information, for items where similarity can be defined. For example, it searches for past images with a similarity of a threshold or higher, or a predetermined number of past images in descending order of similarity. In addition, for items where similarity cannot be defined, the information processing system 20 searches for past images that have a value that matches the value of the item in the current image 110's linking information.

[0091] The linked information generally includes both items for which similarity can be defined and items for which similarity cannot be defined. In this case, the information processing system 20 performs a search, for example, using the following procedure. That is, the information processing system 20 first searches for past images in the database where the value of an item for which similarity cannot be defined matches the value of that item in the current image 110. If there are multiple items for which similarity cannot be defined, the system searches for past images where the values ​​of all of those items match the values ​​of the corresponding items in the current image 110. Then, from the past images included in the search results, the information processing system 20 searches for past images with values ​​that have a high similarity to the value of the item in the current image 110 for items for which similarity can be defined. If there are multiple items for which similarity can be defined, the system searches for past images with a high overall score (i.e., strongly similar) obtained by combining the similarity scores for each of those items. In the above, if there are multiple items for which similarity cannot be defined, the system searches for past images in which all of the values ​​of those multiple items match the values ​​of the corresponding items in the current image 110. However, it is also possible to relax this condition. For example, instead of requiring all items to match, the condition could be that it is sufficient if a certain percentage or more of the items match. Alternatively, it may be predetermined which items among those for which similarity cannot be defined require a matching value and which do not. In this case, the information processing system 20 searches for past images in which the values ​​of all items for which a matching value is required match those of the corresponding items in the current image 110. Items for which a matching value is not required may be excluded from the search conditions. The information processing system 20 searches for past images that satisfy these relaxed conditions, and from the past images included in the search results, it searches for past images with values ​​that have a high similarity to the values ​​of the corresponding items in the current image 110 for items for which similarity can be defined.

[0092] Items for which similarity can be defined include, for example, quantitative data and data that can be mapped to a distance space. For example, linguistic data such as words, phrases, sentences, and texts can be mapped into a distance-definable vector space by vectorizing them using a large-scale linguistic model. For example, quantitative data such as display field of view, depth, scan width, blood flow velocity, and tumor brightness are items for which similarity can be defined. Linguistic data such as annotations are also items for which similarity can be defined. Furthermore, blood flow shape, tumor shape, and tumor images can also have their degree of similarity defined as images. In addition, while cross-sectional recognition results are represented by cross-sectional names, it is possible to define the degree of similarity between cross-sectional names, such as whether they are cross-sections of the same organ or not, and the distance between cross-sections. Therefore, cross-sectional recognition results can be treated as items for which similarity can be defined.

[0093] On the other hand, items for which similarity cannot be defined are typically nominal scale data in statistics. Nominal scale data are names or other elements that have meaning solely to distinguish them from others. For example, the probe used, scan mode, and whether or not measurement was performed are all items for which similarity cannot be defined.

[0094] Thus, in step S16, the information processing system 20 narrows down past images that are close to the current image 110 in terms of associated information.

[0095] Next, the information processing system 20 calculates the image similarity between each past image retrieved in step S16 and the current image 110 (S18). Various known methods can be used to calculate the image similarity. Among these methods, for example, there is a method that obtains feature quantities such as vectors representing the features of an image and calculates the similarity between the feature quantities between the current image 110 and the past image.

[0096] In step S18, the information processing system 20 calculates a relevance score for each of the past images based on the image similarity calculated earlier. This relevance score represents the strength of the relationship between the past image and the current image 110. For example, the stronger the relationship between the past image and the current image 110, the higher the relevance score. In one example, the relevance score may be the image similarity calculated in step S18 itself. In another example, the relevance score may be determined based on the image similarity calculated in step S18 and a combined score of the similarity of each item in the linked information described above. In calculating this combined score, points may be added or subtracted for one or more specific items in the linked information depending on the degree of similarity between the current image 110 and the past images. For example, the higher the degree of similarity, the greater the amount of points added, and if the degree of similarity falls below a certain level, points may be subtracted. As a specific example, the smaller the difference in flow rate settings between the current image 110 and the past images, the greater the amount of points added.

[0097] The information processing system 20 extracts several past images from the past images retrieved in step S16, in descending order of the relevance score calculated in step S18, as images that are strongly related to the current image 110 (S20). In this step, for example, past images with a relevance score above a predetermined threshold are extracted. In another example, in this step, a predetermined number of past images are extracted in descending order of relevance score.

[0098] Then, the information processing system 20 displays the thumbnails 132 of past images extracted in step S20 in the past image area 130 of the display screen 100, sorted in descending order of relevance score (S22).

[0099] Figure 10 schematically shows an example of the past image region 130 displayed in step S22. In this example, within the past image region 130, thumbnails 132 of past images extracted in step S20 are arranged from left to right in order of decreasing relevance score. The example past image region 130 contains thumbnails 132 of past images of the liver of a particular subject that are strongly related to the current image 110, which represents a specific cross-section of the liver of the same subject. In the illustrated example, the relevance scores of the past images represented by these thumbnails 132 with respect to the current image 110 are 95%, 90%, 73%, 64%, ... from left to right. Although not shown in the illustration, the values ​​of one or more specific items of the linking information, such as the date the past image represented by the thumbnail 132 was taken, may be displayed in the vicinity of each thumbnail 132 (for example, below the thumbnail 132).

[0100] The information processing system 20 may, when displaying thumbnails 132 of past images within the past image area 130 according to the procedure in Figure 9, display the past image with the highest relevance score among them as a reference image 120 on the display screen 100.

[0101] When a key image is considered, step S22 may be the procedure shown in Figure 11, for example. That is, the information processing system 20 determines whether or not a key image is included in the past images extracted in step S20 (S220). This determination can be made by checking the key information flag (see Figure 8) of the linking information of those past images. If the result of the determination in step S220 is Yes, the information processing system 20 places the key image from those past images at the beginning of the past image area 130, and then displays the other past images arranged in descending order of relevance score (S222).

[0102] Figure 12 shows an example of the past image region 130 displayed by step S222. In the example in Figure 12, the thumbnail 132k of the key image is placed at the left edge of the past image region 130, and to its right, thumbnails 132 of other past images are arranged in descending order of relevance score.

[0103] Thus, in step S222, regardless of the relevance score value, the thumbnail 132k of the key image is placed at the beginning of the past image region 130.

[0104] If the result of step S220 is No, the information processing system 20 performs the same processing as in step S22 in Figure 9. That is, the information processing system 20 displays the thumbnails 132 of past images extracted in step S20 in the past image area 130 in descending order of relevance score (S224).

[0105] As explained above, in the examples of Figures 11 and 12, the thumbnail 132k of the key image is placed at the beginning of the past image area 130 (the leftmost position in Figure 12), making it easy to find.

[0106] In this example, the past image corresponding to the thumbnail placed at the beginning of the past image area 130 in step S222 may also be automatically displayed as the reference image 120.

[0107] <Example of control based on whether or not the Protocol Assistant function is used> A partial modification of the procedure in Figure 9, applicable when considering the protocol assistant function, will be explained with reference to Figure 13. Figure 13 shows the part that can be replaced with step S22 in Figure 9.

[0108] The information processing system 20, for example, after step S20, determines whether or not the protocol assistant function is being used in the current inspection (S230). If the result of the determination in step S230 is Yes, the information processing system 20 identifies a past image from the past images obtained by the search in step S16 that has the same protocol / process ID as the protocol / process ID of the inspection process currently being executed (S232). The information processing system 20 then places the thumbnail of the past image identified in step S232 at the beginning of the past image area 130, and then displays the thumbnails 132 of other past images extracted in step S20 in descending order of relevance score (S234). If multiple past images are identified in step S232, the information processing system 20 arranges these identified past images from the beginning of the past image area 130 in descending order of relevance score, and then arranges the other past images after them.

[0109] To avoid complexity, the diagram has been omitted, but it is possible that in step S232, a past image with the same protocol and process ID as the currently executed inspection process cannot be identified. In this case, in step S234, the information processing system 20 simply displays the thumbnails 132 of the past images extracted in step S20 in the past image area 130, sorted by relevance score from highest to lowest.

[0110] If the protocol assistant function is not used (i.e., the result of the determination in step S230 is No), the information processing system 20 displays the thumbnails 132 of the past images extracted in step S20 in the past image area 130 in descending order of relevance score (S236).

[0111] Furthermore, when considering both the key image and the protocol assistant function, step S22 in Figure 9 may be modified as illustrated in Figure 14. In Figure 14, steps similar to those included in the procedure in Figure 13 are denoted by the same reference numerals. Also, Figure 15 shows an example of the past image region 130 displayed by step S242.

[0112] In this example, if the information processing system 20 is currently using the protocol assistant, it identifies past images with the same protocol / process ID as the currently running inspection process from the search results of step S16 (S232). Furthermore, the information processing system 20 determines whether or not there is a key image among the past images extracted in step S20 (S240). If the result of this determination is Yes, the information processing system 20 places the thumbnail 132p of the past image identified in step S232 at the front, starting from the left edge of the past image area 130, and places the key image 132k found in step S240 to its right (see Figure 15). Then, it displays the thumbnails 132 of the other past images extracted in step S20 arranged in descending order of relevance score (S242).

[0113] If the result of step S242 is No, the information processing system 20 places the thumbnail 132p of the past image identified in step S232 at the beginning, starting from the left edge of the past image area 130. Then, it displays the thumbnails 132 of the other past images extracted in step S20 in descending order of relevance score (S244).

[0114] If it is determined in step S230 that the protocol assistant function is not being used, the information processing system 20 executes the processing procedure shown in Figure 11 (S246). Furthermore, additional examinations may be performed while a certain examination step of a protocol is being executed using the protocol assistant function. For example, a user who has observed an ultrasound image obtained during the currently running examination step may interrupt the execution of the protocol and instruct an additional examination not included in that examination step (e.g., ultrasound imaging in a different mode) in order to investigate further. In such cases, the information processing system 20 may assign a higher score to an ultrasound image of an additional examination linked to the same protocol / step ID as the interrupted examination step, if such an image is found among past images obtained from previous examinations of the same patient. Since additional examinations performed in the past using the same examination step are likely to examine the same area or lesion as the current additional examination, assigning a higher score to the image of the past additional examination can increase the display ranking of its thumbnail within the past image area 130.

[0115] <Examples of control strategies that take tumors into consideration> Next, a modified version of the processing procedure in Figure 9 will be explained with reference to Figure 16. In this modified version, the image similarity between tumor portions between the current image 110 and past images is also taken into consideration when ranking the search results. Figure 16 shows the processing of the part of the procedure in Figure 9 from step S18 onwards.

[0116] In this modified example, the information processing system 20 performs tumor detection on the current image 110 using a system such as CAD (S30). If no tumor-like portion is detected in the current image 110 during this tumor detection (the result of the determination in S32 is No), the information processing system 20 calculates the relevance score for each past image based on the image similarity calculated in step S18 and the search score which is a combination of the similarity of each item of the aforementioned linking information (S34). Next, the information processing system 20 extracts several past images from the past images retrieved in step S16, in descending order of relevance score, as images that are strongly related to the current image 110 (S20a). Then, the information processing system 20 displays the thumbnails 132 of the past images extracted in step S20a in the past image area 130 of the display screen 100, sorted in descending order of relevance score (S22).

[0117] If the result of step S32 is Yes, the information processing system 20 calculates the image similarity between each past image searched in step S16 and the tumor portion in the current image 110 detected in S30 (S36). In this calculation, for example, for each searched past image, the system searches for the portion in that past image that is most similar to the tumor portion, and then calculates the image similarity between the portion found through the search and the tumor portion in the current image 110. This calculation can be performed, for example, by using a template matching method that uses the image of the tumor portion in the current image 110 as a template.

[0118] Next, the information processing system 20 calculates a relevance score for each past image based on the search score which is a combination of the image similarity of the entire image calculated in step S18, the image similarity of the tumor portion calculated in step S36, and the similarity of each item of the aforementioned linking information (S38). The information processing system 20 then extracts several past images in descending order of relevance score (S20a), and displays the thumbnails 132 of these extracted past images in the past image area 130 in descending order of relevance score (S22).

[0119] <Example considering moving images> The current image 110 displayed on the display screen 100 is typically a moving image. The user can make the current image 110 on the display screen 100 a still image by pausing the moving image or by frame-by-frame selecting a specific frame. In one example, the information processing system 20 performs the procedure in Figure 9 on the current image 110 as a still image determined by the user in this way. Alternatively, the information processing system 20 may select a representative frame from the displayed moving image and perform the procedure in Figure 9 with the selected frame as the current image 110.

[0120] Furthermore, the information processing system 20 may have a function to register moving images in a database. For example, if the user instructs the system to register the currently displayed moving image in the database, the information processing system 20 will execute the procedure shown in Figure 17.

[0121] In the procedure shown in Figure 17, the information processing system 20 calculates a feature score for each frame of the video image to be registered in the database (S110). The feature score is an index value that indicates the degree to which the image in that frame contributes to the diagnosis.

[0122] For example, the amount of blur in the frame's image could be included as one of the features. In this case, the feature score should be defined such that the less blur there is, the higher the feature score of the frame.

[0123] Alternatively, the recognition rate of organs from the frame image may be included as one of the features. The recognition rate of organs can be determined, for example, by scene analysis. In this case, the feature score should be defined such that the higher the recognition rate, the higher the feature score of the frame. Alternatively, the degree of missing organ parts in the frame image may be included as one of the features. In this case, the feature score should be defined such that the greater the missing parts, the higher the feature score of the frame. Missing organ parts in an image can occur, for example, due to bone shading.

[0124] Alternatively, whether or not a frame contains a portion with tumor characteristics may be included as one of the features. In this case, the feature score should be defined such that the feature score is higher when a portion with tumor characteristics is included than when it is not. The portion with tumor characteristics can be identified, for example, by an AI-based tumor detection function such as the aforementioned CAD. Alternatively, the size of the portion with tumor characteristics within the frame may be included as one of the features. In this case, the feature score should be defined such that the larger the portion with tumor characteristics, the higher the feature score. Furthermore, the confidence level for detecting the portion with tumor characteristics may be included as one of the features. For this confidence level, the confidence level output by the aforementioned tumor detection function when it detects a portion with tumor characteristics can be used.

[0125] Alternatively, blood flow sensitivity may be included as one of the features. The feature score should be defined such that frames with good blood flow sensitivity have higher feature scores. Furthermore, the amount of blood flow aliasing may also be included as one of the features. The feature score should be defined such that frames with less blood flow aliasing have higher feature scores. Blood flow sensitivity and the amount of blood flow aliasing can be determined, for example, by evaluating the color distribution of blood flow information shown in a color flow image or by analyzing the texture.

[0126] The above outlines some feature items that form the basis of feature scores, along with some examples of the relationship between the values ​​of these items and the feature scores. These are merely illustrative examples. It is not necessary to use all of these items. Furthermore, one or more items other than those listed may also be reflected in the feature scores.

[0127] Once the feature score for each frame in the video is calculated, the information processing system 20 selects the frame with the highest feature score as the representative image (S112). The representative image is a still image that represents the video. The information processing system 20 performs the above-described analysis on the representative image and generates association information that includes the results of the analysis (S114). For items of the association information that have already been calculated in step S110, the results of that calculation are reused.

[0128] The information processing system 20 then registers the video images to be registered, the representative images selected in step S112, and the linking information generated in step S114 in the database, associating them with each other (S116).

[0129] When considering the case where video is registered in the database in this manner, in the past image search process in the procedure of Figure 9, a representative image is processed instead of the video. That is, in step S18 of Figure 9, the information processing system 20 calculates the image similarity between the current image 110 and the representative image for entries in the database where a set of video, representative image, and linking information is registered (hereinafter referred to as a video entry), and calculates a relevance score based on that image similarity. Furthermore, if the representative image is extracted in step S20 based on the relevance score calculated in this way, a thumbnail of that representative image is displayed in the past image area 130 in step S22. Furthermore, if the user selects a thumbnail of a representative image displayed in the past image area 130, the information processing system 20 displays that representative image or its corresponding video as a reference image 120. Whether to display a representative image or a video as the reference image 120 can be switched according to the user's instructions.

[0130] Furthermore, multiple organs may be displayed within a single video. In such cases, the frame that best represents the characteristics of each organ often differs for each organ. Therefore, a representative image may be selected individually for each of these multiple organs. For example, for each organ, the frame with the highest feature score for that organ may be selected as the representative image. Note that each individual organ displayed within each frame of the video can be recognized through the analysis process described above.

[0131] <Other variations> Further variations will be explained with reference to Figure 18. The procedure illustrated in Figure 18 is executed when the information processing system 20 displays a new reference image 120 or switches the reference image 120 to a different image.

[0132] In the procedure shown in Figure 18, the information processing system 20 determines whether measurement information is included in the linking information corresponding to the displayed reference image 120 (S300). The measurement information includes, for example, information indicating the item measured (i.e., the type of measurement) and the value of the measurement result. If the determination result in step S300 is Yes, the information processing system 20 starts a process to perform the measurement corresponding to that measurement information for the current inspection (S302). In one example, the process for performing the measurement is the measurement of the item indicated by the measurement information itself. In another example, the process for performing the measurement may include other processes necessary to perform the measurement of that item (for example, inputting information necessary for the measurement).

[0133] In step S302, the information processing system 20 automatically performs measurements of the measurement items indicated by the measurement information. For example, if the current image 110 is a blood flow velocity waveform in PW mode, and the measurement item included in the measurement information is the maximum value of blood flow velocity, the information processing system 20 determines the maximum value of blood flow velocity indicated by the waveform of the current image 110.

[0134] In another example, in step S302, the information processing system 20 performs a process to prompt the user for input of information necessary to perform the measurement of that item. Once the necessary information is input, the information processing system 20 uses that information to perform the measurement and records the measurement result.

[0135] For example, if the linking information of reference image 120 includes the measurement result of the tissue length, the information processing system 20 displays reference points on the current image 110 to identify both ends of the area to be measured. The user moves these reference points to the appropriate positions. After the positions of the reference points are set appropriately, the information processing system 20 measures and records the distance between the two reference points.

[0136] If the result of step S300 is No, the information processing system 20 skips step S302 and terminates the process.

[0137] Further variations will be explained with reference to Figure 19. Figure 19 shows an example of the procedure by which the information processing system 20 automatically selects the layout of the display screen 100.

[0138] As a prerequisite for this procedure, cross-sections are classified into two types: Type 1 and others. If it is stipulated that measurements be performed on the ultrasound image of a cross-section, that cross-section falls under Type 1. Furthermore, if it is stipulated that the shape of the ultrasound image of a cross-section be compared with past images of the same cross-section, that cross-section also falls under Type 1. Whether or not each cross-section falls under Type 1 is registered in advance in the information processing system 20.

[0139] The procedure in Figure 19 is performed, for example, after the analysis process in step S12 of the procedure in Figure 9.

[0140] In this procedure, the information processing system 20 determines whether the cross-sectional recognition result for the current image 110 (which is obtained by the analysis process) corresponds to type 1 (S400). If the result of this determination is Yes, the information processing system 20 selects a dual layout (see Figure 3) as the layout for the display screen 100 (S402).

[0141] On the other hand, if the result of step S400 is No, the information processing system 20 determines whether or not findings have been generated for the current image 110 through analysis processing (S404). If the result of the determination in step S404 is Yes, the information processing system 20 selects a dual layout (S402).

[0142] Furthermore, if the result of step S404 is No, the information processing system 20 determines whether the current mode of the ultrasound diagnostic system 1 is a specific mode (S406). If the result of this determination is Yes, the information processing system 20 selects a dual layout (S402).

[0143] The "specific modes" referred to here include continuous wave Doppler mode, pulsed Doppler mode, powered Doppler mode (e.g., eFlow), TDI (Tissue Doppler Imaging) mode, DFI (Detective Flow Imaging) mode, etc. For example, in continuous wave Doppler and pulsed Doppler modes, waveforms showing time-series changes in blood flow velocity are displayed, and useful information can be obtained by comparing these waveform images with past waveform images. Furthermore, for images obtained with powered Doppler, TDI, and DFI, it is important to examine how the shape and texture represented by the image have changed from the past. In the normal layout (Figure 2), where the reference image 120 is smaller than the current image 110, it is difficult to compare the current image 110 with the past image, the reference image 120. For these reasons, when using these specific modes, a dual layout is adopted to facilitate comparison between the current image 110 and the reference image 120.

[0144] If the result of step S406 is No, the information processing system 20 selects the normal layout (see Figure 2) as the layout for the display screen 100 (S408).

[0145] After step S402 or S404, the information processing system 20 generates and displays the display screen 100 of the selected layout.

[0146] In this modified version, when the current image 110 of a cross-section requiring measurement or comparison with past images is displayed, it is displayed in a dual layout. In the dual layout, the reference image 120 is displayed larger than in the normal layout, making it easier to see the position of the measurement cursor within the reference image 120. Also, in the dual layout, the current image 110 and the reference image 120 are displayed at the same size, making it easier for the user to compare the shapes of the two.

[0147] Furthermore, in this modified version, if a tumor or other abnormality is detected in the current image 110, the reference image 120 and the current image 110 are displayed at the same size, making it easier to compare the reference image 120 and the current image 110. This comparison allows for recognition of how the size and characteristics of the tumor or other abnormality have changed from the state shown in the reference image 120 at a past point in time.

[0148] In this example, when findings are generated, a dual layout display is performed. When findings are generated by the information processing system 20, it means that some part of the current image 110 is found to be suspected of being abnormal. In such cases, by displaying the reference image 120 in a dual layout at the same size as the current image 110, the user can compare the two images to check for the presence or absence of abnormalities and their condition.

[0149] In this embodiment, each process is executed on any computer. Furthermore, any computer may execute these processes using a processor as hardware, a program as software, or a combination thereof. In that case, the processor is configured to work in cooperation with the program to execute the various processes in this embodiment, and can function as a unit or means in this embodiment. Also, the execution order of the processes by the processor is not limited to the order described and may be changed as appropriate. Any computer may be a general-purpose computer, a computer designed for a specific purpose, a workstation, or any other system capable of executing each process.

[0150] A processor may consist of one or more hardware components, and the type of hardware is not limited. For example, a processor may consist of a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a programmable logic device such as an FPGA (Field Programmable Gate Array), a dedicated circuit for executing a specific process such as an ASIC (Application Specific Integrated Circuit), a GPU (Graphic Processing Unit), or an NPU (Neural Processing Unit). Furthermore, the type of hardware may be a combination of different types of hardware. When multiple hardware components are configured to execute one or more processes of a given processor, these components may reside in physically separate devices or in the same device. Also, in any embodiment, the order of each process performed by the processor is not limited to the order described above and may be changed as appropriate. Hardware is composed of electrical circuits (circuitry) that combine circuit elements such as semiconductor elements.

[0151] Furthermore, the program may be firmware or software such as microcode. Alternatively, the program may be, for example, a set of program modules, each function of which may be implemented by a processor configured to perform its respective function. The program may be program code or multiple code segments stored on one or more non-temporary computer-readable media (e.g., storage media or other storage). The program may be divided and stored on multiple non-temporary computer-readable media located on physically separate devices. Program code or code segments may represent any combination of procedures, functions, subprograms, routines, subroutines, modules, software packages, classes, or instructions, data structures, or program statements. Program code or code segments may be connected to other code segments or hardware circuits by sending and receiving information, data, arguments, parameters, or memory contents. [Explanation of Symbols]

[0152] 1 Ultrasound diagnostic system, 10 Probe, 12 Transmitter / Receiver, 14 Image forming unit, 16 Display processing unit, 18 Display device, 20 Information processing system.

Claims

1. Equipped with a processor, The aforementioned processor, A first ultrasound image is generated from the signal received by the ultrasound probe. The process for displaying the ultrasound image in Item 1 is executed. Perform the analysis of the first ultrasound image described above to obtain the analysis results. A search is performed for past ultrasound images that have the same or similar analysis results as the analysis results for the first ultrasound image described above. For each of the past ultrasound images obtained as a result of the search, a score representing the similarity to the first ultrasound image is calculated. Display control is performed to display one or more images selected from the past ultrasound images obtained as a result of the search, based on the score, together with the first ultrasound image. An ultrasound diagnostic system characterized by the following features.

2. The ultrasound diagnostic system according to claim 1, wherein the analysis is a process of obtaining a cross-sectional recognition result as the analysis result, which indicates which of a plurality of predetermined cross-sections the first ultrasound image is an image of.

3. The first ultrasound image is an image of the blood flow velocity waveform obtained by the Doppler method. The ultrasound diagnostic system according to claim 1, wherein the analysis is a process of determining the maximum flow velocity of the blood flow velocity waveform as the analysis result.

4. The aforementioned processor, The process of detecting the tumor portion from the first ultrasound image is performed. In calculating the score, the score between the past ultrasound image and the first ultrasound image is calculated based on a first similarity between the entire past ultrasound image and the first ultrasound image, and a second similarity between the tumor portions. The ultrasound diagnostic system according to claim 1.

5. The aforementioned processor, The ultrasound diagnostic system according to claim 1, characterized in that when saving the first ultrasound image, the analysis result obtained by the analysis is saved as one of the attribute information items of the first ultrasound image.

6. The aforementioned processor, If the first ultrasound image is a moving image, the moving image, a representative still image from the moving image, and the analysis results obtained from the analysis of the still image are stored. The ultrasound diagnostic system according to claim 1.

7. The aforementioned processor, In the search described above, memo information entered by the user for the displayed first ultrasound image is obtained, and past ultrasound image data that has the same or similar analysis results as the analysis result and has the same or similar memo information as the obtained memo information is searched. The ultrasound diagnostic system according to claim 1.

8. The aforementioned processor, The ultrasound diagnostic system according to claim 1, characterized in that, in the display control, a process is performed to display thumbnails of a plurality of images selected from the past ultrasound images obtained as a result of the search, based on the score, in a thumbnail display field provided near the first ultrasound image, arranged in descending order of the score.

9. The aforementioned processor, In the display control described above, if the image designated as the key image is among the multiple images selected based on the score from the past ultrasound images obtained as a result of the search, the thumbnail of the image designated as the key image is displayed at the top of the thumbnail display area, regardless of the score for that image. The ultrasound diagnostic system according to claim 8, characterized in that it is as described above.

10. The aforementioned processor, When a protocol assistant function is used to assist in the execution of a protocol consisting of one or more inspection steps, the display control displays a thumbnail of a past ultrasound image obtained in the same inspection step of the same protocol as the currently executed protocol at the top of the thumbnail display field, and then displays a thumbnail of the image designated as the key image in the next position. The ultrasound diagnostic system according to feature 9.

11. The aforementioned processor, When a protocol assistant function is used to assist in the execution of a protocol consisting of one or more inspection steps, the display control displays a thumbnail of a past ultrasound image obtained in the same inspection step of the same protocol as the currently executed protocol at the top of the thumbnail display area. The ultrasound diagnostic system according to claim 8, characterized in that it is as described above.

12. The aforementioned processor, The ultrasound diagnostic system according to claim 1, wherein in the display control, the past ultrasound image with the highest score is displayed alongside the first ultrasound image as a reference image for the first ultrasound image.

13. The aforementioned processor, If information is obtained indicating that measurements were performed on the past ultrasound image displayed as the reference image, the process for performing the measurements on the displayed first ultrasound image is executed. The ultrasound diagnostic system according to claim 12, characterized in that it is the same as described above.

14. The aforementioned processor, Based on the cross-sectional recognition results, the layout for displaying the first ultrasound image and the reference image selected from past ultrasound images is selected. The ultrasound diagnostic system according to claim 2, characterized in that it is the same as described in claim 2.

15. The aforementioned processor, If the cross-sectional recognition result corresponds to a predetermined first type of cross-section, a dual layout is selected in which the first ultrasound image and the reference image are placed side by side at the same size. The ultrasound diagnostic system according to claim 14.

16. The aforementioned processor, When findings are generated by the above analysis, a dual layout is selected for displaying the first ultrasound image and the reference image selected from past ultrasound images, in which the first ultrasound image and the reference image are displayed side by side at the same size. The ultrasound diagnostic system according to claim 1.

17. The aforementioned processor, When the mode of the ultrasound diagnostic system is in a specific mode when the first ultrasound image is generated, a dual layout is selected in which the first ultrasound image and the reference image selected from past ultrasound images are displayed side by side at the same size. The ultrasound diagnostic system according to claim 1.

Images