Assisted medical imaging segmentation

Abstract

This document discloses a medical system (100, 300, 400). Execution of machine-executable instructions (120) causes a computing system (110) to repeatedly perform the following operations: receive (204) segmentation editing data (130) from a segmentation entry tool (128); collect (206) location-specific segmentation tool usage metadata (134) describing the use of the segmentation entry tool to enter segmentation editing data at the editing location; determine (208) an anatomical location by using the editing location; retrieve (210) expected segmentation tool usage metadata from a segmentation tool usage database (122) by using the anatomical location; determine (212) the presence of anomalous segmentation editing data (806) by comparing the location-specific segmentation tool usage metadata with the expected segmentation tool usage metadata; provide (214) signals (142, 404, 808) if anomalous segmentation editing data is determined to exist; and construct (216) a segmentation result of a medical image (144) by using the segmentation editing data.

Description

Technical Field

[0001] This invention relates to medical imaging, and more particularly to the segmentation of medical images. Background Technology

[0002] Various imaging modalities exist that enable the acquisition of medical images describing the internal anatomy of an object. For example, magnetic resonance imaging (MRI), computed tomography (CT), positron emission tomography (PET), and single-photon emission tomography (SPECT) can be used to image an object. Medical images are often segmented. In segmentation, various normal and / or abnormal regions are identified and delineated. Segmentation can be extremely useful for things like tracking tumor growth over time or controlling the amount of radiation delivered to different organs during radiotherapy planning. Summary of the Invention

[0003] The present invention provides a medical system, a computer program, and a method in the independent claims. Embodiments are given in the dependent claims.

[0004] In one aspect, the present invention provides a medical system according to claim 1. Specifically, the medical system includes a memory storing both machine-executable instructions and a segmentation tool usage database. The segmentation tool usage database includes expected segmentation tool usage metadata indexed according to anatomical locations. An anatomical location is a region adjacent to or within an anatomical atlas reference location. The medical system also includes a computing system. Execution of the machine-executable instructions causes the computing system to receive medical images.

[0005] The execution of machine-executable instructions also causes the computing system to render the medical system on a user interface. The user interface includes a segmentation entry tool. The segmentation entry tool is a tool used to enter, manipulate, or edit the segmentation results of a medical image. The execution of the machine-executable instructions also causes the computing system to repeatedly receive segmentation editing data from the segmentation entry tool. Segmentation metadata includes editing locations within the medical image. Editing locations are the neighborhood surrounding the segmentation entry tool. The execution of the machine-executable instructions also causes the computing system to repeatedly collect location-specific segmentation tool usage metadata, which describes the use of the segmentation entry tool to enter segmentation editing data at the editing location. In other words, location-specific segmentation tool usage metadata is collected, wherein such location-specific segmentation tool usage metadata describes the use of the segmentation entry tool during the entry of segmentation editing data within the editing location.

[0006] The segmentation tool uses metadata including any of the following: maximum segmentation tool speed, minimum segmentation tool speed, average segmentation tool speed, segmentation tool position, dwell time of the segmentation tool in the editing position, overall annotation speed, mouse speed, visual gaze tracking, visual gaze tracking speed, visual gaze speed, pause duration of segmentation tool use, number of pauses during segmentation tool use, variation in segmentation tool speed, segmentation tool size, use of the segmentation tool's eraser function, and combinations thereof.

[0007] The execution of machine-executable instructions also enables the computing system to repeatedly determine anatomical locations by using edit locations.

[0008] The execution of machine-executable instructions also enables the computational system to repeatedly retrieve the expected segmentation tool's metadata from the segmentation tool's database using anatomical locations. Furthermore, the execution of these machine-executable instructions enables the computational system to repeatedly determine the presence of anomalous segmentation edit data by comparing the location-specific segmentation tool's metadata with the expected segmentation tool's metadata. The execution of these machine-executable instructions also enables the computational system to repeatedly provide signals when anomalous segmentation edit data is determined to exist. Finally, the execution of these machine-executable instructions enables the computational system to repeatedly construct the segmentation result of the medical image using the segmentation edit data.

[0009] In another aspect, the present invention provides a method for medical imaging according to claim 11. The method includes receiving a medical image. The method also includes drawing the medical image on a user interface. The user interface includes a segmentation entry tool. The segmentation entry tool is a tool for entering, manipulating, or editing segmentation results of a medical image. The method also includes repeatedly receiving segmentation editing data from the segmentation entry tool. The segmentation editing data includes edit locations within the medical image. Edit locations are neighborhoods surrounding the segmentation entry tool. The method also includes repeatedly collecting location-specific segmentation tool usage metadata, which describes the use of the segmentation entry tool during the entry of segmentation editing data within the edit locations. The segmentation tool usage metadata includes any of the following: maximum segmentation entry tool speed, minimum segmentation entry tool speed, average segmentation entry tool speed, segmentation entry tool position, dwell time of the segmentation entry tool in the editing position, overall annotation speed, mouse speed, visual gaze tracking, visual gaze tracking speed, visual gaze speed, pause duration of segmentation entry tool usage, number of pauses during segmentation entry tool usage, variation in segmentation entry tool speed, segmentation entry tool size, use of the segmentation entry tool's eraser function, and combinations thereof. The method also includes repeatedly determining anatomical locations by using editing locations. Anatomical locations are regions adjacent to or within anatomical atlas reference locations. The method also includes repeatedly retrieving expected segmentation tool usage metadata from a segmentation tool usage database by using anatomical locations. The segmentation tool usage database includes expected segmentation tool usage metadata indexed according to anatomical locations.

[0010] The method also includes repeatedly determining the presence of anomalous segmentation edit data by comparing the metadata used by the location-specific segmentation tool with the metadata used by the expected segmentation tool. The method also includes repeatedly providing a signal when anomalous segmentation edit data is determined to exist. The method further includes repeatedly constructing the segmentation result of the medical image using the segmentation edit data.

[0011] In another aspect, the present invention provides a computer program according to claim 12, the computer program including machine-executable instructions and also including a tool usage database. The segmentation tool usage database includes expected segmentation tool usage metadata indexed according to anatomical locations. Anatomical locations are regions adjacent to or within anatomical atlas reference locations. Execution of the machine-executable instructions causes the computing system to receive medical images.

[0012] The execution of machine-executable instructions also enables the computing system to render medical images on a user interface. The user interface includes a segmentation entry tool. The segmentation entry tool is used to enter, manipulate, or edit the segmentation results of a medical image. The execution of the machine-executable instructions also enables the computing system to repeatedly receive segmentation editing data from the segmentation entry tool. The segmentation editing data includes edit locations within the medical image. Edit locations are the neighborhood surrounding the segmentation entry tool. The execution of the machine-executable instructions also enables the computing system to repeatedly collect location-specific segmentation tool metadata, which describes the use of the segmentation entry tool during the entry of segmentation editing data within the edit location. The segmentation tool usage metadata includes any of the following: maximum segmentation tool speed, minimum segmentation tool speed, average segmentation tool speed, segmentation tool position, dwell time of the segmentation tool at the editing position, overall annotation speed, mouse speed, visual gaze tracking, visual gaze tracking speed, visual gaze speed, pause duration of segmentation tool usage, number of pauses during segmentation tool usage, variation in segmentation tool speed, segmentation tool size, use of the segmentation tool eraser function, and combinations thereof. The execution of machine-executable instructions also causes the computational system to repeatedly determine anatomical locations by using the editing positions. The execution of machine-executable instructions also causes the computational system to repeatedly retrieve expected segmentation tool usage metadata from the segmentation tool usage database by using the anatomical locations.

[0013] The execution of machine-executable instructions also enables the computational system to determine the presence of anomalous segmentation edit data by comparing the location-specific segmentation tool with the expected segmentation tool using metadata. Furthermore, the execution of machine-executable instructions allows the computational system to provide a signal when anomalous segmentation edit data is determined to exist. Finally, the execution of machine-executable instructions enables the computational system to construct segmentation results for medical images using the segmentation edit data. Attached Figure Description

[0014] In the following description, preferred embodiments of the invention will be illustrated by way of example only and with reference to the accompanying drawings, in which: Figure 1 An example of a medical system is illustrated.

[0015] Figure 2 The illustration shows the use of Figure 1 A flowchart of the methods used in medical systems.

[0016] Figure 3 The illustration shows another example of a medical system.

[0017] Figure 4 The illustration shows another example of a medical system.

[0018] Figure 5 An example of an image classifier neural network is illustrated.

[0019] Figure 6 Another example of an image classifier neural network is illustrated.

[0020] Figure 7 The diagram illustrates several views of the user interface.

[0021] Figure 8 The illustration shows the detection of abnormal data segmentation.

[0022] List of reference numerals in the attached diagram: 100 Medical Systems 102 Computer 104 Computing System 106 Hardware Interfaces 108 User Interface 110 Memory 120 Machine-executable instructions 122 The splitting tool uses a database. 124 Medical Images 124' Cropped medical image 126 User Interface 128 Segment Input Tool 130 Segment Edit Data 134 Location-Specific Segmentation Tool Uses Metadata 136 Anatomical Location 138 Expected Segmentation Tool Uses Metadata Comparison of metadata used by location-specific segmentation tools and metadata used by expected segmentation tools (140) 142 signal 144 Segmentation Results 146 Image classifiers or image classifier neural networks 146' Image Classifier Neural Network 148 Edge Detection Algorithms 150 detected edges 152 Detected edge smoothness 154 Segmentation result smoothness 156 Segmentation Module 158 Segmentation Instruction Database 160 split command 170 Editing position 172 Command Box 200 Receiving Medical Images 202 Drawing medical images on the user interface 204 Receive segmentation editing data from the segmentation input tool 206. Collection of location-specific segmentation tools uses metadata that describes the use of the segmentation entry tool during the entry of segmentation editing data within the editing location. 208. Determining anatomical location by using edit location. 210 Using anatomical locations to retrieve the expected segmentation tool from the database using metadata. 212 Determine whether there is anomalous segmentation edit data by comparing the metadata used by the location-specific segmentation tool with the metadata used by the expected segmentation tool. 214 Provide a signal when abnormal data splitting and editing is determined. 216 Segmentation results of medical images constructed by using segmentation-edited data. 300 Medical System 302 Magnetic Resonance Imaging System 304 magnet 306 The bore of the magnet 308 Imaging Area 309 Field of view 310 Magnetic Gradient Coil 312 Magnetic Gradient Coil Power Supply 314 RF coil 316 transceiver 318 Objects 320 Object support 330 Pulse Sequence Command 332 k spatial data 334 Magnetic Resonance Imaging Images 400 Medical System 402 Initial segmentation result 404 pointer 700 User Interface View 702 User Interface View 705 Message Indicator 704 First Message Box 706 Second Message Box 800 images 802 image 804 image 806 Problem Area 808 message (signal) Detailed Implementation

[0023] Elements with the same numbers in these figures are equivalent elements or perform the same function. If the functions are equivalent, elements that have been discussed previously will not necessarily be discussed in later figures.

[0024] In the example, the medical system includes a storage device containing machine-executable instructions and a database of segmentation tool usage. The segmentation tool usage database includes metadata indexed according to anatomical location regarding the intended segmentation tool usage.

[0025] Segmentation tools utilize databases, for example, historical data containing information and prior knowledge about the problem being addressed. In the case of left ventricular segmentation, such databases contain information about difficult and ambiguous regions that are typically hard to delineate. These databases can be built upon clinical expert knowledge about the specific difficulties in labeling the problem, for example, through expert interviews or even by collecting and archiving data while using medical systems.

[0026] As used in this article, a segmentation tool can encompass tools or control elements for the user interface used to input or edit segmentation results. Various metadata about the segmentation tool can be collected. For example, the speed at which the segmentation tool is used can be monitored; the screen zoom level can be monitored.

[0027] Additionally, when entering specific segmentation results, activities performed using the segmentation tool, such as erasing or modifying the segmentation results, can be used to categorize how the segmentation tool is used. The metadata expected to be used by the segmentation tool could be, for example, the average or range of specific metadata typical of a particular anatomical location. This can be beneficial because various anatomical locations may have different levels of complexity for segmentation. This means that, due to the typical segmentation complexity for an anatomical location, the metadata when manually entering segmentation results may also have certain characteristics, such as being very complex or detailed due to the anatomical location, and then this situation may be repeatedly encountered by the same user or different users in segmentation.

[0028] The medical system also includes a computing system. A computing system can represent one or more computing systems located at one or more locations. Execution of machine-executable instructions causes the computing system to receive medical images. Medical images can be, for example, computed tomographic images. Medical images can show or detail the internal anatomical structures of an object. Execution of the machine-executable instructions also causes the computing system to render the medical images on a user interface. The user interface includes segmentation input tools. As described above, segmentation input tools are tools that can be used to input, manipulate, or edit the segmentation results of medical images. As used herein, segmentation results are lines or surfaces used to divide a medical image into anatomically significant regions. For example, segmentation results can often be used to divide a medical image into different tissue types or organs. Sometimes, segmentation results can be used to identify the location or extent of abnormal anatomical structures, such as tumors or lesions.

[0029] The execution of machine-executable instructions also causes the computational system to repeatedly receive segmentation editing data from the segmentation entry tool. Segmentation editing data includes edit locations within the medical image. Segmentation editing data is data that can be used to construct or modify the segmentation result. A location within the medical image is a location identified within the medical image itself. For example, this could include the coordinates of voxels or pixels being edited near the segmentation result being modified or edited. The execution of machine-executable instructions also causes the computational system to repeatedly collect location-specific segmentation tool usage metadata, which describes the use of the segmentation entry tool to enter segmentation editing data at the edit location. Segmentation results for anatomical regions or abnormal structures within a medical image may have different levels of detail or different levels of editing, which may be necessary or desirable when entering the segmentation result. Location-specific segmentation tool usage metadata is segmentation tool usage metadata collected for a specific location or neighborhood within the medical image. In different examples, this edit location can take on different sizes. In some examples, the edit location may be a distance of several pixels or several voxels. In other instances, the definition of how large the edit location is can be larger.

[0030] The execution of machine-executable instructions also allows the computational system to repeatedly determine anatomical locations by using edited locations. An anatomical location is a location within a standardized location or an anatomical atlas reference location. In this step, there is a transformation from a physician-marked location within a specific medical image to a reference anatomical location. In different examples, the anatomical location can take different forms. In some instances, the anatomical location can indicate a standard distance from a specific anatomical landmark. In other examples, the anatomical location can be a general area or an area adjacent to or within an anatomical structure. For example, an anatomical location can be defined as a bounding box that includes one or more specific organs. In other examples, the anatomical location can be a large or primary area of ​​an object; for example, an anatomical location can be as large as an area like the chest.

[0031] The execution of machine-executable instructions also causes the computational system to repeatedly retrieve expected segmentation tool usage metadata from the segmentation tool usage database using anatomical locations. The execution of these machine-executable instructions further causes the computational system to repeatedly determine the presence of anomalous segmentation editing data by comparing location-specific segmentation tool usage metadata with expected segmentation tool usage metadata. In these steps, the location-specific segmentation tool usage metadata is compared with the expected segmentation tool usage metadata. It can provide detection of anomalous segmentation editing data if, for example, they change by more than a predetermined amount or if they fall outside the range that can be specified in the expected segmentation tool usage metadata. For example, the user may have used an excessively large or small zoom area when entering segmentation results, and may have moved the segmentation editing tool too quickly or too fast compared to normal operations.

[0032] The execution of machine-executable instructions also enables the computational system to repeatedly provide signals when abnormal segmentation editing data is identified. The signals can have different effects depending on what the signal is. For example, a signal can trigger behavior in the user interface, such as forcing the user to return and further edit the segmentation results in a specific region. This signal could be, for example, a warning signal. In other examples, the signal can provide detailed instructions for the operator of the medical system to follow. The execution of machine-executable instructions also enables the computational system to repeatedly construct the segmentation results of the medical image using the segmentation editing data. Examples can provide higher quality or more consistent segmentation results for medical images compared to not providing signals when abnormal segmentation editing data is identified.

[0033] In clinical practice, segmentation results from medical images are performed regularly. Examples of segmentation results can be used for treatment planning (e.g., tumor segmentation), and the results can provide hundreds to thousands of potential measurements that offer profound insights into a patient's health. These segmentation results are typically performed manually or semi-automatically.

[0034] While automation has been helpful in reducing the amount of time spent performing segmentation, major problems remain to be addressed: learning the segmentation process takes a long time and involves a lot of training, and inter-observer variability is a major problem when clinicians have the option to perform or correct the segmentation results.

[0035] An example could be a system that runs in the background while a user performs segmentation. The tool could track user behavior such as mouse speed, annotation speed, position within the image, gaze, etc. If the tool detects a potential problem or error, the user receives a warning signal, possibly along with an explanation of the error.

[0036] The accuracy of these segmentation results is critical, as incorrect segmentation can have fatal consequences. Incorrect segmentation of tumors or organs at risk can lead to improper planning during radiotherapy and damage to healthy tissue. Incorrect measurements derived from incorrect segmentation can result in incorrect diagnoses. This is also crucial for clinical research; for example, in multiple sclerosis (MS), drug effectiveness is measured by observing the progression of lesions over time. These small lesions require highly accurate segmentation results. Examples can assist operators in performing more accurate segmentation of medical images.

[0037] In various examples, medical images can take different forms. For instance, it can be a two-dimensional slice or image. In other examples, a medical image can be a three-dimensional dataset. In yet another example, a medical image is a four-dimensional dataset. This four-dimensional dataset is a three-dimensional image that changes over time.

[0038] For different types of images, the segmentation results can take different forms. For example, if the medical image is merely a two-dimensional image, the segmentation result could simply be drawn lines. In other examples, when it is two-dimensional, the segmentation result could be splines or other types of shapes that can be stretched and deformed. In yet another example, when the medical image is a three-dimensional dataset, the segmentation result could be a deformable model, such as a three-dimensional manifold that can be twisted or moved. The type of segmentation entry tool can also depend on the exact type of the model or the type of segmentation result. If the segmentation result is merely lines, the segmentation entry tool could be similar to or analogous to a drawing tool used to draw, erase, or manipulate lines. If a three-dimensional model exists, the segmentation entry tool could be, for example, a tool used to stretch or deform a three-dimensional manifold.

[0039] In some examples, the edit location can be a small neighborhood surrounding the segmentation entry tool. In other examples, the edit location can be a larger area encompassing all or part of the medical image.

[0040] In another example, the memory also stores an image classifier. The image classifier is configured to output anatomical locations in response to a received medical image of the object and edit locations within the medical image as input. Execution of machine-executable instructions causes the computational system to receive the anatomical locations in response to inputting the medical image and edit locations into the image classifier. This example can be beneficial because it provides an accurate and automated means of determining anatomical locations using edit locations.

[0041] Image classifiers can be implemented in different ways to detect anatomical regions in an image (e.g., deep learning-based or model-based segmentation or object detection models). For example, in the case of left ventricular segmentation, this could be segmentation of the mitral valve region, myocardium, endocardial boundary, anterior wall, inferior wall, anterolateral wall, inferior wall, basal region of the anterior or inferior septal wall, intermediate region, and apical region. Another example is MS and brain lesions, where an anatomical detector detects the coarse location of MS lesions. The anatomical detector does not need to be very accurate; that is, it does not generate precise segmentations of objects. Coarse anatomical region indicators are sufficient.

[0042] In one example, the image classifier is an image classifier neural network. The medical image and edit locations can be input into the image classifier neural network in different ways. In one example, the medical image is input as a complete medical image, and the edit locations are coordinates within the medical image. In another example, the medical image and edit locations are input into the image classifier neural network by cropping or reducing the medical image to a portion represented only by the edit locations.

[0043] Image classifier neural networks can be implemented in various ways. For example, it can be implemented as a type of convolutional neural network commonly used for image classification. Examples include EfficientNet, VisionTransformer (ViT), ConvNext, DenseNet, Inception, ResNet, ResNeXt, LeNet-5, AlexNet, and VGG16 convolutional neural network architectures. Image classifier neural networks can be trained by creating training data. For example, a medical image with edit locations can be prepared (either as coordinates or by cropping the image) and then labeled using anatomical locations. This can then be used, for example, to train an image classifier neural network using deep learning. If the edit locations are provided as coordinates within the medical image, several additional inputs can be added to the input layer of the neural network. These additional inputs to the input layer can be used to encode the coordinates of the edit locations instead of the input image data. This can be achieved by modifying the input layer. Additional vector-sized inputs can be added to the network.

[0044] Cropping medical images to regions representing the edit locations enables image classifier neural networks to assign categories to regions represented by the cropped images.

[0045] Additional options exist for detecting / classifying anatomical structures. First, object detection networks such as YOLO (with many versions like YOLO V5-YOLO V8), R-CNN, or RetinaNet are used. These networks directly predict bounding boxes plus their categories. The category of the box can be an anatomical region. The network can be trained using pairs of images and manually labeled bounding boxes (which can also be derived, for example, from manually labeled coordinates / landmarks or segmentation masks).

[0046] Another approach is to use networks such as U-Net, nnUnet, TransUnet, or DeepLabV3 for direct segmentation of anatomical structures. This network doesn't need to be highly accurate, as people are only interested in coarse regions rather than precise boundaries. The network is trained using pairs of images and segmentation masks derived from manual annotations.

[0047] In some examples, the image classifier neural network (which may be a deep learning-based anatomical detector) and / or the algorithm used to generate initial segmentation results in a semi-automatic case outputs an uncertainty measure, such as based on softmax probability, Monte Carlo dropout, or other uncertainty techniques.

[0048] In another example, the anatomical location is determined in a different way by editing the location. For example, data can be received manually from the user interface. In yet another example, the anatomical location can be determined by reading metadata from the medical image; for example, data indicating the anatomical location may exist in the DICOM file, or information or initial segmentation results of the medical image may exist, which can provide the anatomical location as a function of its position within the medical image.

[0049] In another example, the execution of the machine-executable instructions also causes the computing system to compute one or more statistical measures based on the location-specific segmentation tool using metadata. The execution of the machine-executable instructions further enables the computing system to detect anomalous segmentation edits by comparing one or more statistical measures obtained by the location-specific segmentation tool using metadata with those obtained by the expected segmentation tool using metadata. This could be an effective means of automatically detecting anomalous segmentation edits.

[0050] The segmentation result can be one or more statistical measures that are specific to a particular editing location. For example, a statistical measure could be the mean, average, maximum, or minimum of various values ​​within the metadata used by the segmentation tool. In another example, the memory also stores a segmentation module configured to output an initial segmentation result in response to receiving a medical image as input. In different examples, the segmentation module can take different forms. For example, there could be an algorithmic segmentation module capable of performing the initial segmentation result using a model fitted to detected anatomical landmarks. In other examples, the segmentation module could be, for example, a neural network, such as U-net, which has been trained to output the initial segmentation result. Therefore, the segmentation module can be any of the various types of automated segmentation modules currently available.

[0051] The execution of machine-executable instructions also causes the computing system to receive initial segmentation results in response to inputting medical images into the segmentation module. Drawing the medical image on the user interface includes displaying the initial segmentation results on the user interface. Constructing the segmentation results of the medical image using segmentation editing data includes editing the initial segmentation results using the segmentation editing data. This example can be beneficial because it provides a good starting point for manually entering segmentation results.

[0052] In another example, the signal includes a display pointer that points from the edit position to the closest original position of the initial segmentation result. In this example, the pointer is an arrow pointing from the current position of the segmentation result to its original position. This can be useful if the segmentation result has been incorrectly edited.

[0053] In another example, the segmentation module is configured to detect and segment abnormal anatomical structures. Abnormal anatomical structures can be, for example, tissues that are not normal anatomical structures, such as scar tissue or tumor tissue. This example can be beneficial because it can provide a means to automatically measure such anatomical structures (e.g., tumor size).

[0054] In another example, the signal includes displaying a pre-defined warning message. This could be, for example, a standard warning instructing the user to be more careful or to redo parts of the segmented result.

[0055] In another example, the memory also stores a segmentation instruction database, which includes segmentation instructions indexed according to anatomical locations. Execution of the machine-executable instructions also causes the computing system to receive segmentation instructions in response to querying the segmentation instruction database using anatomical locations. Providing signals includes displaying the segmentation instructions on a user interface.

[0056] This example can be useful because there may be specific steps an operator should take to correctly segment a particular anatomical location. The segmentation instructions can then be used by the operator to appropriately segment the area that provided the signal. The operator can then properly resegment the area responsible for providing the signal by simply following the instructions.

[0057] Signals include overlays of drawings at the edit location that are configured to modify the segmentation result. For example, if a segment of the segmentation result is detected, triggering or providing a signal, that portion of the segmentation result can be modified. For instance, the thickness or line style can be modified, such as a dashed line or a line style containing hyphens. In other examples, the color of the segmentation result at the edit location can be modified to emphasize to the operator that more care should be taken or that the segmentation result should be redone.

[0058] In another example, the memory also stores an edge detection algorithm configured to output detected edges in response to receiving a medical image as input. The edge detection algorithm could be, for example, the algorithmic edge detection algorithm described above for detecting edges in an image. Various existing algorithms exist that can be used to perform this operation. Execution of the machine-executable instructions also causes the computing system to receive the detected edges obtained in response to receiving the medical image as input. Execution of the machine-executable instructions further causes the computing system to calculate the smoothness of the detected edges at a predetermined distance from the editing position. For example, edges can be detected using standard edge detection filters such as Laplace or Canny.

[0059] The execution of the machine-executable instructions also causes the computing system to calculate the smoothness of the segmentation result of the medical image within a predetermined distance from the editing position. If the difference between the smoothness of the segmentation result and the detected smoothness exceeds a predetermined threshold, the execution of the machine-executable instructions also causes the computing system to trigger a signal.

[0060] In this example, the smoothness of the edges detected in the medical image is compared with the smoothness of the segmentation result. This can be a useful means of automatically detecting whether the segmentation does not represent the actual segmentation of the medical image.

[0061] In another example, the segmentation tool uses metadata including any of the following: maximum segmentation entry tool speed, minimum segmentation entry tool speed, average segmentation entry tool speed, segmentation entry tool position, dwell time of the segmentation entry tool in the editing position, overall annotation speed, mouse speed, visual gaze tracking, visual gaze tracking speed, visual gaze speed, pause duration of segmentation entry tool usage, number of pauses during segmentation entry tool usage, variation in segmentation entry tool speed, segmentation entry tool size, use of the segmentation entry tool's eraser function, and combinations thereof.

[0062] Of course, various things can be implemented by implementing or including systems for tracking the eye position of the operator of the medical system, such as visual gaze tracking, visual gaze tracking speed, visual gaze tracking duration, and pause duration of the input tool.

[0063] In another example, a medical image is any of the following: X-ray, ultrasound image, fluorescence microscopy image, magnetic resonance image, tomographic medical image, computed tomography image, single-photon emission tomography image, or positron emission tomography image.

[0064] In another example, the medical system also includes a medical imaging system configured to acquire medical images. Execution of machine-executable instructions further enables the medical imaging system to acquire medical images.

[0065] In another example, the medical imaging system is any of the following: a computed tomography system, a diagnostic ultrasound system, a digital fluorescence microscope system, an X-ray system, a single-photon emission tomography system, and a positron emission tomography system.

[0066] Figure 1 An example of a medical system 100 according to an example is illustrated. The medical system 100 is shown as including a computer 102. The computer 102 includes a computing system 104. An optional hardware interface 106 is provided that can be used to incorporate additional components into the medical system 100. The hardware interface 106 is shown as communicating with the computing system 104. A user interface 108 and a memory 110 are also provided, which also communicate with the computing system 104. The memory 110 is intended to represent various types of memory that the computing system 104 can access.

[0067] Memory 110 is shown to contain machine-executable instructions 120. Machine-executable instructions 120 enable computing system 104 to perform various numerical, computational, and other tasks, such as controlling other components of medical system 100 (if they are present). Memory 110 is also shown to contain a segmentation tool usage database 122. The segmentation tool usage database 122 stores expected segmentation tool usage metadata 138 as functions for editing location 170. In different examples, this could be implemented as a table, file, or relational database, for example.

[0068] Memory 110 is also shown as containing medical image 124. Memory 110 is also shown as containing segmentation editing data 130. Interface 108 is shown as implementing a graphical user interface 126. The graphical user interface 126 shows the rendering of medical image 124 and the segmentation result 144 that has been edited. A segmentation entry tool 128 is present. There is a neighborhood surrounding segmentation entry tool 128, which is an editing location 170. The segmentation entry tool 128 is controlled, and the segmentation editing data 130 is executed (either inputting or modifying the segmentation result 144). Within editing location 170, location-specific segmentation tools use metadata 134 recorded in memory 110. This is metadata about the use of segmentation entry tool 128 as a function of editing location 170. Editing location 170 can be a small region as indicated in the current image, or it can be the entire image or a different region or anatomical region within medical image 124.

[0069] Location-specific segmentation tool usage metadata 134 is shown as being stored in memory 110. Edit location 170 is then referenced as anatomical location 136. Anatomical location 136 is also shown as being stored in memory 110. The segmentation tool usage database 122 is queried using anatomical location 136, and expected segmentation tool usage metadata 138 is returned. Memory 110 is also shown as storing a comparison between location-specific segmentation tool usage metadata 134 and expected segmentation tool usage metadata 138. If the comparison indicates an anomaly or the comparison is outside a predetermined range, a signal 142 is generated. Signal 142 can trigger various actions of the medical system 100 under different circumstances.

[0070] Memory 110 is shown for storing segmentation results and for drawing the segmentation results on the graphical user interface 124. Memory 110 is shown to contain an optional image classifier 146, which is capable of receiving the medical image 124 and the edit location 170 as input and outputting the anatomical location 136 in response. Memory 110 is also shown to contain an optional edge detection algorithm 148 capable of outputting optional detected edges 150. The image classifier may be an image classifier neural network.

[0071] Optional detected edges 150 are shown as being stored in memory. The memory is also shown as storing optional detected edge smoothness 152 calculated based on the detected edges 150. The memory 110 is also shown as containing optional segmentation result smoothness 154 calculated for segmentation results 144 within the edit position 170. A comparison between the detected edge smoothness 152 and the segmentation result smoothness 154 can also be used to trigger the signal 142.

[0072] Memory 110 is also shown to include an optional segmentation module 156. Optional segmentation module 156 can be used to provide initial segmentation results for medical image 124. Memory 110 is also shown to include an optional segmentation instruction database 158. This can be used to provide optional segmentation instructions 160, which can be provided, for example, in response to a generation signal 142. For example, segmentation instruction database 158 can provide location-specific segmentation instructions 160. Graphical user interface 124 may include instruction boxes that can be used to display segmentation instructions 160.

[0073] Figure 2 The illustrated operation is shown. Figure 1 A flowchart of a method for a medical system 100. In step 200, a medical image 124 is received. Next, in step 202, the medical image 124 is drawn on a user interface 126. The user interface 126 includes a segmentation entry tool 128. In step 204, segmentation editing data 130 is received from the segmentation entry tool 128. The segmentation editing data 130 includes an edit location 170 within the medical image 124. In step 206, location-specific segmentation tool usage metadata 134 is collected for the edit location 170. In step 208, an anatomical location 136 is determined based on the edit location 170. In step 210, expected segmentation tool usage metadata 138 is retrieved from a segmentation tool usage database 122 using the anatomical location 136. In step 212, the presence of anomalous segmentation editing data is determined by comparing the location-specific segmentation tool usage metadata 134 with the expected segmentation tool usage metadata 138. In step 214, a signal 142 is provided if anomalous segmentation editing data is determined to exist or detected. In step 216, the segmentation result 144 is constructed or edited by using the segmentation edit data 130.

[0074] Figure 3 Another example of a medical system 300 is illustrated. Figure 3 The medical system 300 shown in the figure and Figure 1 Similar to medical system 100, but medical system 300 is shown to additionally include magnetic resonance imaging system 302. Taking magnetic resonance imaging system 302 as an example, the magnetic resonance imaging system is an example of a medical imaging system for acquiring medical images 124. The magnetic resonance imaging system can be replaced, for example, by a computed tomography system, diagnostic ultrasound system, digital fluorescence microscope system, X-ray system, single-photon emission computed tomography system, or positron emission tomography system.

[0075] The magnetic resonance imaging system 302 includes a magnet 304. The magnet 304 is a superconducting cylindrical magnet with a through-hole 306. Different types of magnets are also feasible; for example, split cylindrical magnets and so-called open magnets can also be used. A split cylindrical magnet is similar to a standard cylindrical magnet, except that the cryostat has been divided into two sections to allow access to the equiplanar plane of the magnet; such a magnet can, for example, be used in conjunction with charged particle beam therapy. An open magnet has two magnet sections, one above the other, with a sufficiently large space between them to receive the object: the arrangement of the two sections is similar to that of Helmholtz coils. Open magnets are preferred because the object is less restricted. An assembly of superconducting coils is located inside the cryostat of the cylindrical magnet.

[0076] An imaging region 308 exists within the aperture 306 of the cylindrical magnet 304, in which the magnetic field is sufficiently strong and homogeneous to perform magnetic resonance imaging. A field of view 309 is shown within the imaging region 308. The acquired k-space data is typically acquired with respect to the field of view 309. The region of interest may be the same as the field of view 309, or it may be a sub-volume of the field of view 309. An object 318 is shown supported by an object support 320 such that at least a portion of the object 318 is within both the imaging region 308 and the field of view 309.

[0077] An assembly of magnetic field gradient coils 310 is also present within the aperture 306 of the magnet. These magnetic field gradient coils are used to acquire preliminary k-space data for spatial encoding of the magnetic spins within the imaging region 308 of the magnet 304. The magnetic field gradient coils 310 are connected to a magnetic field gradient coil power supply 312. The magnetic field gradient coils 310 are intended to be representative. Typically, the magnetic field gradient coils 310 comprise three separate coil assemblies for spatial encoding in three orthogonal spatial directions. The magnetic field gradient power supply supplies current to the magnetic field gradient coils. The current supplied to the magnetic field gradient coils 310 is controlled as a function of time and can be ramped or pulsed.

[0078] Adjacent to the imaging region 308 is an RF coil 314, which is used to manipulate the orientation of magnetic spins within the imaging region 308 and to receive radio transmissions from the spins also within the imaging region 308. The RF antenna may comprise multiple coil elements. The RF antenna may also be referred to as a channel or antenna. The RF coil 314 is connected to an RF transceiver 316. The RF coil 314 and the RF transceiver 316 may be replaced by separate transmit and receive coils, and separate transmitters and receivers. It should be understood that the RF coil 314 and the RF transceiver 316 are representative. The RF coil 314 is also intended to represent a dedicated transmit antenna and a dedicated receive antenna. Similarly, the transceiver 316 may also represent separate transmitters and receivers. The RF coil 314 may also have multiple receive / transmit elements, and the RF transceiver 316 may have multiple receive / transmit channels.

[0079] Transceiver 316 and gradient controller 312 are shown as hardware interface 106 connected to computer system 102.

[0080] Memory 110 is shown to additionally contain pulse sequence commands 330. Pulse sequence commands 330 are commands or data that can be translated into commands to control the magnetic resonance imaging system 302 to acquire k-space data 332 according to a three-dimensional medical magnetic resonance imaging protocol. Memory 110 is shown to contain k-space data 332 that has been acquired by controlling the magnetic resonance imaging system 302 using pulse sequence commands 330. Memory 110 is also shown to contain magnetic resonance images 334, which are examples of medical images 124. Depending on the magnetic resonance imaging protocol, the magnetic resonance images can be a single two-dimensional image, a stack of two-dimensional images forming a three-dimensional dataset, or a three-dimensional dataset.

[0081] Figure 4 The diagram illustrates the relationship with Figure 1 The depicted medical system 100 is another example of a medical system 400 similar to the one described. In this case, the segmentation module 156 is used to construct the initial segmentation result 402. This is represented by dashed lines on the user interface 124. When a signal is generated, it produces a pointer 404 that points from the segmentation entry tool 128 to the closest position of the initial segmentation result 402. This can be useful guidance, for example, if the segmentation result 144 has been over-edited or is currently incorrect. This can be useful in helping the user or suggesting directions for modifying the segmentation result 144.

[0082] Figure 5An example of an image classifier neural network 146 is illustrated. It receives a medical image 124 and an edit location 170 as input. The edit location 170 may, for example, be a value used to encode the position of the segmentation entry tool 128 within the medical image 124. The output is an anatomical location 136. In this case, the output of the image classifier neural network 146 is multiple outputs for different predetermined anatomical locations 136. The output at each of these points indicates the probability that the edit location 170 is located at a specific anatomical location 136. This is typical for convolutional neural networks used to provide image classification.

[0083] Figure 6 An alternative implementation of the image classifier neural network 146 is illustrated. In this example, there is a single input as a cropped version 124' of the medical image. The cropped version 124' represents the portion of the medical image 124 that is currently considered to be edited at position 170. The output of the neural network 146' is again a plurality of anatomical positions 136. The output at each of these positions represents the probability that edited position 170 is a specific anatomical position 136. This structure is also typical for convolutional neural networks used for image classifiers.

[0084] As described above, the example can include different components. One component is the user interface 108 or graphical user interface (GUI), where the user performs segmentation or manually modifies manually entered or automatically segmented results. In the background, the computing system uses a location-specific segmentation tool to track user behavior using metadata 134, detects problematic behavior, correlates it with anatomical regions via the location-specific segmentation tool using metadata 134, and correlates it with prior knowledge about these regions via the expected segmentation tool using metadata 138, and outputs a warning (signal 142), possibly along with additional user information. The following... Figure 7 The image shows a sample instance of the User Interface 108 GUI.

[0085] Figure 7Two views 700 and 702 of the user interface 124 are shown. The complete segmentation result 144 is shown in both views. In image 700, a first message box 704 indicates the area of ​​the segmentation result 144, which may contain errors detected by the medical system 100. If the operator of the medical system wants more information, they can click on the information indicator 705, and the first message box 704 transforms into a second message box 706, as illustrated in image 702. The second message box 706 provides the operator with more detailed information. Message box 706 is a pop-up window shown to the user due to a detected problem and is an example of signal 142. By clicking on the information indicator 705, the user receives more information: 1) the user is very fast in this area of ​​the system, detected by mouse speed; 2) wall visibility appears to be difficult, which is derived by the system based on the uncertainty of the DL-based anatomy detector; 3) additional prior information is available, pulled from the database.

[0086] Figure 8 The illustration depicts the detection of abnormal segmentation edits. Image 800 shows the segmentation result 144, and the tracking measurement or annotation speed was recorded. In region 806 shown in image 800, excessively fast tracking speed has been detected. Image 802 indicates how to determine the edit location 170 for the segmentation result 144. The box labeled 170 is the edit location 170 for the problem region 806, which is marked in image 800. By using this edit location, the region with problem 806 is matched to anatomical location 136 using a neural network. This then causes the segmentation tool to be queried using database 122 and provides message 808. This provides instructions on how the segmentation result in region 806 can be improved.

[0087] In more detail, Figure 8 The steps illustrated include: 1. When the image to be segmented is opened, the anatomy detector (image classifier 146) can be triggered in the background. The user can continue segmentation / correction while the anatomy detector is running.

[0088] 2. During segmentation / correction, user behavior is tracked by recording location-specific segmentation tool usage data 134, and user behavior is analyzed in the background by comparing location-specific segmentation tool usage metadata 134 with expected segmentation tool usage metadata 138.

[0089] 3. If an outlier / problem 806 is detected, the pixel location of the problem can be associated with the anatomical information. Message 808 is then prepared for the user based on the anatomical information.

[0090] 4. A pop-up message 808 is displayed in the user GUI, which has buttons for viewing full information about the problem, as shown in image 804. For example, the user can be provided with information about the error and potential help. The message to the user can be constructed from predefined text blocks, or the message can be generated by a language model that receives information fragments as words and outputs the complete text.

[0091] It should be understood that one or more examples of the foregoing examples or embodiments of the present invention can be combined as long as the combined embodiments are not mutually exclusive.

[0092] As those skilled in the art will understand, aspects of the present invention can be embodied as apparatus, method, or computer program product. Accordingly, aspects of the present invention can take the form of a completely hardware embodiment, a completely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, all of which are generally referred to herein as “circuit,” “module,” or “system.” Furthermore, aspects of the present invention can take the form of a computer program product embodied thereon on one or more computer-readable media having computer-executable code thereon.

[0093] Any combination of one or more computer-readable media can be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. As used herein, "computer-readable storage medium" includes any tangible storage medium capable of storing instructions executable by a processor or computing system of a computing device. A computer-readable storage medium can be referred to as a computer-readable non-transitory storage medium. A computer-readable storage medium can also be referred to as a tangible computer-readable medium. In some embodiments, a computer-readable storage medium may also store data accessible by a computing system of a computing device. Examples of computer-readable storage media include, but are not limited to: floppy disks, magnetic hard disk drives, solid-state drives, flash memory, USB thumb drives, random access memory (RAM), read-only memory (ROM), optical discs, magneto-optical discs, and register files of computing systems. Examples of optical discs include optical discs (CDs) and digital versatile discs (DVDs), such as CD-ROMs, CD-RWs, CD-Rs, DVD-ROMs, DVD-RWs, or DVD-R discs. The term computer-readable storage medium also refers to various types of recording media accessible by a computer device via a network or communication link. For example, data can be retrieved via a modem over the Internet or via a local area network. Computer-executable code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, fiber optic cable, RF, or any suitable combination thereof.

[0094] Computer-readable signal media may include propagated data signals embodying computer-executable code therein, for example, in baseband or as a carrier wave. Such propagated signals may take any of a variety of forms, including but not limited to electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium that is not a computer-readable storage medium and is capable of communicating, propagating, or transporting programs used by or in conjunction with an instruction execution system, apparatus, or device.

[0095] "Computer memory" or "memory" is an example of a computer-readable storage medium. Computer memory is any memory that can be directly accessed by a computing system. "Computer storage device" or "storage device" is another example of a computer-readable storage medium. A computer storage device is any non-volatile computer-readable storage medium. In some embodiments, a computer storage device may also be computer memory, and vice versa.

[0096] As used herein, "computing system" encompasses electronic components capable of executing programs or machine-executable instructions or computer-executable code. References to computing systems, including examples of "computing systems," should be interpreted as potentially encompassing more than one computing system or processing core. A computing system can, for example, be a multi-core processor. A computing system can also refer to a collection of computing systems within a single computer system or distributed across multiple computer systems. The term "computing system" should also be interpreted as potentially referring to a collection or network of computing devices, each including a processor or computing system. Machine-executable code or instructions can be executed by multiple computing systems or processors, and this execution can be within the same computing device or even distributed across multiple computing devices.

[0097] Machine-executable instructions or computer-executable code may include instructions or programs that cause a processor or other computing system to perform aspects of the invention. Computer-executable code for performing the operations of the aspects of the invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Java, Smalltalk, C++, etc.) and conventional procedural programming languages ​​(such as the "C" programming language or similar programming languages), and compiled into machine-executable instructions. In some instances, the computer-executable code may be in the form of a high-level language or in a pre-compiled form and may be used in conjunction with an interpreter that generates machine-executable instructions at runtime. In other instances, the machine-executable instructions or computer-executable code may be in the form of a programmed array of programmable logic gates.

[0098] Computer executable code can: execute entirely on the user's computer, execute partially on the user's computer, execute as a standalone software package, execute partially on the user's computer and partially on a remote computer, or execute entirely on a remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet through an Internet service provider).

[0099] Aspects of the invention are described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block or portion of a block in a flowchart, illustration, and / or block diagram may, where applicable, be implemented by computer program instructions in the form of computer-executable code. It should also be understood that combinations of blocks in different flowcharts, illustrations, and / or block diagrams may be combined when not mutually exclusive. These computer program instructions may be provided to a computing system of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, executable via the computing system of the computer or other programmable data processing apparatus, create units for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0100] These machine-executable instructions or computer program instructions may also be stored in a computer-readable medium that can instruct a computer, other programmable data processing apparatus or other device to operate in a particular manner, such that the instructions stored in the computer-readable medium produce an article of writing which includes instructions that implement the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0101] Machine-executable instructions or computer program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer-implemented process, such that the instructions that execute on the computer or other programmable apparatus provide for implementing the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0102] The term "user interface" as used in this document refers to an interface that allows a user or operator to interact with a computer or computer system. A "user interface" can also be called a "human-machine interface device." A user interface can provide information or data to and / or receive information or data from an operator. A user interface enables the computer to receive input from the operator and to provide output from the computer to the user. In other words, a user interface allows an operator to control or manipulate a computer, and the interface allows the computer to indicate the effects of the operator's control or manipulation. Displaying data or information on a monitor or graphical user interface is an example of providing information to an operator. Receiving data via a keyboard, mouse, trackball, touchpad, joystick, graphics tablet, joystick, game controller, webcam, headset, pedal, wired gloves, remote control, and accelerometer are all examples of user interface components capable of receiving information or data from an operator.

[0103] As used herein, "hardware interface" includes interfaces that enable a computer system to interact with and / or control external computing devices and / or devices. A hardware interface can allow a computing system to send control signals or instructions to external computing devices and / or devices. A hardware interface can also enable a computing system to exchange data with external computing devices and / or devices. Examples of hardware interfaces include, but are not limited to: Universal Serial Bus (USB), IEEE 1394 port, parallel port, IEEE 1284 port, serial port, RS-232 port, IEEE-488 port, Bluetooth connectivity, wireless LAN connectivity, TCP / IP connectivity, Ethernet connectivity, control voltage interface, MIDI interface, analog input interface, and digital input interface.

[0104] As used herein, "display" or "display device" includes output devices or user interfaces suitable for displaying images or data. Displays can output visual, audio, and / or tactile data. Examples of displays include, but are not limited to: computer monitors, television screens, touchscreens, tactile electronic displays, Braille screens, cathode ray tubes (CRTs), memory tubes, bistable displays, electronic paper, vector displays, flat panel displays, vacuum fluorescent displays (VFs), light-emitting diode (LED) displays, electroluminescent displays (ELDs), plasma display panels (PDPs), liquid crystal displays (LCDs), organic light-emitting diode (OLED) displays, projectors, and head-mounted displays.

[0105] Although the invention has been illustrated and described in detail in the accompanying drawings and the foregoing description, such illustrations and descriptions are to be considered illustrative or exemplary rather than limiting; the invention is not limited to the disclosed embodiments.

[0106] By studying the accompanying drawings, the disclosure, and the appended claims, those skilled in the art will be able to understand and implement other variations of the disclosed embodiments in practicing the claimed invention. In the claims, the word "comprising" does not exclude other elements or steps, and the quantifiers "a" or "an" do not exclude a plurality. A single processor or other unit may implement the functions of several items recited in the claims. The mere fact that certain measures are recited in dissimilar dependent claims does not indicate that combinations of these measures cannot be advantageously used. Computer programs may be stored / distributed on suitable media, such as optical storage media or solid-state media supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems. No reference numerals in the claims should be construed as limiting the scope.

Claims

1. A medical system (100, 300, 400), comprising: A memory (110) stores machine-executable instructions (120) and a segmentation tool usage database (122), wherein the segmentation tool usage database includes expected segmentation tool usage metadata (138) indexed according to anatomical locations (136), wherein the anatomical location is a region adjacent to or within an anatomical atlas reference location; A computing system wherein the execution of the machine-executable instructions causes the computing system to perform the following operations: Receive (200) medical images (124); The medical image is drawn (202) on a user interface (126), wherein the user interface includes a segmentation input tool; The segmentation input tool is a tool used to input, manipulate, or edit the segmentation results of the medical image; The execution of the machine-executable instructions also causes the computing system to repeatedly perform the following operations: Receive (204) segmentation editing data (130) from the segmentation input tool, wherein the segmentation editing data includes editing locations (170) within the medical image, wherein the editing locations are neighborhoods surrounding the segmentation input tool; Collect (206) location-specific segmentation tool usage metadata (134), which describes the use of the segmentation entry tool during the entry of the segmentation editing data within the editing location, wherein the segmentation tool usage metadata includes any one of the following: maximum segmentation entry tool speed, minimum segmentation entry tool speed, average segmentation entry tool speed, segmentation entry tool position, dwell time of the segmentation entry tool in the editing location, overall annotation speed, mouse speed, visual gaze tracking, visual gaze tracking speed, visual gaze speed, pause duration of the use of the segmentation entry tool, number of pauses during the use of the segmentation entry tool, variation of the segmentation entry tool speed, segmentation entry tool size, use of the eraser function of the segmentation entry tool, and combinations thereof; The anatomical location (208) is determined by using the edited location; The expected segmentation tool uses metadata retrieved from the database using the anatomical location (210); (212) The presence of anomalous segmentation edit data is determined (806) by comparing the metadata used by the location-specific segmentation tool with the metadata used by the expected segmentation tool. Provide (214) signals (142, 404, 808) if the presence of the abnormal segmented edited data is confirmed; and The segmentation result (144) of the medical image is constructed (216) by using the segmentation editing data.

2. The medical system according to claim 1, wherein, The memory also stores an image classifier (146, 146'), wherein the image classifier is configured to output the anatomical location in response to the medical image of the receiving object and the edit position within the medical image as input, wherein execution of the machine-executable instructions causes the computing system to receive the anatomical location in response to inputting the medical image and the edit position into the image classifier.

3. The medical system according to claim 1 or 2, wherein, The execution of the machine-executable instructions also causes the computing system to perform the following operations: The location-specific segmentation tool uses metadata to calculate one or more statistical metrics. as well as The anomalous segmentation edit data is detected by comparing one or more statistical measures of the metadata used by the location-specific segmentation tool with the metadata used by the expected segmentation tool.

4. The medical system according to any one of the preceding claims, wherein, The memory also stores a segmentation module (156) configured to output an initial segmentation result in response to receiving the medical image as input, wherein execution of the machine-executable instructions further causes the computing system to receive the initial segmentation result in response to inputting the medical image into the segmentation module, wherein drawing the medical image on the user interface includes displaying the initial segmentation result on the user interface, and wherein constructing the segmentation result of the medical image using the segmentation editing data includes editing the initial segmentation result using the segmentation editing data.

5. The medical system according to claim 4, wherein, The signal includes a display pointer (404) located at the edit position, which points to the closest original position of the initial segmentation result.

6. The medical system according to claim 4 or 5, wherein, The segmentation module is configured to detect abnormal anatomical structures and segment them.

7. The medical system according to any one of the preceding claims, wherein, The signals include displaying predetermined warning messages (172, 706).

8. The medical system according to any one of the preceding claims, wherein, The memory also stores a segmentation instruction database (158), the segmentation instruction database including segmentation instructions indexed according to the anatomical location, wherein execution of the machine-executable instructions further causes the computing system to receive segmentation instructions in response to querying the segmentation instruction database using the anatomical location; and wherein providing the signal includes displaying the segmentation instructions on the user interface.

9. The medical system according to any one of the preceding claims, wherein, The signal includes an overlay configured to modify the drawing at the edit position of the segmentation result.

10. The medical system according to any one of the preceding claims, wherein, The memory also stores an edge detection algorithm (148) configured to output detected edges in response to receiving the medical image as input, wherein execution of the machine-executable instructions further causes the computing system to perform the following operations: Receive the detected edges obtained in response to receiving the medical image as input; Calculate the smoothness of the detected edges within a predetermined distance from the edit position; Calculate the smoothness of the segmentation result of the medical image within the predetermined distance from the editing position; If the smoothness of the segmentation result differs from the smoothness of the detection by more than a predetermined threshold, the signal is triggered.

11. A medical imaging method, in, The method includes: Receive (200) medical images (124); The medical image is drawn (202) on a user interface (126), wherein the user interface includes a segmentation input tool, wherein the segmentation input tool is a tool for inputting, manipulating or editing the segmentation results of the medical image; The method further includes repeatedly performing the following operations: Receive (204) segmentation editing data (130) from the segmentation input tool, wherein the segmentation editing data includes editing locations (170) within the medical image, wherein the editing locations are neighborhoods surrounding the segmentation input tool; Collect (206) location-specific segmentation tool usage metadata (134), which describes the use of the segmentation entry tool during the entry of the segmentation editing data within the editing location, wherein the segmentation tool usage metadata includes any one of the following: maximum segmentation entry tool speed, minimum segmentation entry tool speed, average segmentation entry tool speed, segmentation entry tool position, dwell time of the segmentation entry tool in the editing location, overall annotation speed, mouse speed, visual gaze tracking, visual gaze tracking speed, visual gaze speed, pause duration of the use of the segmentation entry tool, number of pauses during the use of the segmentation entry tool, variation of the segmentation entry tool speed, segmentation entry tool size, use of the eraser function of the segmentation entry tool, and combinations thereof; The anatomical location (136) is determined by using the edited location (208), wherein the anatomical location is a region adjacent to or within an anatomical atlas reference location; (210) Expected segmentation tool usage metadata is retrieved from the segmentation tool usage database (122) by using the anatomical location, wherein the segmentation tool usage database includes expected segmentation tool usage metadata indexed according to the anatomical location; (212) The presence of anomalous segmentation edit data is determined (806) by comparing the metadata used by the location-specific segmentation tool with the metadata used by the expected segmentation tool. Provide (214) signals (142, 404, 808) if the presence of the abnormal segmented edited data is confirmed; and The segmentation result (144) of the medical image is constructed (216) by using the segmentation editing data.

12. A computer program comprising machine-executable instructions (120) and a segmentation tool using a database (122), wherein, The segmentation tool uses a database including expected segmentation tool usage metadata (138) indexed according to anatomical locations (136), wherein the anatomical location is a region adjacent to or within an anatomical atlas reference location, and wherein the execution of the machine-executable instructions causes the computing system (104) to perform the following operations: Receive (200) medical images (124); The medical image is drawn (202) on a user interface (126), wherein the user interface includes a segmentation input tool (128). The segmentation input tool is a tool used to input, manipulate, or edit the segmentation results of the medical image; The execution of the machine-executable instructions also causes the computing system to repeatedly perform the following operations: Receive (204) segmentation editing data (130) from the segmentation input tool, wherein the segmentation editing data includes editing locations (170) within the medical image, wherein the editing locations are neighborhoods surrounding the segmentation input tool; Collect (206) location-specific segmentation tool usage metadata, which describes the use of the segmentation entry tool during the entry of the segmentation editing data within the editing location, wherein the segmentation tool usage metadata includes any one of the following: maximum segmentation entry tool speed, minimum segmentation entry tool speed, average segmentation entry tool speed, segmentation entry tool position, dwell time of the segmentation entry tool in the editing location, overall annotation speed, mouse speed, visual gaze tracking, visual gaze tracking speed, visual gaze speed, pause duration of the segmentation entry tool usage, number of pauses during the use of the segmentation entry tool, variation of segmentation entry tool speed, segmentation entry tool size, use of the eraser function of the segmentation entry tool, and combinations thereof; The anatomical location (208) is determined by using the edited location; The expected segmentation tool uses metadata retrieved from the database using the anatomical location (210); (212) The presence of anomalous segmentation edit data is determined (806) by comparing the metadata used by the location-specific segmentation tool with the metadata used by the expected segmentation tool. Provide (214) signals (142, 404, 808) if the presence of the abnormal segmented edited data is confirmed; and The segmentation result (144) of the medical image is constructed (216) by using the segmentation editing data.

13. The computer program according to claim 12, wherein, The computer program further includes an image classifier (146, 146'), wherein the image classifier is configured to output the anatomical location in response to the medical image of the receiving object and the edit position within the medical image as input, wherein execution of the machine-executable instructions causes the computing system to receive the anatomical location in response to inputting the medical image and the edit position into the image classifier.

14. The computer program according to claim 12 or 13, wherein, The execution of the machine-executable instructions also causes the computing system to perform the following operations: The location-specific segmentation tool uses metadata to calculate one or more statistical metrics. as well as The anomalous segmentation edit data is detected by comparing one or more statistical measures of the metadata used by the location-specific segmentation tool with the metadata used by the expected segmentation tool.

Images