Medical image annotation method, system and device

Abstract

Embodiments of the present specification provide a medical image annotation method and system, the method being executed by at least one processor. The method includes: obtaining a target medical image to be annotated, where the target medical image is a three-dimensional image; obtaining a first annotation path for a first slice of the target medical image by a user; and determining, based on the first annotation path, a corresponding second annotation path in a second slice of the target medical image, where the first slice and the second slice correspond to different perspectives.

Description

Technical Field

[0001] This specification relates to the field of image processing, and in particular to a medical image annotation method, system, and device. Background Art

[0002] Medical image annotation is a highly specialized task, typically completed manually by users. However, users sometimes need to centrally annotate patient images, which can be labor-intensive and subject to inconsistent annotator standards. Furthermore, traditional stand-alone annotation software requires users to upload the data to be annotated one by one, annotate each page, and then save the completed data.

[0003] Therefore, it is necessary to provide a medical image annotation method and system that can reduce the annotation workload and improve the annotation quality. Summary of the Invention

[0004] One embodiment of the present specification provides a medical image annotation method, executed by at least one processor, the method comprising: obtaining a target medical image to be annotated, the target medical image being a three-dimensional image; obtaining a first annotation path for a first slice of the target medical image by a user; and determining a corresponding second annotation path in a second slice of the target medical image based on the first annotation path, the first slice and the second slice corresponding to different perspectives.

[0005] One embodiment of the present specification provides a medical image annotation system, the system comprising: an acquisition module for acquiring a target medical image to be annotated and acquiring a first annotation path of a user for a first slice of the target medical image, wherein the target medical image is a three-dimensional image; and a determination module for determining a corresponding second annotation path in a second slice of the target medical image based on the first annotation path, wherein the first slice and the second slice correspond to different viewing angles.

[0006] One embodiment of this specification provides a medical image annotation device, which includes a processor and a memory; the memory is used to store instructions, and when the instructions are executed by the processor, the device causes the device to implement the aforementioned medical image annotation method.

[0007] One of the embodiments of the present specification provides a medical image annotation system, which includes: a remote database, which is configured to store at least one medical image and its annotation information; a user terminal, which is configured to receive annotation instructions input by a user for a target medical image among the at least one medical image; and a server, which is configured to update the annotation information of the target medical image in the remote database in response to the annotation instructions. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] This specification will be further described in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, like numbers represent like structures, wherein:

[0009] Figure 1 is a schematic diagram of an application scenario of a medical image annotation system according to some embodiments of this specification;

[0010] Figure 2 is a module diagram of a medical image annotation system according to some embodiments of this specification;

[0011] Figure 3 is an exemplary flow chart of a medical image annotation method according to some embodiments of this specification;

[0012] Figure 4 is an exemplary flow chart of generating a second annotation path according to some embodiments of this specification;

[0013] Figure 5 is an exemplary flowchart of determining an index value of a point on a first annotation path according to some embodiments of this specification;

[0014] Figure 6 is a schematic diagram of a annotation model selection interface of a one-stop annotation platform according to some embodiments of this specification;

[0015] Figure 7 This is a schematic diagram of the result of automatically outlining a lung model according to the automatic annotation mode based on the one-stop annotation platform as shown in some embodiments of this specification;

[0016] Figure 8 This is a schematic diagram of a multi-dimensional annotation and display function interface of a one-stop annotation platform according to some embodiments of this specification;

[0017] Figure 9 This is a schematic diagram of an interface for modifying annotation results based on a one-stop annotation platform according to some embodiments of this specification;

[0018] Figure 10 This is a schematic diagram of a review result display interface on a one-stop annotation platform according to some embodiments of this specification. DETAILED DESCRIPTION

[0019] To more clearly illustrate the technical solutions of the embodiments of this specification, the following briefly describes the drawings required for describing the embodiments. Obviously, the drawings described below are merely examples or embodiments of this specification. Those skilled in the art can apply this specification to other similar scenarios based on these drawings without inventive effort. Unless otherwise apparent from the context or otherwise noted, the same reference numerals in the figures represent the same structure or operation.

[0020] It should be understood that the terms "system," "device," "unit," and / or "module" used herein are a method for distinguishing different components, elements, parts, portions, or assemblies at different levels. However, if other terms can achieve the same purpose, the terms may be replaced by other expressions.

[0021] As used in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not refer to the singular but also include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0022] Flowcharts are used throughout this specification to illustrate the operations performed by systems according to embodiments of this specification. It should be understood that preceding or following operations do not necessarily need to be performed in exact order. Instead, the steps may be processed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0023] Figure 1 is a schematic diagram of an application scenario of the medical image annotation system 100 according to some embodiments of this specification.

[0024] Medical image processing often requires annotating medical images to facilitate disease assessment, diagnosis, and treatment based on the annotated images. The medical image annotation system 100 can be used to annotate medical images. In some embodiments, the medical image annotation system 100 can include features such as multi-dimensional annotation, flexible annotation mode selection, and annotation result review.

[0025] like Figure 1 As shown, in some embodiments, the medical image annotation system 100 may include a server 110 , a network 120 , a scanning device 130 , a user terminal 140 , and a remote database 150 .

[0026] The server 110 can process data and / or information obtained from the scanning device 130, the user terminal 140, and the remote database 150. For example, the server 110 can obtain computed tomography (CT) scan images, PET (Positron Emission Computed Tomography) scan images, etc. from the scanning device 130 and analyze and process them. For another example, the server 110 can obtain annotated medical images and their annotation information from the remote database 150. For another example, the server 110 can obtain a user-entered annotation instruction for a target medical image in at least one medical image from the user terminal 140.

[0027] In some embodiments, the server 110 is configured to update the annotation information of the target medical image in the remote database 150 in response to the acquired annotation instruction. In some embodiments, the target medical image may be a three-dimensional image constructed based on sampling of a scanning device. The annotation instruction includes a first annotation path for a first slice of the target medical image. In response to the annotation instruction, the server 110 may determine a corresponding second annotation path in a second slice of the target medical image based on the first annotation path. The server 110 may further instruct the user terminal 140 to display the second slice and the second annotation path. The first slice and the second slice correspond to different viewing angles. For more information on the operations related to obtaining annotation instructions and updating the annotation information of the target medical image, see Figure 3 The corresponding content.

[0028] In some embodiments, server 110 may be local or remote. For example, server 110 may access information and / or data from scanning device 130, user terminal 140, and remote database 150 via network 120. In some embodiments, server 110 may be part of user terminal 140 or scanning device 130. For example, server 110 may be integrated into scanning device 130 to analyze and process acquired medical images.

[0029] In some embodiments, the server 110 may include a processor that can execute program instructions. The processor may include various common general-purpose central processing units (CPUs), graphics processing units (GPUs), microprocessor units (MPUs), application-specific integrated circuits (ASICs), or other types of integrated circuits.

[0030] The network 120 may include any suitable network capable of facilitating information and / or data exchange within the system. In some embodiments, at least one component of the medical image annotation system 100 (e.g., the server 110, the scanning device 130, the user terminal 140, and the remote database 150) may exchange information and / or data with at least one other component of the medical image annotation system 100 via the network 120. For example, the server 110 may obtain scan data or a medical image of a scanned object from the scanning device 130 via the network 120. In some embodiments, the network 120 may include at least one network access point. For example, the network 120 may include a wired and / or wireless network access point, such as a base station and / or an Internet exchange point, and at least one component of the medical image annotation system 100 may connect to the network 120 via the access point to exchange data and / or information.

[0031] The scanning device 130 may be an imaging device for disease diagnosis or research purposes. The scanning device 130 scans a scan object within a detection area or a scan area to obtain scan data and a scan image of the scan object.

[0032] In some embodiments, the scanning device 130 may include a single-modality scanner 130-1 and / or a multi-modality scanner 130-2. The single-modality scanner 130-1 may include, for example, an ultrasound (US) scanner, an X-ray scanner, a CT scanner, an MRI scanner, a PET scanner, or any combination thereof.

[0033] The multimodal scanner 130-2 may include, for example, an X-ray imaging-magnetic resonance imaging (X-ray-MRI) scanner, a positron emission tomography-X-ray imaging (Positron Emission Computed Tomography-X-ray, PET-X-ray) scanner, a single photon emission computed tomography-magnetic resonance imaging (SPECT-MRI) scanner, a positron emission tomography-computed tomography (Positron Emission Computed Tomography-Computed Tomography, PET-CT) scanner, a digital subtraction angiography-magnetic resonance imaging (DSA-MRI) scanner, etc.

[0034] In some embodiments, the scanned image acquired by the scanning device 130 may be a three-dimensional medical image. In some embodiments, the scanning device 130 may upload the acquired scanned image to a remote database 150 for storage or send it to the server 110 for processing.

[0035] User terminal 140 may be an electronic device used by a user, including various mobile devices, smart devices, wearable devices, etc. Examples include, but are not limited to, mobile phone 140-1, tablet computer 140-2, and notebook computer 140-3. In some embodiments, a user may send a labeling instruction to server 110 via user terminal 140 to label a target medical image in at least one medical image. In other embodiments, the user may receive the labeling results from server 110 via user terminal 140.

[0036] Remote database 150 may store data, instructions, and / or any other information. In some embodiments, remote database 150 may store data and / or instructions that server 110 may use to execute or perform the exemplary methods described herein.

[0037] In some embodiments, the remote database 150 can be connected to the network 120 to communicate with at least one other component in the medical image annotation system 100 (e.g., the server 110, the scanning device 130, or the user terminal 140). At least one component in the medical image annotation system 100 can access data stored in the remote database 150 via the network 120. In some embodiments, the data stored in the remote database 150 includes annotated medical images and their annotation information, medical images to be annotated, etc. In some embodiments, the remote database 150 can be part of the server 110.

[0038] In some embodiments, a one-stop medical image annotation platform can be constructed based on the above-mentioned remote database 150, user terminal 140, server 110, and network 120. Annotation refers to the process of outlining the outline of the ROI (Region of Interest) in the medical image. The ROI may include specific organs, tissues, and / or areas with specific characteristics (for example, artifact areas). By annotating the ROI in the medical image, the ROI can be analyzed and processed more conveniently and in a targeted manner. In some embodiments, the one-stop annotation platform can realize functions such as multi-dimensional annotation, free selection of annotation mode, adjustment of annotation results, and review of annotation results.

[0039] Multi-dimensional annotation means that when the user completes the annotation of a certain perspective, the one-stop annotation platform will display the corresponding annotation position in other perspectives at the same time. For example, after the user marks the ROI in the coronal plane of the medical image, the one-stop annotation platform can simultaneously mark and display the corresponding ROI position in the transverse and / or sagittal plane of the medical image based on the processing of the acquired annotation data. Figure 8 As shown, the user can add a lung annotation result 810 to the cross-sectional image displayed on the right side of the terminal interface. Based on the user's annotation, the one-stop annotation platform can simultaneously annotate the lungs in the sagittal and coronal planes, and display the coronal annotation 820 and sagittal annotation 830 on the left side of the user terminal interface.

[0040] In some embodiments, the one-stop annotation platform can provide users with a variety of annotation modes, such as free annotation, semi-automatic annotation, and automatic annotation. Free annotation refers to manual annotation. For example, a user can manually annotate the ROI by controlling the free pen displayed on the interface of the user terminal 140. Semi-automatic annotation refers to manual annotation using the magnetic lasso tool. Unlike free annotation, the magnetic lasso tool used in semi-automatic annotation can automatically fit the outline of the ROI to achieve a quick outlining effect. In some embodiments, the user can set the step size of the magnetic lasso (that is, the distance between two adjacent control points of the magnetic lasso). In some embodiments, the user can adjust the annotation results of the magnetic lasso by adjusting the control points.

[0041] Automatic labeling refers to the use of automatic labeling algorithms or models to automatically label the outline of the ROI in the medical image. In some embodiments, the automatic labeling model may include Convolutional Neural Networks (CNN), Deep Neural Networks (DNN), etc. or any combination thereof. The automatic labeling algorithm may be an algorithm for a certain part or organ of the scanned target object, such as the automatic labeling algorithm may include the Sobel algorithm, Roberts algorithm, Prewitt algorithm, Canny algorithm or Laplacian algorithm, etc. or any combination thereof. The scanned target object may be a human body or an animal, etc., the scanned part may be the chest, abdomen, head, etc., and the scanned organ may be the breast, esophagus, heart, etc. located in the chest, or the kidney, liver, pancreas, gallbladder, etc. located in the abdomen. Figure 6 The following is a schematic diagram of the interface for outlining the lung model using the automatic annotation mode on the one-stop annotation platform. Figure 7 As shown, it includes a delineated area 710 and a delineated area 720 .

[0042] In some embodiments, when image processing personnel, such as researchers, use models for automatic annotation, they no longer need to perform layer-by-layer annotation. For example, they can automatically identify and annotate the contours of ROIs in different slices of a 3D medical image based on a magnetic lasso or algorithmic model. This can effectively improve annotation efficiency and reduce user effort.

[0043] In some embodiments, the one-stop annotation platform can implement the function of adjusting the annotation results. For example, the user can adjust and / or erase the outline of the annotation. For another example, after the user uses the lasso tool to perform semi-automatic annotation, the user can adjust the step size of the control point, add, delete, or roll back the control point, etc. Figure 9The figure shows a schematic diagram of further erasing and annotating the annotation results of the lung in the transverse section based on the one-stop annotation platform. Figure 9 As shown, the dotted area corresponding to area 910 is Figure 7 The area that is erased on the annotation result is the area corresponding to area 920. Figure 7 A new annotation area is added to the annotation results.

[0044] In some embodiments, the one-stop annotation platform can implement an annotation result review function. The review results may include review passed (i.e., the annotation quality is qualified), review failed (i.e., the annotation quality is unqualified), review cancelled, etc. Figure 10 Shown is a schematic diagram of the review result display interface on the one-stop annotation platform.

[0045] In some embodiments, a review function can be used to view and manage the annotation and review status of medical images. For example, it can be used to check whether a medical image has been annotated, whether the annotation results have been reviewed, and whether the annotation results have passed review. For example, a status tag can be assigned to each medical image, with the status tag indicating the annotation status and review status of each medical image. When the annotation or review status of a medical image changes, the server simply updates the tag value of the medical image accordingly. For example, a tag value of 0 can be pre-set to indicate the review status, such as a tag value of 0 indicating unreviewed, a tag value of 1 indicating under review, and a tag value of 2 indicating completed review. The review status can be determined by reading the tag value, and the update can be achieved by updating the tag value. When the amount of medical image data reaches a certain level and / or there are many annotators, data confusion or duplicate annotation and review can easily occur. The annotation result review function provided in this embodiment can improve the data management efficiency of the annotation platform and avoid data confusion and duplicate annotation.

[0046] In some embodiments, the user can access the one-stop annotation platform through the annotation software or browser installed on the user terminal 140. For example, the one-stop annotation platform can display the corresponding functional modules on the annotation software or browser of the user terminal 140. As an example only, the functional modules may include one or more of an annotation method selection module, an annotation image type selection module, an annotation result adjustment module, and a review module. Based on the annotation method selection module, the user can select the annotation method. Based on the annotation image type selection module, the user can select the type of image to be annotated. Based on the annotation result adjustment module, the user can adjust and / or erase the annotation results. Based on the review module, the user can review the annotation results.

[0047] In some embodiments, when using a one-stop annotation platform for annotation, users do not need to upload, download, transfer, etc. data. They can directly access and modify the data in the remote database through the browser or client of the user terminal 140, thereby greatly improving the efficiency of data processing.

[0048] In some embodiments, to enhance the data security of the entire one-stop annotation platform, a login password and corresponding encrypted token can be set for each registered user. When a user logs into the platform, their identity can be verified. For example, only after the user enters the correct password and passes the token verification as required can they be considered authenticated. Only after passing the authentication can they successfully log in and access the corresponding data.

[0049] In some embodiments, the data in the remote database 150 is stored in an encrypted manner. When the user passes the identity authentication, the encrypted data is sent to the server 110 for decryption, and the decrypted data is then presented to the user via the user terminal 140.

[0050] It should be noted that the medical image annotation system 100 described herein is provided for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art will appreciate that various modifications and variations can be made based on the description herein. For example, the scanning device 130 and the user terminal 140 may each have their own independent storage module. However, such modifications and variations would not deviate from the scope of this application.

[0051] Figure 2 is a schematic diagram of a medical image annotation system 200 according to some embodiments of this specification.

[0052] In some embodiments, the medical image annotation system 200 may include an acquisition module 210 and a determination module 220 .

[0053] The acquisition module 210 can be used to acquire a target medical image to be annotated and to acquire a first annotation path of a user for a first slice of the target medical image, wherein the target medical image is a three-dimensional image. The three-dimensional image described in this embodiment can be a three-dimensional image directly acquired based on a three-dimensional image acquisition device, or a three-dimensional image constructed based on a plurality of acquired two-dimensional images, or a three-dimensional image acquired based on other feasible methods, which is not limited in this embodiment. For a detailed description of the function of the acquisition module 210, see Figure 3 Step 310 and step 320 are not described again here.

[0054] In some embodiments, the acquisition module 210 may also be used to acquire the review result of the second annotation path in the second slice input by the user via the user terminal. For instructions on acquiring the review result of the second annotation path, see Figure 3 Relevant content of step 350.

[0055] The determination module is configured to determine a second annotation path corresponding to a second slice of the target medical image based on the first annotation path, wherein the first slice and the second slice correspond to different viewing angles. Figure 3 Step 330 will not be described again here.

[0056] In some embodiments, the medical image annotation system 200 may further include other modules, such as an indication module 230 .

[0057] The instruction module 230 is used to instruct the user terminal to display the second slice and the second annotation path. Figure 3 Step 340 will not be described again here.

[0058] It should be understood that the illustrated medical image annotation system 200 and its modules can be implemented in various ways. For example, in some embodiments, the system and its modules can be implemented using hardware, software, or a combination of software and hardware. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated hardware. Those skilled in the art will appreciate that the above-described methods and systems can be implemented using computer-executable instructions and / or contained in processor control code, such as provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The system and its modules of the present application can be implemented not only with hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips or transistors, or programmable hardware devices such as field programmable gate arrays or programmable logic devices, but can also be implemented using software, such as executed by various types of processors, or a combination of the above-described hardware circuits and software (e.g., firmware).

[0059] It should be noted that the above description of the medical image annotation system 200 and its modules is for convenience only and does not limit this specification to the scope of the embodiments illustrated. It is understood that those skilled in the art, after understanding the principles of the system, may arbitrarily combine the modules or form subsystems connected to other modules without departing from these principles. For example, the acquisition module 210 and the determination module 220 may share a storage module, or each module may have its own storage module. Such variations are within the scope of protection of this application.

[0060] Figure 3 FIG3 is an exemplary flow chart of a method for multi-dimensionally labeling a medical image according to some embodiments of the present specification. In some embodiments, process 300 may be executed by at least one processor.

[0061] In some embodiments, process 300 is implemented based on a one-stop service platform provided by the medical image annotation system 200 .

[0062] like Figure 3 As shown, process 300 may include:

[0063] Step 310 : Acquire a target medical image to be annotated, where the target medical image is a three-dimensional image. In some embodiments, step 310 may be performed by the acquisition module 210 .

[0064] In some embodiments, the target medical image can be acquired by scanning a target object using the scanning device 130. For example, the target medical image can be a CT image or a magnetic resonance (MR) image. In some embodiments, the scanned target object can be a part, tissue, or organ of a human or animal (e.g., the head, neck, chest, abdomen, pelvis, etc.).

[0065] In some embodiments, the acquisition module 210 can obtain the target medical image to be annotated in various ways. For example, the acquisition module 210 can obtain a scanned image from the scanning device 130 as the target medical image to be annotated. In another example, the acquisition module 210 can obtain a medical image uploaded by a user from the user terminal 140 as the target medical image to be annotated. In another example, the acquisition module 210 can obtain a medical image stored in the remote database 150 as the target medical image to be annotated.

[0066] Step 320 : Acquire a first annotation path of a first slice of the target medical image by the user. In some embodiments, step 320 may be performed by the acquisition module 210 .

[0067] The first slice may be a two-dimensional image corresponding to the first section in the target medical image. For example, the scanned object may be a human body, and the first slice may correspond to a cross-section, sagittal plane, coronal plane, or any other section of the human body.

[0068] The first annotation path refers to a curve or straight line that outlines the ROI in the first slice. For example, the outline of the lymph node outlined by the user in the first slice based on free annotation or semi-automatic annotation can be used as the first annotation path. For another example, the user can adjust the initial annotation path of the lymph node outline (such as the annotation path automatically generated by the annotation platform), and the adjusted annotation path can be used as the first annotation path. In some embodiments, the first annotation path can be a closed curve (for example, a closed curve surrounding the ROI) or an unclosed curve or straight line.

[0069] In some embodiments, the first annotation path can be viewed as a contour line of the ROI composed of a plurality of pixels. In some embodiments, the pixel values ​​of the pixels on the first annotation path can be modified to distinguish the pixels on the first annotation path from other pixels on the first slice of the target medical image. The pixel values ​​can be modified based on user needs.

[0070] In some embodiments, the first annotation path can be represented or stored using the position information of the pixels on it. In some embodiments, the first annotation path can be represented by the coordinates of each pixel on it. For example, the first annotation path is composed of n pixels, where the coordinates of each pixel include the layer number, row number, and column number of the pixel.

[0071] The acquisition module 210 can obtain, based on the user terminal 140, a first annotation path for a first slice of the target medical image. For example, the acquisition module 210 can obtain, through the user terminal 140, the outline of a lymph node drawn by the user in the first slice using a free annotation method as the first annotation path. The acquisition module 210 can also obtain the first annotation path for the first slice of the target medical image from the remote database 150. For example, the first slice annotated with the first annotation path can be stored in the remote database 150, and the acquisition module 210 can read the first annotation path for the first slice from the remote database 150.

[0072] Step 330 : Based on the first annotation path, determine a second annotation path corresponding to a second slice of the target medical image, where the first slice and the second slice correspond to different viewing angles. In some embodiments, step 330 may be performed by the determination module 220 .

[0073] The second slice may be a two-dimensional image corresponding to the second section in the target medical image. For example, the scanned object may be a human body, and the second slice may correspond to a cross-section, sagittal plane, coronal plane, or any other section of the human body.

[0074] In some embodiments, the first slice and the second slice correspond to different viewing angles. If a first section corresponding to the first slice and a second section corresponding to the second slice intersect, the first slice and the second slice are considered to correspond to different viewing angles. For example, the first slice corresponds to a cross-section of the human body, and the second slice corresponds to a sagittal or coronal cross-section of the human body.

[0075] The second annotation path refers to a curve or line that annotates the ROI in the second slice. In some embodiments, the second annotation path can be viewed as a curve or line formed by a plurality of annotated pixels on the second slice. In some embodiments, the pixels forming the second annotation path correspond to the same physical points in physical space as the pixels forming the first annotation path. Therefore, the corresponding second annotation path in the second slice of the target medical image can be determined based on the first annotation path.

[0076] In some embodiments, the second annotation path can be a closed curve (e.g., a closed curve enclosing the ROI), a straight line, or one or more pixels. In some embodiments, the second annotation path can be represented or saved using the position information of the pixels thereon. In some embodiments, the second annotation path can be represented by the coordinates of each pixel thereon. For example, the second annotation path is composed of n pixels, wherein the coordinates of each pixel include information about the layer number, row number, and column number of the pixel.

[0077] In some embodiments, the determination module 220 may automatically determine corresponding second annotation paths in multiple second slices of the target medical image based on the first annotation path. The multiple second slices may be two-dimensional images of multiple different layers corresponding to the second section in the target medical image. For example, the determination module 220 may automatically generate a second annotation path for the first layer image, the second layer image, and the nth layer image of the target medical image in the sagittal plane based on the first annotation path in the transverse plane.

[0078] In some embodiments, the determination module 220 can determine the corresponding second annotation path in the second slice of the target medical image based on the acquired first annotation path in a variety of ways. In some embodiments, the determination module 220 can also determine the corresponding second annotation path in the second slice of the target medical image based on the first annotation path in a manner such as automatic generation. For example, based on the multi-dimensional annotation function of the one-stop annotation platform, it is possible to automatically determine the corresponding second annotation path in the second slice of the target medical image based on the first annotation path. For an explanation of automatically determining the corresponding second annotation path in the second slice of the target medical image based on the first annotation path, see Figure 4 .

[0079] In some embodiments, process 300 may further include the following steps:

[0080] Step 340 : Instruct the user terminal to display the second slice and the second annotated path. In some embodiments, step 340 may be performed by the instruction module 230 .

[0081] For example, after determining the second annotated path for the second slice, the instruction module 230 may issue a control instruction to the user terminal to display the second slice and the second annotated path. In some embodiments, the instruction module 230 may instruct the user terminal to simultaneously display the annotated paths in the first slice and the second slice, making it easier for the user to compare and view the ROI, thereby facilitating more accurate diagnostic analysis results.

[0082] Step 350 : Acquire the review result of the second annotated path in the second slice input by the user via the user terminal. In some embodiments, step 330 may be performed by the acquisition module 210 .

[0083] The user can input the review result of the second annotation path in the second slice based on the user terminal. The review result may include review passed, review failed, review cancelled, etc. For more information about review, see Figure 1 In some embodiments, the review results may also include the identity information of the reviewer. For example, a label showing the identity of the reviewer may be added to the target medical image so that other users can view the reviewer's information while viewing the review results.

[0084] By displaying the second slice and the second annotation path, as well as the review status or review results of the second annotation path on the user terminal, the user can refer to the first slice information and the second slice information at the same time to conduct a more comprehensive analysis of the scanned object and obtain more accurate analysis results.

[0085] Figure 4is an exemplary flow chart of a process 400 for generating a second annotation path according to some embodiments of this specification. In some embodiments, the process 400 can be used to implement Figure 3 In some embodiments, the process 400 may be performed by the determination module 220.

[0086] like Figure 4 As shown, process 400 may include the following steps:

[0087] Step 410: construct a cache array based on the target medical image.

[0088] The buffer array is an array used to store information about each point in the target medical image. The multiple points in the target medical image refer to multiple pixels in the target medical image. In some embodiments, the target medical image can be viewed as a three-dimensional image composed of multiple layers of two-dimensional images. The image size can be represented by width * height * depth (i.e., the number of layers), where each layer is composed of a number of pixels.

[0089] In some embodiments, the cache array includes multiple elements corresponding to multiple points in the target medical image. In some embodiments, the elements in the cache array can have a one-to-one correspondence with the pixels of each layer of the target medical image. For example, the element value of an element in the cache array can be equal to the pixel value of a corresponding pixel in the target medical image. In some embodiments, the correspondence between the elements in the cache array and the pixels of the target medical image can be pre-set.

[0090] In some embodiments, the cache array can have the same size as the target medical image. For example, if the target medical image is a three-dimensional image of 256*256*256 (width*height*depth), the cache array can be three-dimensional data of 256*256*256 (width*height*depth). In some embodiments, the cache array can be a one-dimensional array. For example, if the target medical image is composed of n layers of two-dimensional images, each layer has 20 pixels, the cache array can be a one-dimensional array of 20n*1. The 1st to 20th elements of the cache array record the pixel values ​​of the first layer, the 21st to 40th elements record the pixel values ​​of the second layer, and so on.

[0091] In some embodiments, each element in the cache array has a unique index value. That is, the index value can be used as a unique identifier for the element, and the unique element in the cache array can be found through the index value. In some embodiments, in a one-dimensional cache array, the index value of an element can be equal to its element sequence number in the cache array. As an example only, the index value of the first element can be 0, and the index value of the 20th element can be 19. In some embodiments, the index value of the element corresponding to the pixel in the cache array can be determined based on the position information of the pixel (for example, the number of layers, rows, and columns) and the format information of the target medical image. For example, if the target medical image consists of n layers of two-dimensional images, and each layer has 5*4 pixels, the index value of the element corresponding to the pixel located at the xth layer, yth row, and zth column is (20*x+5*y+z).

[0092] Step 420: Determine at least one target element from the multiple elements, where the at least one target element corresponds to a point on the first annotation path.

[0093] The target element refers to the element in the cache array whose element value needs to be updated, which corresponds to the pixel point on the first annotation path. In some embodiments, the element in the cache array corresponding to the pixel point on the first annotation path can be determined, and the element can be used as the target element. In some embodiments, when the first annotation path is modified, the element in the cache array corresponding to the pixel point involved in the modification can be used as the target element. For example, when a point on the first annotation path is removed or a pixel point in the first slice of the target medical image is added to the first annotation path, the element in the cache array corresponding to the removed or added pixel point can be used as the target element.

[0094] The determination module 220 may determine at least one target element in various ways. For example, the determination module 220 may traverse all pixels on the first annotation path and use the element corresponding to the pixel in the cache array as the target element.

[0095] In some embodiments, the determination module 220 may determine at least one target element by first determining an index value of a point on the first annotation path, and then traversing the index values ​​of the points on the first annotation path.

[0096] The index value of a point on the first annotation path can refer to the index value of the element corresponding to that point in the cache array. As described above, there is a one-to-one correspondence between elements in the cache array and points in the target medical image, and the index value can serve as a unique identifier for the element in the cache array. By determining the index value of a point on the first annotation path, that is, determining the index value of the corresponding element in the cache array, the element corresponding to that point can be quickly located in the cache array.

[0097] In some embodiments, the determination module 220 can determine the index value of the point on the first annotation path based on the position information of the point on the first annotation path in the target medical image. For example, if the target medical image is composed of k layers of two-dimensional images, each layer has 5*5 pixels, and a pixel point on the first annotation path is located at the xth layer, yth row, and zth column of the target medical image, then the index value of the pixel point can be (25*x+5*y+z). For another example, if a pixel point on the first annotation path is located at the 2nd layer, 3rd row, and 120th column of the target medical image, then the index value of the pixel point can be {2:{3:{120}}}.

[0098] In some embodiments, the determination module 220 may determine the at least one target element by traversing the index values ​​of the points on the first annotation path.

[0099] In some embodiments, the determination module 220 can process the index values ​​of the points on the first annotation path one by one to determine the corresponding target element in the cache array based on the index value. In some embodiments, the determination module 220 can find the element corresponding to the index value in the cache array as the target element based on the read index value. For example, the aforementioned determined index value {2:{3:{120}}}, that is, the target element may be the element in the cache array corresponding to the pixel point located in the 2nd layer, the 3rd row, and the 120th column of the target medical image.

[0100] In some embodiments, the index value can be a range, for example, the index value can be {99:{4:[136,142]}}, that is, the target element may be the element in the cache array corresponding to the pixel points between columns 136 and 142 in the 4th row of the 99th layer in the target medical image.

[0101] Compared to traversing the entire cache array to determine the target element, this embodiment determines the target element by traversing the index values ​​of the points on the first annotation path, which can reduce the amount of data processing and effectively improve data processing efficiency.

[0102] In some embodiments, the determination module 220 may generate edge data to record position information related to the first annotation path, and determine the index value of the point on the first annotation path based on the edge data, thereby determining the target element in the cache array. For more information about edge data and determining the target element based on edge data, see Figure 5 .

[0103] Step 430: Update the at least one target element in the cache array.

[0104] After determining the target element, the determination module 220 can modify the element value of the target element. For example, the determination module 220 can modify the element value of the target element to a preset label value. In some embodiments, the preset label value can be a pixel value that causes the pixel corresponding to the target element to appear as a preset color in the rendered image, for example, a pixel value that causes the pixel to be red.

[0105] Step 440: Determine the second annotation path in the second slice based on the updated cache array.

[0106] The determination module 220 may determine a second target element corresponding to the second annotation path in the updated cache array, where the second target element is an element whose element value is equal to a preset label value and is located in the second slice.

[0107] The determination module 220 can determine the pixels corresponding to the second target element based on the correspondence between the elements in the cache array and the pixels of the target medical image. All pixels corresponding to the second target element constitute the second annotation path in the second slice. For example, if the second slice is the mth layer of the target medical image, the determination module 220 can first determine the element corresponding to the mth layer in the updated cache array as a candidate element. Then, the determination module 220 can determine the element whose element value equals the preset label value among the candidate elements as the second target element. All pixels corresponding to the second target element in the target medical image constitute the second annotation path. For another example, the determination module 220 can first render and update the target medical image based on the updated cache array, and then intercept the second slice from the rendered and updated target medical image. Because the element value of the second target element can be set to a preset value, the pixel values ​​of the pixels corresponding to the second target element can also be set to a preset value (e.g., a pixel value that causes it to appear red in the rendered image). Therefore, in the intercepted second slice, the second annotation path is composed of pixels displayed in the target color.

[0108] In some embodiments, by automatically generating the annotation path of the target medical image from other perspectives based on the annotation path outlined by the user from one perspective of the target medical image, multi-dimensional annotation is achieved, which can help improve the annotation accuracy, improve the user's three-dimensional cognition of the annotated image, and improve the efficiency of human-computer interaction.

[0109] Figure 5 is an exemplary flow chart of a process 500 for determining the index value of a point on a first annotation path according to some embodiments of this specification. In some embodiments, the process 500 can be used to assist in implementing Figure 4 In some embodiments, the process 500 may be performed by the determination module 220.

[0110] like Figure 5 As shown, process 500 may include the following steps:

[0111] Step 510: Generate edge data based on the first annotated path. The edge data includes position information related to the first annotated path.

[0112] For example, the edge data may include the position coordinates of at least two feature pixels on the first annotation path. The feature pixels may include all or part of the pixels on the first annotation path. The position coordinates of the feature pixels may include information about the layer, row, and column where they are located. For example, a feature pixel is located in layer 1, row 10, column 8 of the target medical image.

[0113] In some embodiments, determination module 220 may segment the first annotated path into at least two path segments. Determination module 220 may then generate edge data based on the at least two path segments, where the edge data includes location information of the start and end points of each path segment. The start and end points of the path segments may serve as the feature pixels described above.

[0114] In some embodiments, the path segments may have a specific length and / or shape. For example, the determination module 220 may segment the first annotation path according to a preset path length to obtain multiple path segments of the same length. For another example, the determination module 220 may segment the first annotation path according to a preset shape to obtain multiple path segments of similar shapes. In some embodiments, when the determination module 220 segments the first annotation path according to the shape of the path, the first annotation path may be segmented in the manner of straight line segmentation. For example, continuous pixel points on a straight line in the first annotation path may be divided into a path segment, and the curved segments on the first annotation path may be divided into multiple path segments that are approximately straight lines.

[0115] The determination module 220 can generate edge data based on the path segments obtained by segmenting the first annotated path. In some embodiments, the edge data may include the location information of the start and end points of each path segment, and the determination module 220 can generate the edge data by reading the location coordinates of the start and end points of each path segment. As an example only, the data structure of the edge data can be {Slice:{y:[start1,end1,start2,end2,start3,end3,...]}}. Among them, Slice represents the layer, y represents the row, start and end represent the starting and ending columns respectively. For example, {99:{4:[136,162,191,208,367,411]}} represents the 99th layer, and there are three path segments in the 4th row, where the first segment starts from the 136th column and ends at the 162nd column, the second segment starts from the 191st column and ends at the 208th column, and the third segment starts from the 367th column and ends at the 411th column.

[0116] In some embodiments, the edge data may be stored in a storage device after being generated. When the user loads a slice of the target medical image again, the determination module 220 may obtain the edge data from the storage device and generate a labeling result on the slice based on the edge data.

[0117] In some embodiments, by constructing edge data to record the location information of the starting and ending points of each path segment, the segmentation information of the first annotated path and the location information of each path segment can be quickly obtained. Since the edge data only needs to store the locations of the characteristic pixels on the first annotated path, and based on the segmentation method of the path segments, the locations of all pixels in the first annotated path can be inferred based on the locations of the characteristic pixels. Compared to the method that requires saving the locations of all pixels in the first annotated path, the edge data storage method provided by this embodiment can reduce the amount of data stored and improve system performance.

[0118] Step 520: Determine the index value of the point on the first annotation path based on the edge data.

[0119] In some embodiments, the determination module 220 may first obtain format information of the target medical image, and then determine the index value of the point on the first annotation path based on the format information and edge data of the medical image.

[0120] The format information of the target medical image refers to the size information of the target medical image, for example, information such as the width, height, and number of layers of the target medical image.

[0121] The index value of the point on the first annotation path refers to the index value of the element corresponding to the point on the first annotation path in the cache array. For more information about the index value of the point on the first annotation path, see Figure 4The relevant content of step 420 is shown.

[0122] In some embodiments, as described above, the edge data may include position information of at least two feature pixels on the first annotated path. For each feature pixel, the determination module 220 may determine an index value for the feature pixel based on the format information of the target medical image and the position information of the feature pixel. In some embodiments, the edge data may include position information for the start and end points of multiple path segments on the first annotated path. For each path segment, the determination module 220 may determine an index value range for the points on the path segment based on the format information of the target medical image and the start and end points of the path segment. The process of calculating the index values ​​of points on the first annotated path can be viewed as the process of calculating the index values ​​of the start and end points of each path segment in a cache array. For example, a path segment may start at row y, column start 1, and end at row y, column end 1, of the slice layer. The determination module 220 may calculate the index value range for the points on the path segment based on the following conversion formula: pos = w*h*slice+y*w+x, where w and h are the width and height of the target medical image, respectively, and x represents the number of columns. The value range of x may be [start1, end1].

[0123] By determining the index value of the point on the first annotation path, it is possible to quickly locate the element corresponding to the point on the first annotation path in the cache array based on the index value. Specifically, based on the special structure of the cache array, it is possible to quickly determine the target element corresponding to the point on the first annotation path in the cache array based on the position information of the point on the first annotation path and the image format information of the target medical image, thereby achieving rapid positioning of the target element. At the same time, the present invention utilizes feature pixel points to construct edge data to store the position information of the point on the first annotation path, which can reduce the storage amount of position information and the subsequent data calculation amount, thereby improving the efficiency of multi-dimensional annotation.

[0124] An embodiment of this specification also provides a medical image annotation device, which includes a processor and a memory, wherein the memory is used to store instructions. When the instructions are executed by the processor, the device implements operations corresponding to any of the medical image annotation methods described above.

[0125] While the basic concepts have been described above, it will be apparent to those skilled in the art that the detailed disclosure is merely illustrative and does not limit this specification. Although not explicitly stated herein, various modifications, improvements, and revisions to this specification may be made by those skilled in the art. Such modifications, improvements, and revisions are suggested in this specification and remain within the spirit and scope of the exemplary embodiments of this specification.

[0126] This specification also uses specific terms to describe the embodiments of this specification. For example, "one embodiment," "an embodiment," and / or "some embodiments" refer to a feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "one embodiment," "an embodiment," or "an alternative embodiment" two or more times in different locations in this specification do not necessarily refer to the same embodiment. Furthermore, certain features, structures, or characteristics of one or more embodiments of this specification may be appropriately combined.

[0127] In addition, unless expressly stated in the claims, the order of the processing elements and sequences, the use of alphanumeric characters, or the use of other names described in this specification are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some of the invention embodiments currently considered useful through various examples, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the spirit and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0128] Similarly, it should be noted that, in order to simplify the presentation of this specification and thus facilitate understanding of one or more embodiments of the invention, the foregoing descriptions of the embodiments of this specification sometimes combine multiple features into a single embodiment, figure, or description thereof. However, this disclosure method does not imply that the subject matter of this specification requires more features than those recited in the claims. In fact, an embodiment may have fewer features than all of the features of a single disclosed embodiment.

[0129] In some embodiments, numbers are used to describe the quantity of components and attributes. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise stated, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the description and claims are approximate values, which may change according to the required characteristics of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of this specification are approximate values, in specific embodiments, the settings of such numerical values ​​are as accurate as possible within the feasible range.

[0130] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, and documents, cited in this specification is hereby incorporated by reference in its entirety. This includes application history documents that are inconsistent with or conflict with the content of this specification, as well as documents (currently or subsequently attached to this specification) that limit the broadest scope of the claims of this specification. It should be noted that if the descriptions, definitions, and / or terminology used in the accompanying materials are inconsistent or conflicting with the content of this specification, the descriptions, definitions, and / or terminology used in this specification will control.

[0131] Finally, it should be understood that the embodiments described in this specification are intended only to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, by way of example and not limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly described and illustrated in this specification.

Claims

1. A medical image annotation method, executed by at least one processor, characterized in that: The method comprises: Acquire a target medical image to be annotated, where the target medical image is a three-dimensional image; Obtaining a first annotation path of a first slice of the target medical image by a user; and Automatically determining a corresponding second annotation path in a second slice of the target medical image based on the first annotation path includes: constructing a cache array based on the target medical image, the cache array including a plurality of elements corresponding to a plurality of pixel points in the target medical image; Determine at least one target element from the multiple elements, where the at least one target element corresponds to a pixel point on the first annotation path; Updating the at least one target element in the cache array; Determining the second annotated path in the second slice based on the updated cache array; The first slice and the second slice correspond to different viewing angles.

2. The method according to claim 1, wherein The determining at least one target element from the plurality of elements comprises: Determining the index value of the point on the first annotation path; and The at least one target element is determined by traversing the index values ​​of the points on the first annotation path.

3. The method according to claim 2, wherein Determining the index value of the point on the first annotation path includes: generating edge data based on the first annotated path, the edge data including position information related to the first annotated path; and The index value of the point on the first annotation path is determined based on the edge data.

4. The method according to claim 3, wherein The generating edge data based on the first annotation path includes: Splitting the first annotation path into at least two path segments; and The edge data is generated based on the at least two path segments, where the edge data includes position information of a start point and an end point of each path segment.

5. The method according to claim 3, wherein Determining the index value of the point on the first annotation path based on the edge data includes: Acquiring format information of the target medical image; and The index value of the point on the first annotation path is determined based on the format information of the medical image and the edge data.

6. The method according to claim 1, wherein The method further comprises: instructing a user terminal to display the second slice and the second annotated path; and A review result of the second annotation path in the second slice input by the user via the user terminal is obtained.

7. A medical image annotation system, characterized in that: The system comprises: an acquisition module, configured to acquire a target medical image to be annotated and obtain a first annotation path of a first slice of the target medical image by a user, wherein the target medical image is a three-dimensional image; A determination module, configured to automatically determine a corresponding second annotation path in a second slice of the target medical image based on the first annotation path, comprising: constructing a cache array based on the target medical image, the cache array including a plurality of elements corresponding to a plurality of pixel points in the target medical image; Determine at least one target element from the multiple elements, where the at least one target element corresponds to a pixel point on the first annotation path; Updating the at least one target element in the cache array; Based on the updated cache array, the second annotation path in the second slice is determined; the first slice and the second slice correspond to different viewing angles.

8. A medical image annotation system, comprising: a remote database configured to store at least one medical image and annotation information thereof; a user terminal configured to receive a labeling instruction input by a user for a target medical image in the at least one medical image; as well as a server, the server being configured to update the annotation information of the target medical image in the remote database in response to the annotation instruction; The annotation instruction includes a first annotation path for a first slice of the target medical image; In order to update the annotation information of the target medical image in the remote database in response to the annotation instruction, the server is further configured to: Automatically determining a corresponding second annotation path in a second slice of the target medical image based on the first annotation path includes: constructing a cache array based on the target medical image, the cache array including a plurality of elements corresponding to a plurality of pixel points in the target medical image; Determine at least one target element from the multiple elements, where the at least one target element corresponds to a pixel point on the first annotation path; Updating the at least one target element in the cache array; Based on the updated cache array, the second annotation path in the second slice is determined; the first slice and the second slice correspond to different viewing angles.

9. The system according to claim 8, wherein The server is further configured to: Instruct the user terminal to display the second slice and the second annotation path.

Images