Methods and devices for processing medical images

Abstract

This application discloses a method and apparatus for processing medical images, relating to the field of medical image processing technology. The method can acquire various post-processed data obtained from post-processing medical image sequences, and obtain a unified dataset based on these post-processed data, thereby controlling the visualization display of a 3D model of the unified dataset. This unified dataset includes various post-processed data in a uniform format and converted to the same coordinate system. Therefore, on the one hand, it facilitates the overall management of various post-processed data; on the other hand, it enables integrated visualization of various post-processed data based on a single 3D model. Compared to displaying multiple models separately, this reduces data redundancy and rendering overhead, improves data processing and display efficiency, and ensures display consistency.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to a method and apparatus for processing medical images. Background Technology

[0002] With the rapid development of medical imaging technology, multimodal medical imaging such as CT and MRI are playing an increasingly central role in clinical diagnosis, surgical planning, medical education, and scientific research. Analyzing and processing two-dimensional medical images (i.e., tomographic sequences), especially the three-dimensional reconstruction and visualization of specific anatomical structures (such as tumors, blood vessels, and bones) or tissues, can provide doctors with a three-dimensional, quantifiable analytical perspective, greatly improving the accuracy of diagnosis and the predictability of treatment plans. Summary of the Invention

[0003] This application provides a method and apparatus for processing medical images, the technical solution of which is as follows:

[0004] On the one hand, a method for processing medical images is provided, the method comprising: Obtain the medical image sequence of the target object; The medical image sequence is post-processed to obtain various post-processed data of the medical image sequence; Based on the various post-processed data, a unified dataset is obtained, wherein the unified dataset includes: the various post-processed data with a unified format and converted to the same coordinate system; Control the visualization of the three-dimensional model of the unified dataset.

[0005] Optionally, based on the various post-processed data, a unified dataset is obtained, including: The various post-processed data are subjected to format unification processing; Spatial normalization is performed on the various post-processed data after the format unification process. The various post-processed data after spatial normalization are integrated to obtain a unified dataset.

[0006] Optionally, the number of medical image sequences is multiple; before performing spatial normalization on the various post-processing data after format unification processing, the method further includes: Obtain the registration transformation relationship between the first image sequence and each of the second image sequences in the plurality of medical image sequences; The spatial normalization process performed on the various post-processed data after format unification includes: Based on the transformation relationship between the pixel coordinate system and the reference coordinate system of the first image sequence, the various post-processed data of the first image sequence after unified format processing are transformed into the reference coordinate system; Based on the registration transformation relationship of each of the second image sequences, and the transformation relationship between the pixel coordinate system of the first image sequence and the reference coordinate system, the various post-processed data of the second image sequences after unified format processing are transformed into the reference coordinate system.

[0007] Optionally, the various post-processing data of the medical image sequence are standardized to obtain a unified dataset, including: Write the unified dataset into the PACS server.

[0008] Optionally, writing the unified dataset to the PACS server includes: Create a new DICOM file; Write the unified dataset into the DICOM file; Upload the DICOM file to the PACS server.

[0009] Optionally, the method further includes: Write the identifier of the medical image sequence to which the unified dataset belongs into the DICOM file.

[0010] Optionally, controlling the visualization of the 3D model of the unified dataset includes: The receiving terminal sends a request to acquire a 3D model of the unified dataset, the acquisition request including the file identifier of the DICOM file; In response to the acquisition request, the DICOM file is obtained from the PACS server based on the file identifier, and the model file of the 3D model is obtained based on the DICOM file; Upload the model file to the file server and obtain the access address of the model file; The access address is sent to the terminal so that the terminal can obtain the model file from the file server based on the access address and perform visualization display of the 3D model based on the model file.

[0011] On the other hand, an imaging workstation is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the medical image processing method as described above.

[0012] In another aspect, a computer-readable storage medium is provided having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the medical image processing method as described above.

[0013] In another aspect, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the medical image processing method as described above.

[0014] In another aspect, a medical image processing system is provided, the processing system comprising: a PACS server, a terminal, and an image workstation as described above.

[0015] The beneficial effects of the technical solution provided in this application include at least the following: This application provides a method and apparatus for processing medical images. The method acquires various post-processed data obtained from post-processing medical image sequences, and obtains a unified dataset based on these post-processed data, thereby controlling the visualization of a 3D model of the unified dataset. This unified dataset includes various post-processed data with a unified format and converted to the same coordinate system. This unifies the format of heterogeneous post-processed data and ensures they correspond to the same physical coordinate space, solving the problem of data silos caused by different formats and enabling the fusion of various post-processed data. Therefore, on the one hand, it facilitates the overall management of various post-processed data; on the other hand, it enables integrated visualization of various post-processed data based on a single 3D model. Compared to displaying multiple models separately, this reduces data redundancy and rendering overhead, improves data processing and display efficiency, and ensures display consistency.

[0016] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the structure of a medical image processing system provided in an embodiment of this application; Figure 2 This is a flowchart of a medical image processing method provided in an embodiment of this application; Figure 3 This is a flowchart of another medical image processing method provided in the embodiments of this application; Figure 4 This is a schematic diagram illustrating a method for obtaining a 3D model file based on a unified dataset, as provided in an embodiment of this application. Figure 5 This is a schematic diagram of the structure of an image workstation provided in an embodiment of this application. Detailed Implementation

[0018] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0019] This application provides a medical image processing system, see [link to relevant documentation]. Figure 1 The processing system may include: an image workstation 100, a picture archiving and communication systems (PACS) server 200, and a terminal 300. The image workstation 100 establishes communication connections with both the PACS server 200 and the terminal 300. A client is installed on the terminal. This client can be a web client (such as a web browser) or other applications. A web client refers to a client based on web-based dynamic page technology.

[0020] The imaging workstation 100 is used for post-processing medical image sequences and obtaining a unified dataset based on the various post-processed data. The PACS server is used to store the unified dataset. The client installed in the terminal 300 is used to visualize the 3D model of the unified dataset. This 3D model can be used as part of a medical report (such as a medical web report).

[0021] Optionally, the image workstation 100 can be a 3D post-processing workstation. The terminal 300 includes, but is not limited to, mobile terminals or fixed terminals. The mobile terminal can be a smartphone, tablet, or laptop. The fixed terminal can be a desktop computer.

[0022] Figure 2 This is a flowchart illustrating a medical image processing method provided in an embodiment of this application. The method is applied to an image workstation, for example... Figure 1 The image workstation shown. See also Figure 2 The method includes: Step 101: Obtain the medical image sequence of the target object.

[0023] The medical image sequence of the target object can be pre-stored on the imaging workstation. Alternatively, the medical image processing system may also include a medical imaging device connected to the imaging workstation. The medical imaging device can scan the area of ​​the target object to obtain a medical image sequence, which can then be sent to the imaging workstation. Accordingly, the imaging workstation can acquire the medical image sequence of the target object.

[0024] There can be one or more medical imaging sequences. When there are multiple medical imaging sequences, the modalities, and / or scan times, and / or the sites they belong to can be different.

[0025] Step 102: Post-process the medical image sequence to obtain various post-processed data of the medical image sequence.

[0026] Various post-processing data can include: analytical data from medical imaging sequences and clinical planning data, etc. Therefore, multiple post-processing data can be combined to form the surgical planning data for the target patient. Each piece of post-processing data can be used as a bookmark.

[0027] The analytical data may include, but is not limited to, the segmentation results and measurement data of anatomical structures. The measurement data may include at least the measurement values ​​calculated based on the segmentation results, such as organ volume, the volume of the region of interest (such as a lesion), and the distance between the region of interest and key tissues.

[0028] Clinical planning data may include, but is not limited to, implant planning data, surgical pathway data, marking data, and osteotomy guidance data. Implant planning data includes information such as the implant's model, size, location, and orientation. Implants may include surgical instruments (such as various sizes of stents, screws, bone plates, etc.) and prostheses (such as artificial joints). Surgical pathway data may include the surgical path, the location of the puncture point, and the endoscope's entry point. Marking data may include the location of marked geometric elements in the medical imaging sequence; geometric elements may include marker points, lines, angles, and regions of interest (such as the extent of tissue resection). Osteotomy guidance data may include the plane, angle, and depth of the osteotomy.

[0029] Therefore, the subsequently generated unified dataset is a structured data object containing segmentation results of multiple anatomical structures, measurement data, and clinical planning data, and has a hierarchical relationship.

[0030] Step 103: Obtain a unified dataset based on various post-processing data from medical image sequences.

[0031] The unified dataset includes multiple post-processed data that are uniformly formatted and transformed into the same coordinate system.

[0032] In this embodiment, the image workstation can standardize various post-processing data from medical image sequences to obtain a unified dataset. This standardization process includes: format unification, spatial normalization, and integration. Format unification converts the data formats of various post-processing data into a single format. Spatial normalization transforms all post-processing data into the same coordinate system. Integration combines the various post-processing data into a unified dataset.

[0033] This results in a unified format for post-processing data obtained from medical image sequences of different parts of the target object, and / or at different times, and / or in different modalities, all corresponding to the same physical coordinate space. This allows for the fusion of various post-processing data, facilitating management and reuse.

[0034] In this embodiment, the unified dataset can be temporarily stored in the local or centralized database of the image workstation for quick access; on the other hand, the unified dataset can be written into a newly created DICOM file and stored on the PACS server to achieve permanent archiving of the unified dataset.

[0035] Step 104: Control the visualization of the 3D model of the unified dataset.

[0036] The three-dimensional model of the unified dataset is obtained by reconstructing the unified dataset in three dimensions.

[0037] In one alternative implementation, the imaging workstation displays a "3D View" button. After the staff completes the surgical planning requirements on the imaging workstation, i.e., after the workstation generates a unified dataset, the staff can touch the "3D View" button. In response to the touch of the "3D View" button, the imaging workstation can perform 3D reconstruction based on the unified dataset temporarily stored locally or in a centralized database, generating and displaying a 3D model, thereby achieving visualization of the 3D model.

[0038] In another alternative implementation, the image workstation can receive a request from the terminal to acquire a 3D model and, in response, retrieve a unified dataset. For example, it can retrieve a DICOM file containing the unified dataset from the PACS server and reconstruct the unified dataset from the DICOM file. Subsequently, the image workstation can generate a 3D model based on the unified dataset, export the model file, upload the model file to a file server, and obtain the access address of the model file. The image workstation can then send this access address to the terminal, allowing the terminal to retrieve the model file and subsequently visualize the 3D model.

[0039] In summary, this application provides a method for processing medical images. This method can acquire various post-processed data obtained from post-processing medical image sequences, and obtain a unified dataset based on these multiple post-processed data, thereby controlling the visualization display of a 3D model of the unified dataset. The unified dataset includes multiple post-processed data with a unified format and converted to the same coordinate system. This unifies the format of heterogeneous post-processed data and ensures they correspond to the same physical coordinate space, solving the problem of data silos caused by different formats and enabling the fusion of multiple post-processed data. Therefore, on the one hand, it facilitates the overall management of multiple post-processed data; on the other hand, it enables integrated visualization of multiple post-processed data based on a single 3D model. Compared to displaying multiple models separately, this reduces data redundancy and rendering overhead, improves data processing and display efficiency, and ensures display consistency.

[0040] This application uses the display of a 3D model of a unified dataset on a client side as an example to illustrate the method provided in this application. This method can be applied to... Figure 1 The processing system shown is described in the following document. Figure 3 The method may include: Step 201: The imaging workstation acquires the medical image sequence of the target object.

[0041] Medical image sequences can be pre-stored on an imaging workstation or scanned by medical imaging equipment and transmitted to the workstation. These medical image sequences can be DICOM (digital imaging and communication medicine) image sequences. A DICOM image sequence includes metadata and pixel data of the target object. The metadata may include a study instance unique identifier (StudyInstanceUID) and a series instance unique identifier (SeriesInstanceUID). These StudyInstanceUID and SeriesInstanceUID uniquely identify a medical image sequence; that is, StudyInstanceUID and SeriesInstanceUID serve as the identifiers for the medical image sequence.

[0042] There can be one or more medical imaging sequences. When there are multiple medical imaging sequences, the modalities, and / or scan times, and / or the sites they belong to can be different.

[0043] Step 202: The image workstation performs post-processing on the medical image sequence to obtain various post-processed data of the medical image sequence.

[0044] For each medical image sequence, the imaging workstation can perform post-processing to obtain various post-processed data. Each type of post-processed data can be considered a bookmark. In this embodiment, a bookmark is the smallest abstract unit representing the result of various post-processing operations on a medical image sequence. That is, through step 202, the imaging workstation can obtain multiple bookmarks for the medical image sequence.

[0045] Post-processing may include at least two or more of the following: segmentation, measurement and labeling, and clinical planning. When post-processing includes segmentation, the post-processing data may include segmentation results; when post-processing includes measurement and labeling, the post-processing data may include both measurement data and labeled data; when post-processing includes clinical planning, the post-processing data may include clinical planning data.

[0046] In this embodiment, the image workstation may include a segmentation module, a measurement and labeling module, and a planning module. Any two of these modules process medical image sequences differently; therefore, the post-processed data output by any two modules typically have different formats, meaning that various post-processed data are usually heterogeneous.

[0047] This segmentation module is used to segment medical image sequences (e.g., using deep learning-based automatic segmentation methods) and outputs segmentation results for at least one region of interest (also known as a segmentation mark). The region of interest can be a target tissue, organ, or lesion. The segmentation result can be mask data (e.g., binary mask volume data) or a surface mesh. The surface mesh is obtained based on the mask data.

[0048] The measurement and labeling module is used to output the measurement values ​​of the segmentation results, and in response to the labeling operations of the staff, to label geometric elements in the medical image sequence and output the label data (also known as geometric primitive mark).

[0049] The planning module outputs clinical planning data in response to staff planning operations. For example, the planning module embeds a parameterized surgical instrument library containing multiple implants. The module responds to staff actions such as implanting implants in medical imaging sequences, inserting surgical instruments, and adjusting the position, orientation, and size of surgical instruments via the graphical interface of the imaging workstation. Ultimately, it outputs the planning data for the implants, i.e., the instrument marks.

[0050] Understandably, each type of post-processed data can be bound to the StudyInstanceUID and SeriesInstanceUID of the corresponding medical image sequence. This allows for accurate association between the post-processed data and the corresponding medical image sequence.

[0051] In this embodiment, staff can select at least two modules in the imaging workstation to post-process medical image sequences according to actual planning needs, obtaining post-processed data. This saves processing resources on the imaging workstation. Alternatively, the imaging workstation can post-process medical image sequences through all modules. Staff can select at least two types of post-processed data from the various post-processed data obtained according to actual planning needs. Correspondingly, the imaging workstation can acquire the post-processed data of the medical image sequence in response to the staff's selection operation.

[0052] Step 203: The image workstation obtains a unified dataset based on various post-processing data.

[0053] This unified dataset can include multiple post-processed data that are in a uniform format and have been transformed into the same coordinate system.

[0054] In this embodiment, the image workstation can standardize various post-processing data to obtain a unified dataset. This standardization process includes: format unification, spatial normalization, and integration. Specifically, the process of the image workstation executing step 203 may include: Step S1: Standardize the format of various post-processing data.

[0055] After obtaining various post-processing data from medical image sequences, the imaging workstation can encode these data according to a unified encoding rule, converting all data formats into the same format. This achieves standardized processing of various post-processing data formats.

[0056] The unified coding rules can be pre-stored in the imaging workstation. These unified coding rules can specify identifiers for multiple tissues, etc. For example, the unified coding rules can specify the liver as "liver".

[0057] Step S2: Perform spatial normalization on the various post-processed data after the format unification process.

[0058] The image workstation can convert various post-processing data after unified encoding to a reference coordinate system to complete the spatial normalization of multiple post-processing data. In this way, the spatial differences caused by different coordinate systems can be eliminated, allowing post-processing data from different times, different sequences, and even different modalities to coexist conflict-free in the same physical coordinate space.

[0059] Optionally, the reference coordinate system can be the object coordinate system of the target object (also known as the patient's physical coordinate system). This avoids the problem of scale distortion caused by scaling operations when displaying the 3D model later.

[0060] In this embodiment of the application, the image workstation can obtain the transformation relationship between the pixel coordinate system and the reference coordinate system of the medical image sequence, and based on the transformation relationship, convert various post-processing data to the reference coordinate system.

[0061] Specifically, for a medical image sequence that is a single medical image sequence, or a medical image sequence that is multiple sequences of the same modality (such as all being CT sequences), and where the position of the target object relative to the medical imaging equipment does not change during the scanning process of obtaining multiple medical image sequences, the image workstation can directly convert various post-processing data to the reference coordinate system based on this transformation relationship.

[0062] For multiple medical image sequences obtained through cross-device scanning (i.e., multiple modal sequences), or situations where the target object moves during scanning (i.e., its position changes relative to the medical imaging equipment), the imaging workstation can obtain the registration transformation relationship between the first image sequence and each of the second image sequences before spatially normalizing the various post-processing data after format standardization. Then, based on the transformation relationship between the pixel coordinate system of the first image sequence and the reference coordinate system, the imaging workstation can transform the various post-processing data of the first image sequence to the reference coordinate system. Furthermore, based on this transformation relationship, and the registration transformation relationship between each second image sequence and the first image sequence, the workstation can transform the various post-processing data of each second image sequence to the reference coordinate system. This achieves spatial normalization of the post-processing data from multiple medical image sequences.

[0063] In this embodiment, the imaging workstation can use a first image sequence among multiple medical image sequences as a reference to register each second image sequence to the first image sequence, thereby obtaining the registration transformation relationship between each second image sequence and the first image sequence.

[0064] The first image sequence can be any one of multiple medical image sequences, or it can be a medical image sequence obtained from the first scan of the target object. The second image sequence can be any medical image sequence other than the first image sequence.

[0065] For example, the image workstation may also include a registration module, which is used to register multiple medical image sequences to align them and output the registration transformation relationship. The registration transformation relationship can be represented by a registration transformation matrix. The registration transformation matrix can be a rigid matrix. Furthermore, the registration module supports both automatic and manual registration methods.

[0066] Step S3: Integrate the various post-processed data after spatial normalization to obtain a unified dataset.

[0067] The image workstation can merge multiple post-processed data after spatial normalization to integrate them into a unified dataset (also known as a unified bookmark).

[0068] In this context, the unified bookmark is defined as a structured data object in the memory and storage of the image workstation. The data in this object is stored or transmitted using a structured data serialization format. The structured data serialization format can be JSON or Protocol Buffers (PB) format.

[0069] As described in steps S1 to S3 above, the image workstation can unify the format of various post-processing data from different sources, modalities, formats, and coordinate systems, convert them to the same coordinate system, and integrate them into a unified dataset that can be analyzed jointly. This ensures that post-processing data obtained from different parts of the target object, and / or different times, and / or different modalities of medical image sequences have a consistent and fusionable spatial relationship within the unified dataset. In other words, the various post-processing data in the unified dataset correspond to the same physical coordinate space, and therefore can coexist without conflict in the same three-dimensional spatial scene, thus enabling spatial fusion processing.

[0070] Therefore, the unified dataset is essentially an ordered, hierarchical list of marks, along with the global parameters (i.e., the reference coordinate system) required to generate the final 3D model. The mark list can include the segmentation marks, geometric primitive marks, and prop marks mentioned earlier. Marks output by any module at any time can be appended to the mark list through an add operation, and are automatically converted to the reference coordinate system when added.

[0071] For example, assuming the post-processing data includes liver segmentation results and stent planning data, the unified bookmark can include a structured object of the segmentation results and a structured object of the planning data.

[0072] The structured object of the segmentation result can include field values ​​for structured fields such as ID, name, source, format, and attributes. The source field value indicates how the segmentation result was obtained. The format field value indicates the representation form of the segmentation result (grid or mask). The attribute field value can represent the color and transparency of the segmentation result, etc.

[0073] Structured objects for planning data can include field values ​​for structured fields such as ID, name, and placement information. The placement information field value can indicate the location and orientation of the support structure.

[0074] In this embodiment, the imaging workstation can generate new post-processed data for the medical image sequence in response to modifications made by staff (such as adjusting the segmentation threshold or moving a marker point), and replace the old post-processed data with the new data, thus updating the post-processed data. This enables non-destructive editing of a unified dataset.

[0075] Step 204: The image workstation stores the unified dataset to the PACS server.

[0076] An image workstation can create a new DICOM file, also known as a DICOM SOP instance. The workstation can write a unified dataset to this DICOM file and then upload the DICOM file containing the unified dataset to the PACS server for storage. For example, the workstation can send a DICOM file to the PACS server for storage via a DICMC-STORE request.

[0077] In this embodiment of the application, the image workstation can serialize the unified dataset to obtain serialized data, and write the serialized data into the DICOM file to write the unified dataset into the DICOM file.

[0078] Understandably, this DICOM file includes custom extended fields (also known as private data elements), into which the image workstation can write serialized data. Optionally, the image workstation can first compress the serialized data and then write the compressed data into the private data elements of the DICOM file. For example, the compressed data can be converted to a Base64 string and written into the private data elements.

[0079] Furthermore, the imaging workstation can also write the identifier of the medical image sequence to which the unified dataset belongs into the DICOM file. Specifically, the DICOM file can also include: standard fields defined by the standard (also called standard data elements). The imaging workstation can also correctly set the association information with the medical image sequence in the standard data elements of the DICOM object. For example, the StudyInstanceUID corresponding to the medical image sequence and the sequence description information can be written into the standard fields. This sequence description information can be determined according to the specific post-processing operation. For example, if the post-processing mainly includes planning processing, the sequence description information can be a 3D Surgical Plan Bookmark.

[0080] Specifically, the Referenced Study Sequence entry for this standard data element can be written to the StudyInstanceUID of the original medical image sequence, and the Referenced Series Sequence entry can be written to the Series Instance UID of the medical image sequence. This accurately associates the DICOM file with the medical image sequence, ensuring that the DICOM file and the medical image sequence belong to the same study. Thus, a native, permanent, and traceable binding between the medical image sequence and the unified dataset within the medical information system can be achieved.

[0081] In this embodiment, the PACS server assigns a globally unique file identifier, SOP Instance UID, to the DICOM file. This SOP Instance UID becomes a permanent and unique identifier for the unified bookmark within the entire healthcare information system. In other words, the DICOM file can function as an independent DICOM Series.

[0082] In this embodiment, by writing the unified dataset (i.e., surgical planning data) into a DICOM file and storing it on a PACS server, the unified dataset can be permanently archived, enjoying the same security, durability, backup, and disaster recovery mechanisms as the original medical image sequences. Simultaneously, associating the DICOM file with the medical image sequences enables a native, permanent, and traceable binding between the unified dataset and the medical image sequences within the medical information system. Furthermore, assigning a globally unique identifier (i.e., SOP Instance UID) to the DICOM file allows for precise retrieval, meeting the highest requirements for data traceability in clinical, legal, and research settings, and automating the process from storing the unified dataset to visualizing the 3D model on the client side.

[0083] Step 205: The terminal sends a request to the image workstation to obtain the 3D model of the unified dataset.

[0084] The client installed on the terminal can send an acquisition request to the image workstation. This acquisition request can carry the file identifier of the DICOM file containing the unified dataset.

[0085] Taking a web client as an example, the image workstation can provide a web-based API, such as a RESTful interface of type GET. The web client can call the image workstation's RESTful interface to send a retrieval request to the image workstation. For example, this retrieval request could be: GET / api / bookmark / {sop_instance_uid} / webmodel.

[0086] Step 206: In response to the 3D model acquisition request sent by the terminal, the image workstation sends the access address of the 3D model file to the terminal.

[0087] In response to a 3D model retrieval request from a terminal, the image workstation can automatically retrieve the DICOM file indicated by the file identifier included in the request from the PACS. Subsequently, the image workstation can obtain the 3D model file based on the DICOM file. Afterward, the image workstation can upload the model file to a file server, obtain the access address of the model file, and then return a response message to the terminal regarding the retrieval request. This response message includes the access address of the model file, which can be a Uniform Resource Locator (URL).

[0088] Optionally, the file server can be a static resource server or a content delivery network (CDN). This URL can be permanent or temporary. It should be noted that this URL is strongly bound to the file identifier (i.e., SOP Instance UID) of the DICOM file containing the unified bookmark. For example, strong binding can be achieved by including the hash value of this file identifier in the URL, or through mapping in the backend database record.

[0089] Therefore, when displaying medical reports through the client, if a certain 3D model needs to be displayed, it is only necessary to know the file identifier of the DICOM file where the unified bookmark of the 3D model is located. The corresponding 3D model can be dynamically obtained and loaded, realizing the automatic and error-free association between the report content and the 3D model source (i.e., unified bookmark).

[0090] In this embodiment, the image workstation can retrieve the DICOM file corresponding to the file identifier from the PACS via a DICOM C-FIND request, C-MOVE request, or database query. Once the image workstation obtains the DICOM file, it can parse the private data elements within the DICOM file to reconstruct a unified dataset, and based on the reconstructed unified dataset, obtain the model file of the 3D model. For example, the image workstation can parse the private data elements in the DICOM file and decompress and deserialize the parsed data to reconstruct the unified dataset.

[0091] The image workstation can perform 3D reconstruction on the unified dataset obtained from the reconstruction to obtain a 3D model, and then export the 3D model as a model file. For details, see... Figure 4 The image workstation can traverse all post-processed data in a unified dataset. For each post-processed data point encountered, if it's a segmentation result of an anatomical structure and the segmentation result is a surface mesh, the image workstation can perform lightweight optimization on the segmentation result. If the post-processed data is implant planning data, it can first triangulate the implant planning data to generate a mesh, and then instantiate the mesh. If the post-processed data is labeled data, such as lines or surface points, it can generate point or tubular meshes based on the labeled data. Since multiple meshes are located in a reference coordinate system, the image workstation can perform unified coordinate space processing on multiple meshes to load each mesh into the same scene and place them in the correct positions. Next, the image workstation can perform Boolean fusion on multiple meshes and set the visualization attributes of each mesh to obtain a single integrated 3D mesh. This yields a 3D model. Afterward, the image workstation can export the 3D mesh in a format suitable for client rendering, thus obtaining the 3D model file.

[0092] The format can include glTF 2.0, and the resulting model file will be a glTF file (.glb). Visualization attributes can include applied color and material properties, etc. Furthermore, the image workstation is equipped with a format conversion engine that can acquire model files based on a unified dataset.

[0093] Understandably, if at least two types of post-processing data in a unified dataset need to be fused (such as post-processing data for different modalities of the same organization), the image workstation can fuse at least two types of post-processing data before performing 3D reconstruction on the unified dataset, and then perform 3D reconstruction based on the fused post-processing data.

[0094] As described in steps 205 and 206 above, the method provided in this application embodiment can dynamically retrieve, parse, and convert files using file identifiers in DICOM files written to a unified dataset, thereby establishing an automated generation and stable association link from PACS storage to Web 3D resources, ensuring on-demand, reliable, and permanent access to 3D content.

[0095] Step 207: The client obtains the model file based on the access address of the model file and displays the 3D model based on the model file.

[0096] Once the client receives the access address of the 3D model file, it can access the file server based on that address to obtain the 3D model file. The client can then load and render the model file to display the 3D model.

[0097] Taking a web client as an example, the 3D engine deployed on the web client (such as Three.js) can read and load the model file of the 3D model and complete the parsing and processing of the model data stored in the model file. The rendering engine of the web client (such as WebGL) can perform rendering operations based on the parsed model data, mapping the model data to the two-dimensional pixel space of the display screen, thereby realizing the visualization display of the 3D model on the web client.

[0098] When loading model files, coordinates (in millimeters) in the reference coordinate system are strictly used. Model data can include: vertex positions, texture information, and anatomical annotations of the 3D model. The vertex positions can be directly derived from coordinates in the unified dataset's centralized object coordinate system. This avoids scaling distortion caused by scaling operations, thus achieving realistic proportion rendering of the target object.

[0099] In this embodiment, when displaying a 3D model, the client can also render measurement tools to allow doctors to perform intuitive measurements within the displayed 3D model. These measurement tools may include at least one of the following: a length measurement tool, an angle measurement tool, an area measurement tool, and a volume measurement tool.

[0100] For example, the client can render a 3D ruler with scale markings. Responding to a user's click on the surface of the 3D model, the client can obtain the coordinates of the point of action of the click, and then calculate the distance between the points of action of the click using the 3D ruler. This distance is the distance in the actual target object.

[0101] With the rapid development of medical imaging technology, multimodal medical imaging such as CT and MRI are playing an increasingly central role in clinical diagnosis, surgical planning, medical education, and scientific research. Analyzing and processing two-dimensional medical images (i.e., tomographic sequences), especially the three-dimensional reconstruction and visualization of specific anatomical structures (such as tumors, blood vessels, and bones) or tissues, can provide doctors with a three-dimensional, quantifiable analytical perspective, greatly improving the accuracy of diagnosis and the predictability of treatment plans.

[0102] Processing two-dimensional medical images typically involves various steps such as segmentation, measurement, and annotation. However, integrating the results of these steps into clinical three-dimensional visualization and surgical planning workflows still presents a series of significant process and systems engineering challenges. 1. Fragmented Processes and Data Silos: Multiple processes such as segmentation, measurement, and annotation are usually completed in different software modules or systems, with varying output formats and scattered storage. This results in the fragmentation of multiple processing results for each object, meaning that the complete 3D planning data of the object is broken down into multiple unrelated files, creating data silos and making it impossible to form a unified, manageable, and accessible "patient 3D status snapshot".

[0103] 2. Disconnect between processing results and original medical images: The generated 3D model files (such as STL and OBJ files) lack a mandatory, system-level association mechanism with the original DICOM images. Once removed from a specific workstation environment, the clinical context of the 3D model (which sequence it corresponds to, whether the spatial coordinates are accurate) is easily lost, leading to difficulties in data traceability and the risk of misuse.

[0104] 3. Lack of clinical-grade persistence and consistency management: 3D planning data is usually stored on doctors' personal workstations or departmental file servers, and is not included in the hospital-level imaging data management system centered on PACS. This not only poses a risk of data loss, but also makes it impossible to effectively compare versions, collaborate, and conduct long-term follow-up tracking of planning results from multiple examinations of the same patient or between different doctors, thus compromising data consistency and traceability.

[0105] 4. Efficiency and Functional Bottlenecks of Web Visualization: In traditional workflows, converting workstation-generated results into a format suitable for web client display is a manual, offline, and tedious process. Furthermore, the converted web model often becomes "dead data" for viewing only, unable to support precise interaction at a true scale (such as measurement).

[0106] This application provides a method for processing medical images. By standardizing various post-processing data from medical image sequences to form a complete and structured "unified bookmark," it breaks down data silos and creates a unified, manageable, and callable data object. By storing the unified dataset on a PACS server and associating it with the medical image sequence, it achieves native, permanent, and traceable binding between post-processing data and the original medical image sequence within the medical information system. This solves problems such as data loss and difficulty in data traceability caused by the separation of post-processing data and medical image sequences, and largely avoids interruptions to clinical workflows. By implementing realistic scale rendering and interaction based on a reference coordinate system in a web client, it supports users to directly perform precise measurements of length, angle, and volume, enabling a leap from visualization to interactive planning. By using the "unified bookmark" as a core data hub, it integrates scattered processing steps such as post-processing (e.g., surgical planning), PACS archiving, and web reporting into a coherent workflow, achieving end-to-end image processing and model display, thus realizing a complete and coherent closed loop from data processing to clinical application. This ensures data integrity throughout the entire chain, from image workstation processing and PACS archiving to web report display, solving the problem of data silos, as well as the chaos and errors caused by multi-format conversion and multi-copy storage, and the traceability problem, thus improving the ease of data access.

[0107] It is understood that the order of steps in the medical image processing method provided in this application embodiment can be appropriately adjusted, and steps can be added or removed as needed. For example, step 204 can be deleted as appropriate. Any variations that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application, and therefore will not be elaborated further.

[0108] In summary, this application provides a method for processing medical images. This method can acquire various post-processed data obtained from post-processing medical image sequences, and obtain a unified dataset based on these multiple post-processed data, thereby controlling the visualization display of a 3D model of the unified dataset. The unified dataset includes multiple post-processed data with a unified format and converted to the same coordinate system. This unifies the format of heterogeneous post-processed data and ensures they correspond to the same physical coordinate space, solving the problem of data silos caused by different formats and enabling the fusion of multiple post-processed data. Therefore, on the one hand, it facilitates the overall management of multiple post-processed data; on the other hand, it enables integrated visualization of multiple post-processed data based on a single 3D model. Compared to displaying multiple models separately, this reduces data redundancy and rendering overhead, improves data processing and display efficiency, and ensures display consistency.

[0109] Figure 5 This is a schematic diagram of the image workstation provided in an embodiment of this application. See also... Figure 5 The image workstation 100 includes a processor 111, which is used for: Obtain the medical image sequence of the target object; Post-processing of medical image sequences yields various post-processed data. A unified dataset is obtained based on multiple post-processed data, which includes multiple post-processed data in a unified format and converted to the same coordinate system. Control the visualization of 3D models of a unified dataset.

[0110] Optionally, the processor 111 can be used for: Standardize the format of various post-processing data; Spatial normalization is performed on various post-processed data after format unification. By integrating various post-processed data after spatial normalization, a unified dataset is obtained.

[0111] Optionally, the number of medical image sequences can be multiple. The processor 111 can also be used for: Before performing spatial normalization on various post-processed data after format unification, the registration transformation relationship between the first image sequence and each second image sequence in multiple medical image sequences is obtained.

[0112] The process by which the processor 111 performs spatial normalization on various post-processed data after format unification can include: Based on the transformation relationship between the pixel coordinate system and the reference coordinate system of the first image sequence, the various post-processed data of the first image sequence after unified format processing are transformed into the reference coordinate system; Based on the registration transformation relationship of each second image sequence, and the transformation relationship between the pixel coordinate system of the first image sequence and the reference coordinate system, the various post-processed data of the second image sequence after unified format processing are transformed into the reference coordinate system.

[0113] Optionally, the processor 111 can also be used for: After standardizing various post-processing data of medical image sequences to obtain a unified dataset, the unified dataset is written to the PACS server.

[0114] Optionally, the processor 111 can be used for: Create a new DICOM file; Write the unified dataset to a DICOM file; Upload the DICOM file to the PACS server.

[0115] Optionally, the processor 111 can also be used for: Write the identifiers of the medical image sequences to which the unified dataset belongs into the DICOM file.

[0116] Optionally, the processor 111 can be used for: The receiving terminal sends a request to acquire a 3D model of a unified dataset. The acquisition request includes the file identifier of the DICOM file. In response to the retrieval request, the DICOM file is retrieved from the PACS server based on the file identifier, and the model file of the 3D model is retrieved based on the DICOM file; Upload the model file to the file server and obtain the access address of the model file; Send an access address to the terminal so that the terminal can retrieve the model file from the file server based on the access address and visualize the 3D model based on the model file.

[0117] In summary, this application provides an imaging workstation capable of acquiring various post-processed data obtained from post-processing medical image sequences, and obtaining a unified dataset based on these post-processed data, thereby controlling the visualization display of a 3D model of the unified dataset. This unified dataset includes various post-processed data in a unified format and converted to the same coordinate system. This unifies the format of heterogeneous post-processed data, ensuring they correspond to the same physical coordinate space, thus solving the problem of data silos caused by different formats and enabling the fusion of various post-processed data. Therefore, on the one hand, it facilitates the overall management of various post-processed data; on the other hand, it enables integrated visualization of various post-processed data based on a single 3D model. Compared to displaying multiple models separately, this reduces data redundancy and rendering overhead, improves data processing and display efficiency, and ensures display consistency.

[0118] like Figure 5 As shown, the image workstation 100 also includes a memory 113. The processor 111 and the memory 113 are connected, for example, via a bus 112. Optionally, the image workstation 100 may also include a transceiver 114. It should be noted that in practical applications, the transceiver 114 is not limited to one, and the structure of this image workstation 110 does not constitute a limitation on the embodiments of this application.

[0119] Processor 111 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 111 may also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0120] Bus 112 may include a pathway for transmitting information between the aforementioned components. Bus 112 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 112 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0121] The memory 113 stores a computer program corresponding to the medical image processing method provided in the above embodiments of this application. This computer program is controlled and executed by the processor 111. The processor 111 executes the computer program stored in the memory 113 to implement the content shown in the aforementioned method embodiments.

[0122] This application provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the medical image processing method provided in the above-described method embodiments. For example, Figure 2 or Figure 3 The method shown.

[0123] This application provides a computer program product, which includes a computer program that, when executed by a processor, implements the medical image processing method provided in the above-described method embodiments. For example, Figure 2 or Figure 3 The method shown.

[0124] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0125] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0126] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0127] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0128] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0129] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method of processing medical images, characterized by, The method includes: Obtain the medical image sequence of the target object; The medical image sequence is post-processed to obtain various post-processed data of the medical image sequence; Based on the various post-processed data, a unified dataset is obtained, wherein the unified dataset includes: the various post-processed data with a unified format and converted to the same coordinate system; Control the visualization of the three-dimensional model of the unified dataset.

2. The method of claim 1, wherein, Based on the aforementioned various post-processed data, a unified dataset is obtained, including: The various post-processed data are subjected to format unification processing; Spatial normalization is performed on the various post-processed data after the format unification process. The various post-processed data after spatial normalization are integrated to obtain a unified dataset.

3. The method according to claim 2, characterized in that, The number of medical image sequences is multiple; before performing spatial normalization on the various post-processed data after format unification processing, the method further includes: Obtain the registration transformation relationship between the first image sequence and each of the second image sequences in the plurality of medical image sequences; The spatial normalization process performed on the various post-processed data after format unification includes: Based on the transformation relationship between the pixel coordinate system and the reference coordinate system of the first image sequence, the various post-processed data of the first image sequence after unified format processing are transformed into the reference coordinate system; Based on the registration transformation relationship of each of the second image sequences, and the transformation relationship between the pixel coordinate system of the first image sequence and the reference coordinate system, the various post-processed data of the second image sequences after unified format processing are transformed into the reference coordinate system.

4. The method according to any one of claims 1 to 3, characterized in that, After standardizing various post-processing data of the medical image sequence to obtain a unified dataset, the method further includes: Write the unified dataset into the PACS server.

5. The method according to claim 4, characterized in that, Writing the unified dataset to the PACS server includes: Create a new DICOM file; Write the unified dataset into the DICOM file; Upload the DICOM file to the PACS server.

6. The method according to claim 5, characterized in that, The method further includes: Write the identifier of the medical image sequence to which the unified dataset belongs into the DICOM file.

7. The method according to claim 5, characterized in that, Controlling the visualization of the 3D model of the unified dataset includes: The receiving terminal sends a request to acquire a 3D model of the unified dataset, the acquisition request including the file identifier of the DICOM file; In response to the acquisition request, the DICOM file is obtained from the PACS server based on the file identifier, and the model file of the 3D model is obtained based on the DICOM file; Upload the model file to the file server and obtain the access address of the model file; The access address is sent to the terminal so that the terminal can obtain the model file from the file server based on the access address and perform visualization display of the 3D model based on the model file.

8. An image workstation, characterized in that, The imaging workstation includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the method as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-8.

10. A medical image processing system, characterized in that, The processing system includes: a PACS server, a terminal, and an image workstation as described in claim 8.

Images