Medical data processing method and device and computer readable storage medium

Abstract

The embodiment of the application provides a medical data processing method and device and a computer readable storage medium. The specific implementation scheme is: acquiring medical images collected at multiple time points for a target patient, and segmenting a lesion image from the medical images; determining a pairing relationship of the lesion image, the pairing relationship representing lesion images of a same lesion at each time point; and constructing lesion timeline data based on the pairing relationship and the lesion image. Based on the scheme, the lesion timeline data can be automatically generated, the effective management of the lesion information is realized in the form of the lesion timeline data, and data support is provided for each link in the surgical business process. The application provides an end-to-end complete system for the business process of surgery, which can cover all links of the business process, standardize the collection and management of data of each link, make the data efficient, and avoid data loss.

Description

Technical Field

[0001] This application relates to the field of medical data processing technology, and more specifically, to a medical data processing method, apparatus, and computer-readable storage medium. Background Technology

[0002] Medical images are images acquired during medical procedures that provide intuitive information about a patient's physical condition and are an indispensable part of medical diagnosis and treatment. Medical images can reflect information about lesions, and how to effectively manage this information has become an important technical challenge.

[0003] Currently, most software serving surgical workflows provides assistance for a specific step. The acquisition, use, and collection of data in each step are often isolated. However, surgical workflows involve multiple steps that are actually interconnected, making it impossible to achieve efficient data flow and utilization, and easily leading to data loss. Summary of the Invention

[0004] This application provides a medical data processing method, apparatus, and computer-readable storage medium, aiming to at least address one of the aforementioned technical deficiencies. The technical solution adopted in this application is as follows:

[0005] In a first aspect, embodiments of this application provide a medical data processing method, the method comprising:

[0006] Acquire medical images of the target patient at multiple time points and segment lesion images from the medical images;

[0007] Determine the pairing relationship of lesion images; the pairing relationship represents the lesion images of the same lesion at various time points.

[0008] Based on the pairing relationships and lesion images, timeline data of lesions are constructed.

[0009] As an optional method, determining the pairing relationship of lesion images includes:

[0010] Determine the deformation field between medical images at different time points;

[0011] Based on the deformation field, medical images at different time points are registered and deformed.

[0012] Based on the results of registration deformation processing, the pairing relationship between the images of each lesion is determined.

[0013] As an alternative approach, the above methods also include:

[0014] Acquire new medical images;

[0015] Determine whether the patient corresponding to the newly added medical image belongs to the target patient;

[0016] In response to the fact that the patient corresponding to the newly added medical image belongs to the target patient, the existing lesion timeline data is updated based on the newly added medical image. The existing lesion timeline data is the lesion timeline data of the patient corresponding to the newly added medical image.

[0017] In response to the fact that the patient corresponding to the newly added medical image does not belong to the target patient, appropriate processing is performed based on the fact that the newly added medical image contains lesions.

[0018] As an optional approach, existing lesion timeline data can be updated based on newly added medical images, including:

[0019] Segmenting newly added lesion images from newly added medical images;

[0020] Determine whether there is new lesion timeline data in the existing lesion timeline data. New lesions are those represented by the new lesion images.

[0021] In response to the existence of lesion timeline data corresponding to newly added lesions in the existing lesion timeline data, the newly added lesion image is inserted into the lesion timeline data corresponding to the newly added lesion based on the time point of the newly added medical image;

[0022] In response to the absence of lesion timeline data corresponding to the newly added lesion in the existing lesion timeline data, lesion timeline data is created based on the newly added medical images.

[0023] As an optional approach, appropriate processing can be performed based on the presence of lesions in the newly added medical images, including:

[0024] In response to the condition that a newly added medical image contains lesions, a timeline data of the lesions is created based on the newly added medical image.

[0025] As an alternative approach, after constructing the lesion timeline data, the above method also includes:

[0026] Display timeline data of lesions based on a predetermined display style;

[0027] Responding to user actions that select at least two target time points based on the displayed lesion timeline data;

[0028] Compare and display the medical images corresponding to the target time point.

[0029] As an optional approach, after comparing and displaying medical images corresponding to the target time point, the above method also includes:

[0030] In response to the user's selection of a target region in a medical image at any target time point displayed, a 3D model of the target region is constructed based on the medical image at each target time point, and the 3D model corresponding to each target time point is displayed.

[0031] As an optional approach, the target area includes:

[0032] The area at a first specified distance from the center of the lesion selected by the user;

[0033] The area at a second specified distance from the target location selected by the user;

[0034] As an optional approach, after displaying the 3D models corresponding to each target time point, the above method also includes:

[0035] In response to the user's control operation on the display of the 3D model corresponding to any target time point, the system performs corresponding display control on the 3D models corresponding to all target time points.

[0036] As an alternative approach, after constructing the lesion timeline data, the above method also includes:

[0037] The surgical-related documents for the target lesion are stored in association with the surgical time points in the lesion timeline data. The surgical-related documents include at least one of the following:

[0038] Preoperative planning documents;

[0039] Intraoperative laparoscopic video;

[0040] Surgical report.

[0041] Secondly, embodiments of this application provide a medical data processing apparatus, the apparatus comprising:

[0042] The lesion segmentation module is used to acquire medical images of the target patient at multiple time points and segment lesion images from the medical images.

[0043] The pairing relationship determination module is used to determine the pairing relationship of lesion images. The pairing relationship represents the lesion images of the same lesion at various time points.

[0044] The lesion timeline data construction module is used to construct lesion timeline data based on pairing relationships and lesion images.

[0045] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described medical data processing method.

[0046] Fourthly, embodiments of this application provide an electronic device, which includes:

[0047] One or more processors; and

[0048] A memory associated with one or more of the processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the medical data processing method.

[0049] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described medical data processing method.

[0050] The beneficial effects of the technical solutions provided in this application are:

[0051] The solution provided in this application acquires medical images of a target patient at multiple time points, segments lesion images from these images, determines the pairing relationships between the lesion images, and constructs lesion timeline data based on these pairing relationships and the lesion images. Based on this solution, lesion timeline data can be automatically generated, enabling effective management of lesion information and providing data support for each stage of the surgical workflow.

[0052] This application provides a complete end-to-end system for the surgical business process, which can cover all aspects of the business process and standardize the collection and management of data at each stage, so that data can flow and be used efficiently and data loss can be avoided. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0054] Figure 1 This is a system architecture diagram applicable to the embodiments of this application;

[0055] Figure 2 A flowchart illustrating the medical data processing method provided in this application embodiment;

[0056] Figure 3 This is a schematic diagram of lesion timeline data provided in an embodiment of this application;

[0057] Figure 4 A schematic diagram of the system structure is provided for the embodiments of this application;

[0058] Figure 5 It is the interface used for manual analysis of lesions in image screening and diagnostic systems;

[0059] Figure 6 A flowchart illustrating the process of determining pairing relationships for the implementation of this application;

[0060] Figure 7 A schematic diagram illustrating the process of adding a new medical image to a database, provided as an embodiment of this application;

[0061] Figure 8 A schematic diagram of the interface of the lesion tracking image follow-up and management system;

[0062] Figure 9 This is a schematic diagram of an interface that compares and displays medical images at different points in time.

[0063] Figure 10 A schematic diagram of the interface for 3D reconstruction in the 3D reconstruction and preoperative system;

[0064] Figure 11 A schematic diagram of the interface for surgical planning in a 3D reconstruction and surgical planning system;

[0065] Figure 12 This is an illustration of how icons for surgery-related documents are displayed on the interface.

[0066] Figure 13 This is a flowchart illustrating the data interaction process within each subsystem of the system provided in this solution.

[0067] Figure 14 This is a schematic diagram of the structure of the medical data processing device provided in the embodiments of this application;

[0068] Figure 15 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0069] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0070] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0071] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0072] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0073] To facilitate understanding of this application, the system architecture on which this application is based will first be described. For example... Figure 1 The diagram shows an exemplary system architecture that can be applied to embodiments of this application, such as... Figure 1 As shown, the system architecture may include: a terminal, a network, and a server.

[0074] A network is a medium used to provide a communication link between a server and a terminal. Networks can include various connection types, such as wired and wireless communication links or fiber optic cables, etc.

[0075] The terminal may include, but is not limited to, mobile phones, tablets, and smart wearable devices. Smart wearable devices may include smartwatches, smart glasses, smart bracelets, VR (Virtual Reality) devices, AR (Augmented Reality) devices, mixed reality devices (i.e., devices that support both virtual and augmented reality), and so on. In this embodiment, the user equipment typically has information display functionality, such as a display screen. In addition, it may also have operation command input functionality, such as inputting operation commands via touchscreen, keyboard, or voice.

[0076] The server can be a dedicated server, a server cluster, or a cloud server. A cloud server, also known as a cloud computing server or cloud host, is a hosting product within the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Servers (VPS) services, such as high management difficulty and weak service scalability.

[0077] In one embodiment, the medical data processing provided in this application can be performed by a terminal or by a server.

[0078] It should be understood that, Figure 1 The number of servers and terminals shown is merely illustrative. Depending on implementation needs, there can be any number of servers and terminals.

[0079] Figure 2 This illustration shows a flowchart of a medical data processing method provided in an embodiment of this application. The method can be performed by… Figure 1 This is executed on the terminal or server in the system shown. For example... Figure 2 As shown, this method mainly includes:

[0080] Step S210: Acquire medical images of the target patient at multiple time points and segment lesion images from the medical images;

[0081] Step S220: Determine the pairing relationship of lesion images. The pairing relationship represents the lesion images of the same lesion at various time points.

[0082] Step S230: Construct timeline data of lesions based on the pairing relationship and lesion images.

[0083] By segmenting lesions in medical images of the same target patient collected at different time points, lesion images at different time points can be obtained, which can reflect the development of the lesions.

[0084] In this example, segmenting the lesion image from the medical image can be achieved based on a preset target detection and instance segmentation algorithm.

[0085] By determining the pairing relationship of lesion images at different time points, it is possible to obtain an image sequence consisting of lesion images of the same lesion at different time points.

[0086] It is understandable that medical images of the same target patient acquired at a single point in time may contain multiple lesions, and multiple lesion images corresponding to these lesions can be segmented from the medical images.

[0087] After determining the pairing relationship, an image sequence consisting of lesion images at different time points can be provided based on the pairing relationship and lesion images. In this embodiment, other lesion-related data, such as lesion attributes, can also be included as lesion timeline data, so that the lesion timeline data can reflect the new occurrence, existence, disappearance, and other attributes of lesions over time. Based on the lesion timeline data, data support can be provided for each stage of the surgical workflow.

[0088] As an example, Figure 3 This is a schematic diagram of timeline data of a lesion provided in an embodiment of this application.

[0089] like Figure 3 As shown, lung nodules 1, 2, and 3 are all lesions. The lesion timeline data can include images of lung nodules 1, 2, and 3 at different time points, as well as descriptive information for each time point. This descriptive information describes the attributes of the lung nodules. In this example, the descriptive information can be automatically generated based on the lesion attributes.

[0090] Reference Figure 3 As shown in the example, lung nodule 1 appeared at two time points and disappeared after the third time point. This is because a laparoscopic surgery was performed after the second time point to remove lung nodule 1. Lung nodules 2 and 3, on the other hand, were detected and successfully matched in medical images at all five time points, with their lesion timeline data spanning all five time points.

[0091] The method provided in this application acquires medical images of a target patient at multiple time points, segments lesion images from these images, determines the pairing relationships between the lesion images, and constructs lesion timeline data based on these pairing relationships and the lesion images. Based on this solution, lesion timeline data can be automatically generated, enabling effective management of lesion information and providing data support for each stage of the surgical workflow.

[0092] This application provides a software system for assisting surgical workflows. The system structure is as follows: Figure 4 As shown, the system includes the following subsystems: image screening and diagnosis system, three-dimensional reconstruction and surgical planning system, intraoperative laparoscopic navigation system, and lesion tracking image follow-up and management system.

[0093] The image screening and diagnostic system is used to automatically detect and segment lesions in newly added medical images, as well as analyze the nature of the lesions. Whether a patient qualifies as a target patient can be determined manually or automatically by the system, and then included in the lesion tracking image follow-up and management system for management.

[0094] As an example, Figure 5 This is an interface used in image screening and diagnostic systems for manual analysis of lesions. The interface can display multiple medical images of a patient, along with a 3D model constructed from those images, to fully visualize the lesions and aid in their manual analysis.

[0095] The 3D reconstruction and surgical planning system is used to build 3D models based on medical images, perform surgical planning based on the 3D models, and generate preoperative planning documents.

[0096] Intraoperative laparoscopic navigation systems are used for intraoperative navigation based on preoperative planning documents.

[0097] The lesion tracking image follow-up and management system is used to generate, store, and manage lesion timeline data.

[0098] The aforementioned subsystems can realize functions such as image screening and diagnosis, surgical planning, intraoperative navigation, and continuous postoperative follow-up. They provide a complete end-to-end process centered on the lesion as the core objective. Lesion management based on lesion timeline data is the foundation of the entire process, making the lesion tracking image follow-up and management system, which manages lesion timeline data, the core system in this case.

[0099] This application provides a complete end-to-end system for the surgical business process, which can cover all aspects of the business process and standardize the collection and management of data at each stage, so that data can flow and be used efficiently and data loss can be avoided.

[0100] In one optional embodiment of this application, determining the pairing relationship of lesion images includes:

[0101] Determine the deformation field between medical images at different time points;

[0102] Based on the deformation field, medical images at different time points are registered and deformed.

[0103] Based on the results of registration deformation processing, the pairing relationship between the images of each lesion is determined.

[0104] In this embodiment of the application, when determining the pairing relationship, the medical images corresponding to different time points can be registered first. The positions of the lesion images in each medical image after registration are corresponding, thereby determining the pairing relationship between the lesion images.

[0105] When registering medical images, the deformation field between different medical images can be determined based on a spatial registration algorithm, and then registration deformation can be performed based on the deformation field to achieve the registration of medical images.

[0106] As an example, Figure 6 A flowchart illustrating the process of determining pairing relationships for the implementation of this application.

[0107] like Figure 6 As shown, the AI ​​Engine, an AI-based registration model, is used to register medical images from different time points. Sequences 1, 2, and 3 represent lesion image sequences segmented from medical images acquired at different time points. After medical image registration, the pairing relationships of lesion images in different lesion image sequences can be obtained based on the registration results.

[0108] Sequence 1 (i.e., lesion image sequence 1) includes lesion images 1-1, 1-2, 1-3, 1-4, 1-5, 1-6, 1-7, and 1-8. Sequence 2 (i.e., lesion image sequence 2) includes lesion images 2-1, 2-2, 2-3, 2-4, 2-5, 2-6, and 2-7. Sequence 3 (i.e., lesion image sequence 3) includes lesion images 3-1, 3-2, 3-3, 3-4, 3-5, 3-6, 3-7, 3-8, and 3-9.

[0109] In this example, medical images at each pair of time points can be registered sequentially according to time order. Specifically, sequence 1 can be registered with sequence 2, and sequence 2 can be registered with sequence 3.

[0110] The pairing relationship between lesion images in sequence 1 and sequence 2 is as follows: lesion image 1-1 corresponds to lesion image 2-1, lesion image 1-2 corresponds to lesion image 2-3, lesion image 1-3 corresponds to lesion image 2-4, lesion image 1-4 corresponds to lesion image 2-2, lesion image 1-5 corresponds to lesion image 2-6, and lesion image 1-7 corresponds to lesion image 2-7.

[0111] The pairing relationships between lesion images in sequence 2 and sequence 3 are as follows: lesion image 2-1 corresponds to lesion image 3-2, lesion image 2-2 corresponds to lesion image 3-3, lesion image 2-3 corresponds to lesion image 3-4, lesion image 2-4 corresponds to lesion image 3-5, lesion image 2-5 corresponds to lesion image 3-7, lesion image 2-6 corresponds to lesion image 3-6, and lesion image 2-7 corresponds to lesion image 3-9.

[0112] In this example, lesion timeline data can be constructed first based on the pairing relationship between sequence 1 and sequence 2. Then, the lesion timeline data can be supplemented based on the pairing relationship between sequence 2 and sequence 3 to obtain the final lesion timeline data. That is, lesion line number 1 (i.e., the lesion timeline data corresponding to lesion number 1) includes lesion image 1-1, lesion image 2-1, and lesion image 3-2; lesion line number 2 (i.e., the lesion timeline data corresponding to lesion number 2) includes lesion image 1-2, lesion image 2-3, and lesion image 3-4; lesion line number 3 (i.e., the lesion timeline data corresponding to lesion number 3) includes lesion image 1-3, lesion image 2-4, and lesion image 3-5; lesion line number 4 (i.e., the lesion timeline data corresponding to lesion number 4) includes lesion image 1-4, lesion image 2-2, and lesion image 3-3; lesion line number 5... The timeline data for lesion number 5 includes images 1-5, 2-6, and 3-6. The timeline data for lesion number 6 includes images 1-6. The timeline data for lesion number 7 includes images 1-7, 2-7, and 3-9. The timeline data for lesion number 8 includes images 1-8. The timeline data for lesion number 9 includes images 2-5 and 3-7. The timeline data for lesion number 10 includes image 3-1. The timeline data for lesion number 11 includes images 3-8.

[0113] In one optional embodiment of this application, the above method further includes:

[0114] Acquire new medical images;

[0115] Determine whether the patient corresponding to the newly added medical image belongs to the target patient;

[0116] In response to the fact that the patient corresponding to the newly added medical image belongs to the target patient, the existing lesion timeline data is updated based on the newly added medical image. The existing lesion timeline data is the lesion timeline data of the patient corresponding to the newly added medical image.

[0117] In response to the fact that the patient corresponding to the newly added medical image does not belong to the target patient, appropriate processing is performed based on the fact that the newly added medical image contains lesions.

[0118] In this embodiment of the application, after generating the timeline data of the lesions of the target patient, the timeline data of the lesions of the target patient can be stored and managed.

[0119] Upon receiving a new medical image, it can be first determined whether the patient corresponding to the new medical image belongs to the target patient. If the patient corresponding to the new medical image belongs to the target patient, it means that the lesion timeline data for that patient already exists. At this time, the lesion timeline data for that patient can be updated based on the new medical image, that is, the existing lesion timeline data can be updated.

[0120] If the patient corresponding to the newly added medical image does not belong to the target patient, it means that there is no lesion timeline data for that patient. In this case, we can obtain information on whether the newly added medical image contains lesions. The information on whether the newly added medical image contains lesions can reflect the specific condition of the patient and provide a basis for subsequent processing.

[0121] In one optional embodiment of this application, updating existing lesion timeline data based on newly added medical images includes:

[0122] Segmenting newly added lesion images from newly added medical images;

[0123] Determine whether there is new lesion timeline data in the existing lesion timeline data. New lesions are those represented by the new lesion images.

[0124] In response to the existence of lesion timeline data corresponding to newly added lesions in the existing lesion timeline data, the newly added lesion image is inserted into the lesion timeline data corresponding to the newly added lesion based on the time point of the newly added medical image;

[0125] In response to the absence of lesion timeline data corresponding to the newly added lesion in the existing lesion timeline data, lesion timeline data is created based on the newly added medical images.

[0126] In this embodiment of the application, when updating existing lesion timeline data based on newly added medical image data, the newly added lesion image can be segmented from the newly added medical image first, and then it can be determined whether the newly added lesion represented by the newly added image already has corresponding lesion timeline data.

[0127] Specifically, newly added medical images can be registered with the patient's previous medical images, and the registration results can be used to determine whether the new lesions already have corresponding timeline data.

[0128] If timeline data for the newly added lesion already exists, the new medical image can be inserted into the timeline data for the corresponding lesion based on the acquisition time of the new medical image.

[0129] If there is no timeline data for the newly added lesion, timeline data for the newly added lesion can be created based on the newly added medical images.

[0130] In one optional embodiment of this application, processing is performed based on the presence of lesions in the newly added medical image, including:

[0131] In response to the condition that a newly added medical image contains lesions, a timeline data of the lesions is created based on the newly added medical image.

[0132] In this embodiment, when the patient corresponding to the newly added medical image is not a target patient, it can be determined whether the presence of lesions in the newly added medical image meets preset conditions. If the conditions are met, the patient corresponding to the newly added medical image can be used as the target patient, and lesion timeline data can be created based on the newly added medical image; if the conditions are not met, the patient corresponding to the newly added medical image will not be used as the target patient.

[0133] The preset conditions can be set according to actual needs, including but not limited to quantity conditions set for the number of lesions and location conditions set for the location of lesions.

[0134] As an example, taking medical images of the lungs as an example, the above preset conditions can be set with reference to the guidelines for the diagnosis of lung nodules. When a newly added medical image contains lesions that meet the preset conditions, it indicates that the patient may have malignant lung nodules and can be managed as a target patient for subsequent diagnosis and treatment.

[0135] As an example, Figure 7 This is a schematic diagram illustrating a process for adding new medical images to a database, as provided in an embodiment of this application.

[0136] like Figure 7 As shown, DICOM image data refers to newly added medical images. Information system integration and automatic information retrieval refer to the integration with the medical imaging system, whereby when the medical imaging system acquires a new medical image, it is automatically retrieved into the image screening and diagnostic system.

[0137] Screening systems, lesion detection, segmentation, and benign / malignant analysis refer to the detection and segmentation of lesions in imaging screening and diagnostic systems, as well as the analysis of the nature of the lesions.

[0138] When it is determined that the newly added images come from a new patient, they can be enrolled manually or automatically. Once enrolled, the patient is designated as a key patient, and their medical data is managed by the lesion tracking image follow-up and management system. In the automatic enrolling method, enrollment criteria can be set according to clinical guidelines and other regulations, and the patient will be enrolled when the criteria are met.

[0139] After enrolling new patients, historical patient information can be retrieved from the hospital information system, and AI-assisted analysis can be used for lesion registration. Then, a timeline registration algorithm is used to generate patient timelines, thus determining the pairing relationships between lesion images to construct lesion timeline data. This lesion timeline data can be stored in a large permanent database. This large permanent database can also store information on key patients, specifically including textual information, DICOM information, AI-identified lung nodules, blood supply analysis, and registration information.

[0140] When it is determined that the newly added image does not originate from a new patient, a timeline registration algorithm can be used to incorporate the existing patient's timeline. This involves updating the existing lesion timeline data based on the newly added medical image. Specifically, it can be determined whether the lesion in the newly added medical image is a new lesion. If it is a new lesion, new lesion timeline data can be generated; if it is not a new lesion, existing lesion timeline data can be incorporated, and finally, the lesion timeline data is stored.

[0141] In one optional embodiment of this application, after constructing the lesion timeline data, the above method further includes:

[0142] Display timeline data of lesions based on a predetermined display style;

[0143] Responding to user actions that select at least two target time points based on the displayed lesion timeline data;

[0144] Compare and display the medical images corresponding to the target time point.

[0145] In this embodiment, lesion timeline data can be displayed intuitively based on a predetermined display style. As an example, the predetermined display style can be as follows: Figure 3 As shown in the image.

[0146] In practical use, there may be a need to compare medical images from multiple different time points. Users can select at least two target time points based on the displayed lesion timeline data, and then trigger the comparison display of the medical images corresponding to the target time points.

[0147] As an example, Figure 8 This is a schematic diagram of the interface of a lesion tracking image follow-up and management system. (Example) Figure 8 As shown in the image, the left side of the interface displays the patient description, which includes the patient's basic information and the reason for enrollment. Users can select a patient by clicking on it. After selecting a patient, the interface displays the patient's lesion timeline data and lists colored medical image icons as time points, reflecting the time when the medical images were acquired.

[0148] In addition to constructing timeline data for lung nodules, this example can also construct timeline data for liver tumors, kidney tumors, bone tumors, etc.

[0149] In this example, the interface of the lesion tracking image follow-up and management system can serve as a starting point for other system interfaces. Users can jump to other system interfaces through operations within this interface.

[0150] Users can select any two time points from the lesion timeline data displayed in the interface of this example, and then trigger a comparison display of the medical images corresponding to these two time points.

[0151] As an example, Figure 9 This is a schematic diagram of an interface that compares and displays medical images at different points in time.

[0152] like Figure 9 As shown in the example, the interface in this example can provide different windows to compare and display medical images corresponding to two different time points.

[0153] In this example, functions such as scrolling, zooming, and measuring medical images can be provided, as well as synchronous scrolling of corresponding frames. That is, after selecting a medical image at a certain time point for display, medical images at the corresponding positions can be selected and displayed at other time points.

[0154] When comparing and displaying medical images from two different time points, information about the lesions, such as their nature, can also be shown.

[0155] In one optional embodiment of this application, after comparing and displaying the medical images corresponding to the target time point, the above method further includes:

[0156] In response to the user's selection of a target region in a medical image at any target time point displayed, a 3D model of the target region is constructed based on the medical image at each target time point, and the 3D model corresponding to each target time point is displayed.

[0157] In this embodiment of the application, when comparing and displaying medical images at multiple target time points, the user can select a target area in any medical image at a target time point, and then construct and display a three-dimensional model of the target area.

[0158] Medical images are mostly two-dimensional. Compared with two-dimensional images, using three-dimensional models to display them is more intuitive and vivid, and also makes it easier for users to observe lesion information efficiently.

[0159] In one optional embodiment of this application, the target area includes:

[0160] The area at a first specified distance from the center of the lesion selected by the user;

[0161] The area at a second specified distance from the target location selected by the user;

[0162] In this embodiment, the user can select a lesion, and then the system can use an area at a first specified distance from the center of the lesion as the target area. After generating a three-dimensional model of the target area, the lesion can be fully displayed, allowing the user to understand the morphology and surrounding structure of the lesion, thereby assessing the nature of the lesion's changes and making surgical decisions.

[0163] In this embodiment, the user can also randomly select a target location, and the system can then use an area at a second specified distance from the target location as the target area. By supporting users to randomly select areas of interest for comparative observation, it helps to enhance the understanding of the patient's overall condition, thereby enabling better surgical planning.

[0164] Taking the comparative observation of lesions in the lung nodules as an example, users can randomly select target locations to compare and observe any other lesions (such as emphysema, fractures, and pulmonary embolism) and structures of interest.

[0165] Reference Figure 9 As shown in the example, after the user selects the target lesion or the current location, the system can automatically generate a 3D model of the target area and display it in the window below the medical image.

[0166] In one optional embodiment of this application, after displaying the 3D model corresponding to each target time point, the above method further includes:

[0167] In response to the user's control operation on the display of the 3D model corresponding to any target time point, the system performs corresponding display control on the 3D models corresponding to all target time points.

[0168] In this embodiment of the application, when displaying a 3D model, the 3D model at one point in time can be rotated and scaled, and the 3D models at other points in time will be rotated and scaled synchronously, making it convenient for users to compare and view.

[0169] As an example, after analyzing lesions based on comparative visualization of medical images, a surgical decision can be made. Surgical planning can be initiated by clicking to launch the 3D reconstruction and surgical planning system at the most recent time point on the timeline of a specific lesion requiring surgery.

[0170] The 3D reconstruction and surgical planning system automatically loads the medical images corresponding to the selected lesion time points from the management system's storage, extracts all anatomical structures based on AI, and performs 3D reconstruction based on these structures to obtain a 3D model.

[0171] Figure 10 This is a schematic diagram of the interface for 3D reconstruction in a 3D reconstruction and surgical planning system.

[0172] like Figure 10 As shown, this interface displays a 3D model in a window and provides a list of lesions. Users can select lesions and selectively display them. Each anatomical structure in the 3D model is independent and can be individually displayed (either shown or not).

[0173] After generating the 3D model, surgical decisions and planning recommendations can be made based on information such as the nature and location of the lesion, including automatic planning functions such as lesion selection, surgical paradigm, and path.

[0174] Figure 11 This is a schematic diagram of the interface for surgical planning in a 3D reconstruction and surgical planning system.

[0175] like Figure 11 As shown, this interface provides a 3D visual representation of each surgical step, allowing users to select and edit them to create a preoperative planning document. The preoperative planning document contains key information such as the final determined surgical approach and pathway.

[0176] As an example, after generating the preoperative planning file, intraoperative navigation can be provided based on the preoperative planning file. During laparoscopic navigation, the intraoperative laparoscopic navigation system can provide real-time identification of various anatomical structures and surgical instruments in the surgical video stream and register them with the preoperative 3D reconstruction, thereby achieving synchronous display of laparoscopic images and 3D reconstruction, providing location information and real-time path navigation, and avoiding the risk of misoperation.

[0177] In this example, the intraoperative laparoscopic navigation system is a standalone workstation that can be pushed into the operating room and is connected to the laparoscopic video for analysis. It provides a high-definition display screen, spatially positioned alongside the laparoscopic display screen, offering synchronized display of laparoscopic video and AI-generated navigation information.

[0178] Taking thoracic surgery as an example, a 3D model can be obtained by reconstructing the medical images of the tissue to be operated on, and intraoperative laparoscopic video images can be acquired. The 3D model is then registered and fused with the intraoperative laparoscopic video images, and anatomical structures are identified in real time within the intraoperative laparoscopic video, such as identifying blood vessel names or corresponding relationships in the 3D reconstruction results. This allows surgeons to obtain real-time name indications or 3D spatial positioning of the anatomical tissues currently in the laparoscopy, thereby effectively achieving surgical navigation. Furthermore, this system can also indicate the surgical path and issue prompts when deviations from the actual surgical path are detected.

[0179] In one optional embodiment of this application, the above method further includes:

[0180] After generating the preoperative planning file, in response to the command to transfer the preoperative planning file, the preoperative planning file is transferred to the intraoperative navigation system using a preset transfer method, which includes any of the following:

[0181] The preoperative planning documents are transferred to a removable storage device so that the removable storage device can be connected to the intraoperative navigation system, thereby transferring the preoperative planning documents to the intraoperative navigation system;

[0182] Based on a wired or wireless connection with the intraoperative navigation system, the preoperative planning documents are transferred to the intraoperative navigation system.

[0183] In this embodiment, the intraoperative navigation system, as described above, relies on the preoperative planning file for its intraoperative navigation function. Therefore, it is necessary to transfer the preoperative planning file from the image comparison, follow-up, and management system to the intraoperative laparoscopic navigation system using a preset transfer method before surgery. The preset transfer method may specifically include transfer based on a removable storage device, transfer based on a wired network connection, or transfer based on a wireless network connection.

[0184] As an example, the intraoperative laparoscopic navigation system is a standalone system that can be moved into the operating room. Preoperative planning documents need to be retrieved from the image comparison and monitoring management system. The image comparison and monitoring management system can connect to a removable storage device, such as a USB flash disk, to transfer the preoperative planning documents. The removable storage device can then be connected to the intraoperative laparoscopic navigation system, allowing the preoperative planning documents to be transferred there. A wired or wireless connection can also be established between the intraoperative laparoscopic navigation system and the image comparison and monitoring management system, allowing the preoperative planning documents in the image comparison and monitoring management system to be sent to the intraoperative laparoscopic navigation system via this connection.

[0185] In one optional embodiment of this application, after constructing the lesion timeline data, the above method further includes:

[0186] The surgical-related documents for the target lesion are stored in association with the surgical time points in the lesion timeline data. The surgical-related documents include at least one of the following:

[0187] Preoperative planning documents;

[0188] Intraoperative laparoscopic video;

[0189] Surgical report.

[0190] In this embodiment, the surgical time point is the time when the surgery is performed on the target lesion, such as the date of the surgery. Preoperative planning documents, intraoperative laparoscopic videos collected during the surgery, and postoperative surgical reports can all be stored in the lesion timeline data at this surgical time point, facilitating understanding of the patient's preoperative and postoperative condition.

[0191] As an example, when visualizing the timeline data of lesions in the interface, surgical-related documents can also be displayed together. Figure 12 This is a diagram illustrating how icons for surgery-related documents are displayed on the interface.

[0192] like Figure 12 As shown, the color medical images in the interface are thumbnails of medical images at corresponding time points, which correspond to the lesion timeline data. The icons for surgical time points can display icons for surgical planning documents, intraoperative laparoscopic videos, and postoperative reports. The icons for surgical planning documents can specifically include both digital and text versions.

[0193] In this example, data such as surgical planning documents, intraoperative laparoscopic videos, and surgical reports can all be stored in the image comparison, follow-up, and management system.

[0194] As an example, Figure 13 This is a flowchart illustrating the data interaction process within each subsystem of the system provided in this solution.

[0195] like Figure 13 As shown, the Picture Archiving and Communication System (PACS) / Computed Tomography (CT) can provide Digital Imaging and Communications in Medicine (DICOM) for image screening and diagnostic systems, namely the newly added medical images mentioned above.

[0196] The image screening and diagnosis system screens newly added medical images for inclusion in the system, that is, it determines whether the patient corresponding to the medical image is a key patient. If so, the newly added medical image is transferred to the image comparison and follow-up management system.

[0197] The image comparison and follow-up management system can provide newly added medical images to the 3D reconstruction and surgical planning system, which will then generate preoperative planning files and transfer them to the image comparison and follow-up management system for storage.

[0198] The image comparison and follow-up management system can provide preoperative planning files to the intraoperative laparoscopic navigation system, which then uses these files for surgical navigation. After the surgery, the intraoperative laparoscopic navigation system can send the surgical video to the image comparison and follow-up management system for storage.

[0199] Based on and Figure 2 The method shown follows the same principle. Figure 14 This application provides a schematic diagram of the structure of a medical data processing device according to an embodiment of the present application. Figure 4 As shown, the medical data processing device 1400 may include:

[0200] The lesion segmentation module 1410 is used to acquire medical images of the target patient at multiple time points and segment lesion images from the medical images;

[0201] The pairing relationship determination module 1420 is used to determine the pairing relationship of the lesion images, wherein the pairing relationship represents the lesion images of the same lesion at various time points;

[0202] The lesion timeline data construction module 1430 is used to construct lesion timeline data based on the pairing relationship and lesion images.

[0203] As an optional approach, the pairing relationship determination module is specifically used for:

[0204] Determine the deformation field between medical images at different time points;

[0205] Based on the deformation field, registration and deformation processing are performed on medical images corresponding to different time points;

[0206] Based on the results of the registration deformation processing, the pairing relationship between the lesion images is determined.

[0207] As an alternative, the above-mentioned device also includes a new medical image processing module (not shown in the figure), used for:

[0208] Acquire new medical images;

[0209] Determine whether the patient corresponding to the newly added medical image belongs to the target patient;

[0210] In response to the fact that the patient corresponding to the newly added medical image belongs to the target patient, the existing lesion timeline data is updated based on the newly added medical image, wherein the existing lesion timeline data is the lesion timeline data of the patient corresponding to the newly added medical image;

[0211] If the patient corresponding to the newly added medical image does not belong to the target patient, appropriate processing is performed based on the presence of lesions in the newly added medical image.

[0212] As an optional approach, the newly added medical image processing module, when updating existing lesion timeline data based on the newly added medical images, is specifically used for:

[0213] The newly added lesion image is segmented from the newly added medical image;

[0214] Determine whether there is new lesion timeline data in the existing lesion timeline data, wherein the new lesion is the lesion represented by the new lesion image;

[0215] In response to the existence of lesion timeline data corresponding to newly added lesions in the existing lesion timeline data, the newly added lesion image is inserted into the lesion timeline data corresponding to the newly added lesion based on the time point of the newly added medical image;

[0216] In response to the absence of lesion timeline data corresponding to the newly added lesion in the existing lesion timeline data, lesion timeline data is created based on the newly added medical image.

[0217] As an optional approach, when the newly added medical image processing module performs corresponding processing based on the presence of lesions in the newly added medical image, it is specifically used for:

[0218] In response to the condition that the newly added medical image contains lesions, a timeline data of the lesions is created based on the newly added medical image.

[0219] As an optional embodiment, the above-mentioned device also includes a comparison display module (not shown in the figure), used for:

[0220] After constructing the lesion timeline data, the lesion timeline data is displayed based on a predetermined display style;

[0221] In response to a user's action of selecting at least two target time points based on the displayed lesion timeline data;

[0222] The medical images corresponding to the target time point are compared and displayed.

[0223] As an alternative, the above-mentioned device also includes a three-dimensional display module (not shown in the figure), used for:

[0224] After comparing and displaying the medical images corresponding to the target time points, in response to the user's operation of selecting a target area in any of the displayed medical images at the target time points, a three-dimensional model of the target area is constructed based on the medical images at each target time point, and the three-dimensional model corresponding to each target time point is displayed.

[0225] As an optional approach, the target area includes:

[0226] The area at a first specified distance from the center of the lesion selected by the user;

[0227] The area at a second specified distance from the target location selected by the user;

[0228] As an alternative, the above-mentioned device also includes a display control module (not shown in the figure), used for:

[0229] After displaying the target time in the 3D model corresponding to each target time point, in response to the user's display control operation on the 3D model corresponding to any target time point, the display control is performed on the 3D models corresponding to all target time points.

[0230] As an optional approach, the above-mentioned device also includes a preoperative planning document transfer module, which is used for:

[0231] After generating the preoperative planning file, in response to the command to transfer the preoperative planning file, the preoperative planning file is transferred to the intraoperative navigation system using a preset transfer method, which includes any of the following:

[0232] The preoperative planning documents are transferred to a removable storage device so that the removable storage device can be connected to the intraoperative navigation system, thereby transferring the preoperative planning documents to the intraoperative navigation system;

[0233] Based on a wired or wireless network connection to the intraoperative navigation system, the preoperative planning documents are transferred to the intraoperative navigation system.

[0234] As an optional embodiment, the above-described apparatus further includes an associated storage module (not shown in the figure), used for:

[0235] After constructing the lesion timeline data, the surgery-related files of the target lesion are associated and stored with the surgery time points in the lesion timeline data. The surgery-related files include:

[0236] Preoperative planning documents;

[0237] Intraoperative laparoscopic video;

[0238] Surgical report.

[0239] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0240] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0241] In addition, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of any of the methods in the foregoing method embodiments.

[0242] And an electronic device, comprising:

[0243] One or more processors; and

[0244] A memory associated with one or more processors, the memory being used to store program instructions that, when read and executed by one or more processors, perform the steps of any of the methods in the foregoing method embodiments.

[0245] In this embodiment, the electronic device can be an independent workstation that can be pushed into the operating room, providing a high-definition display screen. It can be positioned alongside the endoscope display screen in space to provide endoscope video and surgical navigation.

[0246] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0247] in, Figure 15An exemplary architecture of an electronic device is shown, which may include a processor 1510, a video display adapter 1511, a disk drive 1512, an input / output interface 1513, a network interface 1514, and a memory 1520. The processor 1510, video display adapter 1511, disk drive 1512, input / output interface 1513, network interface 1514, and memory 1520 can communicate with each other via a communication bus 1530.

[0248] The processor 1510 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs and implement the technical solution provided in this application.

[0249] The memory 1520 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1520 can store the operating system 1521 for controlling the operation of the electronic device 1500, and the basic input / output system (BIOS) 1522 for controlling the low-level operations of the electronic device 1500. Additionally, it can store a web browser 1523, a data storage management system 1524, and a medical data processing device 1525, etc. The aforementioned medical data processing device 1525 can be the application program that specifically implements the aforementioned steps in this embodiment. In summary, when implementing the technical solution provided in this application through software or firmware, the relevant program code is stored in the memory 1520 and is called and executed by the processor 1510.

[0250] Input / output interface 1513 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0251] Network interface 1514 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0252] Bus 1530 includes a pathway for transmitting information between various components of the device, such as processor 1510, video display adapter 1511, disk drive 1512, input / output interface 1513, network interface 1514, and memory 1520.

[0253] It should be noted that although the above-described device only shows the processor 1510, video display adapter 1511, disk drive 1512, input / output interface 1513, network interface 1514, memory 1520, bus 1530, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.

[0254] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer program product. This computer program product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0255] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A medical data processing method, characterized in that, include: Acquire medical images of a target patient at multiple time points, and segment lesion images from the medical images; Determine the pairing relationship of the lesion images, wherein the pairing relationship represents the lesion images of the same lesion at various time points; Based on the pairing relationship and lesion images, lesion timeline data is constructed.

2. The method according to claim 1, characterized in that, Determining the pairing relationship of the lesion images includes: Determine the deformation field between medical images at different time points; Based on the deformation field, registration and deformation processing are performed on medical images corresponding to different time points; Based on the results of the registration deformation processing, the pairing relationship between the lesion images is determined.

3. The method according to claim 1, characterized in that, The method further includes: Acquire new medical images; Determine whether the patient corresponding to the newly added medical image belongs to the target patient; In response to the fact that the patient corresponding to the newly added medical image belongs to the target patient, the existing lesion timeline data is updated based on the newly added medical image, wherein the existing lesion timeline data is the lesion timeline data of the patient corresponding to the newly added medical image; If the patient corresponding to the newly added medical image does not belong to the target patient, appropriate processing is performed based on the presence of lesions in the newly added medical image.

4. The method according to claim 3, characterized in that, The process of updating existing lesion timeline data based on the newly added medical images includes: The newly added lesion image is segmented from the newly added medical image; Determine whether there is new lesion timeline data in the existing lesion timeline data, wherein the new lesion is the lesion represented by the new lesion image; In response to the existence of lesion timeline data corresponding to newly added lesions in the existing lesion timeline data, the newly added lesion image is inserted into the lesion timeline data corresponding to the newly added lesion based on the time point of the newly added medical image; In response to the absence of lesion timeline data corresponding to the newly added lesion in the existing lesion timeline data, lesion timeline data is created based on the newly added medical image.

5. The method according to claim 3, characterized in that, The corresponding processing based on the presence of lesions in the newly added medical images includes: In response to the condition that the newly added medical image contains lesions, a timeline data of the lesions is created based on the newly added medical image.

6. The method according to any one of claims 1-5, characterized in that, After constructing the lesion timeline data, the method further includes: The timeline data of the lesions are displayed based on a predetermined display style; In response to a user's action of selecting at least two target time points based on the displayed lesion timeline data; The medical images corresponding to the target time point are compared and displayed.

7. The method according to claim 6, characterized in that, After comparing and displaying the medical images corresponding to the target time point, the method further includes: In response to a user selecting a target region in a medical image at any of the displayed target time points, a three-dimensional model of the target region is constructed based on the medical image at each target time point, and the three-dimensional model corresponding to each target time point is displayed.

8. The method according to claim 7, characterized in that, The target area includes: The area at a first specified distance from the center of the lesion selected by the user; The area at a second specified distance from the target location selected by the user.

9. The method according to claim 7, characterized in that, After displaying the 3D models corresponding to each target time point, the method further includes: In response to the user's control operation on the display of the 3D model corresponding to any target time point, the system performs corresponding display control on the 3D models corresponding to all target time points.

10. The method according to any one of claims 1-5, characterized in that, Also includes: After generating the preoperative planning file, in response to a command to transfer the preoperative planning file, the preoperative planning file is transferred to the intraoperative navigation system using a preset transfer method, wherein the preset transfer method includes any one of the following: The preoperative planning file is transferred to a removable storage device, so that the removable storage device can connect to the intraoperative navigation system, thereby transferring the preoperative planning file to the intraoperative navigation system; Based on a wired or wireless network connection with the intraoperative navigation system, the preoperative planning file is transferred to the intraoperative navigation system.

11. The method according to any one of claims 1-5, characterized in that, After constructing the lesion timeline data, the method further includes: The surgical-related files of the target lesion are associated with the surgical time points in the lesion timeline data and stored together. The surgical-related files include at least one of the following: Preoperative planning documents; Intraoperative laparoscopic video; Surgical report.

12. A medical data processing device, characterized in that, include: The lesion segmentation module is used to acquire medical images of the target patient at multiple time points and segment lesion images from the medical images; A pairing relationship determination module is used to determine the pairing relationship of the lesion images, wherein the pairing relationship represents the lesion images of the same lesion at various time points; The lesion timeline data construction module is used to construct lesion timeline data based on the pairing relationship and lesion images.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method according to any one of claims 1-11.

14. An electronic device, characterized in that, include: One or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method according to any one of claims 1-11.

15. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program performs the steps of the method described in any one of claims 1-11.

Images