Target localization methods, target localization systems and electronic equipment
By using target localization methods and models, and leveraging current image data and multimodal imaging, the problem of tumor target location deviation has been solved, achieving precision and stability in radiotherapy.
Patent Information
- Application Number
- CN202510638682.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-16
- Publication Date
- 2026-11-17
AI Technical Summary
In existing technologies, the location of the tumor target area depends on low-dose KV imaging, which has poor display effect. Furthermore, the tumor target area is easily affected by the patient's physiological activities during radiotherapy, leading to positional deviation and affecting the accuracy of radiotherapy.
The target area localization method is adopted. By acquiring the current image data of the target object, the target area localization model is used to determine the target area position offset based on the dynamic mapping relationship. The model is updated by combining multimodal medical image data to achieve accurate target area localization.
Accurately determining the target area deviation ensures the precision of radiotherapy, reduces the impact of target area deviation caused by physiological activities, and improves the radiotherapy effect.
Smart Images

Figure CN123118737A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of medical technology, and more particularly to the field of radiotherapy technology, specifically to a target localization method, a target localization system, and an electronic device. Background Technology
[0002] With the rapid development of precision medicine, it is necessary to accurately locate the target area of the tumor before performing radiotherapy, so that radiotherapy can be carried out precisely on the target area.
[0003] However, in current clinical practice, the location of the tumor target area is mainly determined by low-dose kinematic (KV) imaging. But KV imaging has poor visualization quality and cannot accurately reflect the location of the tumor. Moreover, during radiotherapy, the tumor target area may be affected by the patient's physiological activities, causing the tumor target area to shift, thus affecting the accuracy of radiotherapy to the tumor target area. Summary of the Invention
[0004] This application provides a target area localization method, a target area localization system, and an electronic device, which can accurately locate the target area.
[0005] In a first aspect, this application provides a target area localization method, comprising: acquiring current image data of the target object before irradiating the target area of the target object; inputting the current image data into a first target area localization model, the first target area localization model being used to determine the position offset of the target area based on a dynamic mapping relationship of the target area; the dynamic mapping relationship of the target area is: the correspondence between the position of the target area of the target object and the image data as the target area moves.
[0006] In some embodiments, the current image data includes: body surface optical data and / or X-ray projection images.
[0007] In some embodiments, the method further includes: obtaining a dynamic mapping relationship of the target area; the dynamic mapping relationship of the target area is determined by a second target area localization model based on multiple modal medical image data of the target object acquired during the localization phase.
[0008] In some embodiments, the second target area localization model is further used to: determine the target area based on multiple modal medical image data of the target object before irradiating the target area of the target object; and generate an irradiation plan for the target area based on the target area.
[0009] In some embodiments, the multimodal medical image data includes body surface data, and at least one of the following: X-ray projection images, computed tomography (CT) images, magnetic resonance imaging (MR), positron emission tomography (PET) images, PET-CT images, PET-MR images, single photon emission computed tomography (SPECT) images, SPECT-CT images, ultrasound imaging (US) images, digital subtraction angiographies (DSA) images, and pathological slide images.
[0010] In some embodiments, the first target area localization model is obtained by knowledge distillation of the second target area localization model.
[0011] In some embodiments, knowledge distillation includes one or more of the following: medical image data distillation, model parameter distillation, and irradiation strategy distillation.
[0012] In some embodiments, the method further includes: after the irradiation of the target area is completed, feeding back the irradiation data to the second target area localization model so that the second target area localization model is updated based on the irradiation data.
[0013] In some embodiments, irradiation data may include at least one of the following: abnormal information during the irradiation process, adjustment information of the irradiation strategy during the irradiation process, and the correspondence between the positional offset of the target area and the irradiation dose during the irradiation process.
[0014] In some embodiments, the method further includes: updating the first target area localization model and the second target area localization model based on federated learning.
[0015] In some embodiments, the first target area localization model is further configured to: output first indication information for indicating that the target area be irradiated when the positional offset of the target area is within a preset threshold range.
[0016] In some embodiments, the first target area localization model is further configured to: output second indication information for adjusting the position of the target area when the position offset of the target area is not within a preset threshold range.
[0017] In some embodiments, the method further includes: controlling the radiotherapy device to irradiate the target area when the time interval between the acquisition time of the current image data and the output time of the first indication information is less than a preset time interval.
[0018] In some embodiments, the method further includes: acquiring a radiotherapy guidance image of the target object during the process of acquiring current image data of the target area; inputting the current image data and the radiotherapy guidance image into a first target area localization model to determine the positional offset of the target area.
[0019] In some embodiments, the method further includes: determining the state of the target object based on the current image data of the target area; wherein the state of the target object includes an abnormal motion state and / or a breath-holding state; the above-mentioned inputting the current image data into the first target area localization model includes: inputting the current image data into the first target area localization model when the state of the target object meets a preset state.
[0020] In some embodiments, the method further includes irradiating the target area while the target object is in a breath-holding state and without abnormal movement.
[0021] In some embodiments, the target region is a tumor biological target region.
[0022] In some embodiments, the method of this application can be applied to each fraction of a multifractionated radiotherapy.
[0023] Secondly, this application also provides a target area localization system, including: a radiotherapy device, an image acquisition device, and a proximal device. The radiotherapy device is used to irradiate the target area, the image acquisition device is used to acquire image data of the target object, and the proximal device is equipped with the first target area localization model described in the first aspect.
[0024] In some embodiments, the system may further include a remote device. The remote device is configured with the second target area localization model described in the first aspect.
[0025] In some embodiments, the system further includes a positioning device. The positioning device can be used to simulate the positioning of a target object before irradiating the target area of the target object.
[0026] Secondly, this application also provides an electronic device comprising: a processor and a memory configured to store processor-executable instructions; wherein the processor is configured to execute the instructions to implement any of the optional target localization methods in the first aspect described above.
[0027] Thirdly, this application also provides a non-volatile storage medium storing a computer program, which, when read and executed, implements any of the optional target localization methods in the first aspect described above.
[0028] The target area localization method provided in this application allows the current image data of the target area to be input into a first target area localization model before irradiation of the target area of a target object. This enables the first target area localization model to determine the positional offset of the target area based on its dynamic mapping relationship. Since the dynamic mapping relationship of the target area changes with its movement, it represents the correspondence between the target area's position and the image data. Therefore, the positional offset of the target area determined based on this dynamic mapping relationship takes into account the influence of changes in the patient's physiological activities. Consequently, based on the positional offset of the target area, it is possible to accurately determine whether to irradiate the target area and the irradiation location. Attached Figure Description
[0029] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this application. Wherein:
[0030] Figure 1 This is a schematic diagram of a target area localization system provided in an embodiment of this application;
[0031] Figure 2 A schematic block diagram of a radiotherapy device provided in the embodiments of this application;
[0032] Figure 3 A schematic block diagram of another target area localization system provided in the embodiments of this application;
[0033] Figure 4 A schematic block diagram of yet another target area localization system provided in the embodiments of this application;
[0034] Figure 5 A schematic block diagram of yet another target area localization system provided in the embodiments of this application;
[0035] Figure 6 A flowchart illustrating a target localization method provided in an embodiment of this application;
[0036] Figure 7 A flowchart illustrating another target localization method provided in an embodiment of this application;
[0037] Figure 8 A flowchart illustrating another target localization method provided in an embodiment of this application;
[0038] Figure 9 A flowchart illustrating another target localization method provided in an embodiment of this application;
[0039] Figure 10 A schematic block diagram of a second target area localization model provided in an embodiment of this application;
[0040] Figure 11 A schematic block diagram of a target area positioning device provided in an embodiment of this application;
[0041] Figure 12 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation
[0042] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0043] In the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," or "third" may explicitly or implicitly include one or more of the stated features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.
[0044] In the description of this application, the term "exemplary" is used to mean "serving as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.
[0045] The following explanations of the terms used in the embodiments of this application are provided to facilitate the reader's understanding.
[0046] I. Target Area
[0047] The target volume (also known as the anatomical target volume) refers to the tumor area that needs to be treated with radiotherapy, in order to precisely kill tumor cells while protecting surrounding normal tissues as much as possible. Typically, the target volume can include the tumor target volume, clinical target volume, internal target volume, and planning target volume.
[0048] II. Heterogeneous Regions
[0049] Heterogeneous regions refer to areas within or around a tumor that exhibit significant differences in biological characteristics (such as metabolic activity, hypoxia, gene expression, and cell proliferation) or microenvironment (such as vascular distribution and immune infiltration). For example, heterogeneous regions may include high-risk subregions such as hypoxic areas, proliferative areas, and marginal invasive areas.
[0050] III. Tumor biological target area
[0051] Tumors typically exhibit high biological heterogeneity, with significant differences in cell proliferation rates, blood supply, metabolic activity, and radiosensitivity across different regions within the tumor. Traditional anatomical imaging can only provide information about the target area outline and cannot reveal the biological characteristics within the tumor.
[0052] The biological target volume (BTV) is a dynamically defined target area requiring radiotherapy, based on the heterogeneity of tumor biological characteristics (such as tumor metabolic activity, hypoxia, and proliferative capacity). The BTV can be considered a "re-division" within the traditional tumor target volume, a target within a target volume, more accurately reflecting the heterogeneity and dynamic changes of the tumor, and driving the transformation of radiotherapy from the traditional "anatomical target volume" paradigm to the "biological target volume" paradigm.
[0053] For example, tumor biological target areas may include hypoxic target areas, proliferative target areas, and marginal invasive target areas. Based on different types of tumor biological target areas, corresponding radiotherapy doses are determined in the irradiation protocol. This allows for targeted radiotherapy to the tumor.
[0054] IV. Introduction to the Entire Radiotherapy Process
[0055] Radiation therapy (RT) is a crucial treatment for cancer. Its core process can be divided into several stages, aiming to ensure precise radiation delivery to the tumor tissue while maximizing the protection of surrounding healthy tissue. The following are some of the key stages of RT:
[0056] 1. Diagnostic stage: Determine the necessity, indications, and treatment goals (radical or palliative) of radiotherapy.
[0057] 2. Simulation Stage: Acquire localization images of the patient in the treatment position and establish a radiotherapy reference coordinate system.
[0058] 3. Treatment Planning Phase: Based on the positioning images, develop an irradiation (also known as radiotherapy) plan and verify the feasibility and accuracy of the irradiation plan.
[0059] 4. Radiotherapy phase: This includes the patient setup phase and the beam delivery phase. In the setup phase, the patient is fixed and positioned to ensure that the patient's position is completely consistent with that in the positioning phase, reducing setup errors. In the beam delivery phase, the patient is precisely irradiated according to the irradiation plan to ensure that the dose distribution is consistent with the irradiation plan.
[0060] Normally, the target area can be located using medical imaging data, allowing for irradiation. However, dynamic physiological activities of the patient (such as coughing, breathing, gastrointestinal motility, non-voluntary changes in posture, and heartbeat) may cause the target area to shift, thus affecting the radiotherapy effect.
[0061] Therefore, embodiments of this application provide a target area localization method. This method inputs real-time acquired current image data of the target object into a target area localization model to quickly determine the positional offset of the target area based on a dynamic mapping relationship between the target area's location and the image data. Thus, based on the target area's positional offset, it can be accurately determined whether the target area at the current location can be irradiated, thereby ensuring the accuracy of radiotherapy.
[0062] The embodiments of this application will now be described in detail with reference to the accompanying drawings.
[0063] It should be noted that the various embodiments of this application can be referenced or learned from each other. For example, the same or similar steps, method embodiments, system embodiments and device embodiments can be referenced from each other without limitation.
[0064] Figure 1This is a schematic diagram of a target area localization system provided in an embodiment of this application. The target area localization system may include: an image acquisition device 101, a radiotherapy device 102, and a proximal device 103. The proximal device 103 is connected to the image acquisition device 101 and the radiotherapy device 102. The image acquisition device 101 and the radiotherapy device 102 are both located in the same radiotherapy space, such as the same treatment room.
[0065] In this embodiment of the application, the radiotherapy device 102 can be used to irradiate the target area.
[0066] In some examples, such as Figure 2 As shown, the radiotherapy device 102 may include a gantry 1021, a treatment head 1022, and a support device 1023.
[0067] The gantry 1021 can be a rotatable gantry. The treatment head 1022 can be mounted on the gantry and is used to emit a radiation beam for treatment, irradiating the target object. The radiation beam can be gamma rays, MV-level X-rays, proton rays, etc. For example, the treatment head 1022 can be any one or more of a gamma knife treatment head for rotationally focused radiotherapy, an accelerator treatment head for intensity-modulated radiotherapy, or other radiotherapy heads. In this embodiment, the treatment head 1022 can correspond to different radiotherapy modes, such as rotationally focused radiotherapy and intensity-modulated radiotherapy. The support device 1023 is used to fix, support, and move the target object, and can be a treatment bed.
[0068] In some embodiments, when the target object is on the support device 1023, the rotation of the gantry 1021 can drive the treatment head 1022 to irradiate the target object in 360 degrees, thereby completing radiotherapy.
[0069] In this embodiment, the image acquisition device 101 can be used to acquire image data of a target object (e.g., a patient, a phantom simulating a patient, etc.). The image acquisition device 101 can be used for positioning the target object and / or for real-time image guidance during radiotherapy.
[0070] In some embodiments, the image acquisition device 101 may be a body surface optical device, a cone beam computed tomography (CBCT) device, etc.
[0071] In some examples, the image acquisition device 101 can be integrated into the radiotherapy device 102. For example... Figure 2As shown, when the image acquisition device 101 is a CBCT device, the CBCT device is mounted on the gantry 1021 and includes an X-ray tube 1011A and a detector 1012B. The detector 1012B can receive the imaging beam emitted by the X-ray tube 1011A that passes through the target object and generate a KV-level X-ray projection image. This X-ray projection image is used to indicate the shape and / or location of the target area of the target object.
[0072] In other examples, the image acquisition device 101 may not be integrated into the radiotherapy device 102, but may be located in the same space as the radiotherapy device 102. If the image acquisition device 101 is a surface optical device, there may be one or more surface optical devices, used to acquire the current surface optical data of the target object in pre-treatment positioning scenarios and / or in radiation scenarios during radiotherapy.
[0073] like Figure 3 As shown, the surface optical acquisition device includes a first surface optical acquisition device 1013 and a second surface optical acquisition device 1014. The first surface optical acquisition device 1013 can be positioned above the support device 1023 of the radiotherapy equipment 102, for example, on the ceiling of the space where the radiotherapy equipment 102 is located. The first surface optical acquisition device 1013 can acquire the current surface optical data of the target object in a positioning scenario before radiotherapy. The second surface optical acquisition device 1014 can be positioned facing the target object on the treatment bed of the radiotherapy equipment 102 and can move with the support device 1023, for example, by using a bracket to position it at the foot of the treatment bed of the radiotherapy equipment 102. The second surface optical acquisition device 1014 can acquire the current surface data of the target object in a radiation scenario during radiotherapy. Of course, the surface optical acquisition device can also be positioned in other locations to ensure that the current surface data of the target object can be acquired in a timely manner when locating the target object.
[0074] In some examples, the target localization system provided in this application embodiment may further include a radiotherapy guidance device, which can be used to capture radiotherapy-guided images of the target object. The radiotherapy guidance device may be connected to the proximal device 103.
[0075] In this embodiment, the radiotherapy guidance device can be the same device as the image acquisition device 101 described above, or it can be a different device. For example, the radiotherapy guidance device may include any one or more of a CT scanner, a magnetic resonance imaging (MRI) scanner, a PET scanner, and an infrared acquisition device. Accordingly, the radiotherapy guidance images may include CT images, MRI images, PET images, infrared images, etc.
[0076] In this embodiment, the proximal device 103 may be configured with a first target area localization model. The proximal device may be located in the same radiotherapy space as the image acquisition device 101 and the radiotherapy device 102, such as a treatment room, or it may be located in a control room outside the treatment room.
[0077] In some embodiments, the near-end device 103 may acquire current image data of the target object from the image acquisition device 101 before irradiating the target area of the target object, and input the current image data into the first target area positioning model to determine the position offset of the target area.
[0078] Furthermore, in conjunction with the aforementioned radiotherapy guidance device, during the process of acquiring the current image data of the target object, the radiotherapy guidance device can send the acquired radiotherapy guidance image to the proximal device 103. After acquiring the radiotherapy guidance image of the target object, the proximal device 103 can input the current image data and the radiotherapy guidance image into the first target area localization model to determine the positional offset of the target area.
[0079] In some embodiments, the near-end device 103 can determine the state of the target object based on the current image data of the target object, and input the current image data into the first target area localization model if the state of the target object meets the preset state.
[0080] The state of the target object includes abnormal motion state and / or breath-holding state.
[0081] In some embodiments, the first target area localization model can be used to determine the positional offset of the target area based on the dynamic mapping relationship of the target area. The dynamic mapping relationship of the target area can be: the correspondence between the position of the target area of the target object and the image data as the target area moves.
[0082] In some embodiments, the first target area localization model can also be used to output indication information on whether to irradiate the target area based on the positional offset of the target area.
[0083] For example, if the positional offset of the target area is within a preset threshold range, the first target area localization model can output first indication information to indicate that the target area should be irradiated.
[0084] For example, if the positional offset of the target area is not within a preset threshold range, the first target area localization model can output second indication information to indicate the adjustment of the target area position.
[0085] Furthermore, in order to ensure accurate irradiation of the target area, the proximal device 103 can control the radiotherapy device 102 to irradiate the target area if the time interval between the acquisition time of the current image data of the target area and the output time of the first indication information is less than a preset time interval.
[0086] In some embodiments, when the target object is in a breath-holding state and has no abnormal movement, the proximal device 103 can control the radiotherapy device 102 to irradiate the target area.
[0087] In some examples, the proximal device 103 is communicatively connected to the image acquisition device 101 and the radiotherapy device 102. Before the radiotherapy device 102 irradiates the target area of the target object, the image acquisition device 101 acquires the current image data of the target object and sends it to the proximal device 103, so that the proximal device 103 performs the target area localization method mentioned later.
[0088] In some embodiments, the near-end device 103 may be a computer device with a graphical user interface (GUI), which includes one or more processors, memory, and one or more applications. For example, the near-end device 103 may include a target localization system application, and the processor of the near-end device 103 executes the target localization system application to implement the target localization method provided in the embodiments of this application.
[0089] In the embodiments disclosed herein, the entity of the near-end device 103 may be a terminal or a server with a display, and this application embodiment does not limit it in this regard.
[0090] Optionally, the aforementioned terminal may be at least one of the following devices: smartphone, smartwatch, desktop computer, laptop, virtual reality terminal, augmented reality terminal, wireless terminal, and laptop computer.
[0091] Optionally, the aforementioned server may be a standalone physical server, a server cluster consisting of multiple physical servers, a distributed file system, or at least one of the following cloud servers providing basic cloud computing services: cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data or artificial intelligence platforms. This disclosure does not limit the specific type of server. In some embodiments, the number of servers may be more or fewer, and this disclosure does not limit the specific type of server. Of course, the server may also include other functions to provide more comprehensive and diverse services.
[0092] In some embodiments, the near-end device 103 may run a computer system including a processor for implementing the target localization method provided in the embodiments of this application.
[0093] In some embodiments, such as Figure 4 As shown, the target area localization system provided in this application embodiment may further include a remote device 104. The remote device 104 can be communicatively connected to the near-end device 103 and is configured with a second target area localization model.
[0094] Among them, the second target area localization model is used to determine the dynamic mapping relationship of the target area based on the multimodal medical image data of the target object acquired during the localization phase.
[0095] In some embodiments, the multimodal medical image data includes surface optical data, and at least one of the following:
[0096] X-ray projection images, computed tomography (CT) images, magnetic resonance imaging (MR) images, positron emission tomography (PET) images, PET-CT images, PET-MR images, single-photon emission tomography (SPECT) images, SPECT-CT images, ultrasound (US) images, digital subtraction angiography (DSA) images, and pathological slide images.
[0097] It should be noted that multimodal medical image data are not necessarily localization images acquired during the localization phase. They may be image data acquired before the localization phase, such as during the diagnostic phase. Here, the localization phase emphasizes that the establishment of a dynamic mapping relationship between the target area is carried out during the localization phase.
[0098] In some embodiments, the second target area localization model is further used to determine the target area based on multiple modal medical image data of the target object before irradiating the target area of the target object; and to generate an irradiation plan for the target area based on the target area.
[0099] In some embodiments, the second target area localization model can also be used in the pre-treatment stage to process complex multimodal medical image data (such as CT images, PET images, MRI images, body surface optical data, etc.), and construct a patient's personalized "biological target area global feature library" through high-dimensional feature extraction and deep learning to achieve high-precision target area modeling and knowledge representation.
[0100] In some embodiments, the second target area localization model is also used to acquire irradiation data and update it based on the irradiation data after the irradiation of the target area has ended.
[0101] In some examples, after the near-end device 103 finishes irradiating the target area, it can send irradiation data back to the far-end device 104. Accordingly, the near-end device 103 receives the irradiation data from the far-end device 104.
[0102] The irradiation data may include at least one of the following: abnormal information during the irradiation of the target area, adjustment information of the irradiation strategy during the irradiation of the target area, and the correspondence between the positional offset of the target area and the irradiation dose during the irradiation of the target area.
[0103] In this embodiment, a first target region localization model can be obtained by knowledge distillation of the second target region localization model. The first target region localization model can possess the key features and core functions of the second target region localization model. The first target region localization model has fewer model parameters than the second target region localization model, and compared to the second target region localization model, the first target region localization model has advantages such as lower training and deployment costs and faster inference speed.
[0104] Knowledge distillation can include one or more of the following: medical image data distillation, model parameter distillation, and irradiation strategy distillation.
[0105] In this embodiment, the function of the second target area localization model includes: extracting global features from multiple modal medical data, and identifying and locating the target area based on the extracted global features. Correspondingly, the function of the first target area localization model may include: rapidly locating the target area based on image data of the target area in a real-time scenario.
[0106] In some embodiments, the remote device 104 may be a computer device with a graphical user interface, the computer device including one or more processors, memory, and one or more applications.
[0107] In the embodiments of this disclosure, the entity of the remote device 104 may be a terminal or a server with a display, and this application embodiment does not limit it in this way.
[0108] Optionally, the aforementioned terminal may be at least one of the following devices: smartphone, smartwatch, desktop computer, laptop, virtual reality terminal, augmented reality terminal, wireless terminal, and laptop computer.
[0109] Optionally, the aforementioned server may be a standalone physical server, a server cluster consisting of multiple physical servers, a distributed file system, or at least one of the following cloud servers providing basic cloud computing services: cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data or artificial intelligence platforms. This disclosure does not limit the specific type of server. In some embodiments, the number of servers may be more or fewer, and this disclosure does not limit the specific type of server. Of course, the server may also include other functions to provide more comprehensive and diverse services.
[0110] In some embodiments, the remote device 104 may run a computer system including a processor for implementing the target area localization method provided in the embodiments of this application.
[0111] In some embodiments, the target area localization system described above may further include a localization device. The localization device can be used to simulate the localization of a target object before irradiation of its target area.
[0112] For example, the positioning device can perform simulated positioning based on image data of the target object. For instance, before radiotherapy to the target area of the target object, images of the target area can be acquired to obtain image data of the target area. For example, image acquisition can be performed using at least one of the following devices: computed tomography (CT) equipment, emission computed tomography (ECT) equipment, magnetic resonance imaging (MRI) equipment, PET equipment, and ultrasound examination equipment.
[0113] Furthermore, after obtaining the model location of the target area, the positioning equipment can determine the irradiation plan for the target area of the target object through the radiotherapy planning equipment.
[0114] In some embodiments, the radiotherapy planning equipment may include a TPS client and a TPS server. The TPS client may be at least one of a smartphone, smartwatch, desktop computer, laptop, virtual reality terminal, augmented reality terminal, wireless terminal, and laptop computer. For example, in some embodiments, a user running a radiotherapy planning system on a TPS server via the TPS client can trigger the TPS server to perform an adaptive irradiation protocol optimization process and display the optimized irradiation protocol. This effectively saves user time and provides a more intuitive presentation of the optimized irradiation protocol, allowing users to evaluate the irradiation protocol.
[0115] Figure 5 This is a schematic diagram of another target area localization system provided in an embodiment of this application. The target area localization system may include a model layer, a middleware layer, and a device layer. These layers are described below.
[0116] 1-1, Model Layer
[0117] The model layer can include a first model layer and a second model layer. The first model layer can be deployed on the near-end device 103 and can be configured with the first target area localization model. The second model layer can be deployed on the far-end device 104 and can be configured with the second target area localization model. The first model layer can acquire the current image data of the target object (such as a patient) and, combined with the knowledge of the second target area localization model, quickly perform target area localization and output indication information on whether irradiation should be performed. The second model layer can fuse multiple modal medical image data of the target object and output the core features (knowledge) of the distillation to the first model layer.
[0118] In this embodiment, the second model layer can also be used for global learning, feature extraction, knowledge distillation, and target area modeling of multiple modal medical image data. The first model layer can also be used for real-time inference (such as determining whether to perform radiotherapy based on the original irradiation plan), dynamic adjustment of radiotherapy position and / or dose, target area localization, and dose optimization.
[0119] The second model layer can optimize the complex features and knowledge of the second target area localization model using knowledge distillation techniques to obtain the first target area localization model. In this way, the first target area localization model can possess the core capabilities of the second target area localization model, but with a significant reduction in computational load and model parameters.
[0120] In this embodiment, when the first target area localization model is running on the near-end device, it can feed back the abnormal data and irradiation data generated during the real-time inference process to the second target area localization model, so that the second target area localization model can be updated based on the abnormal data and irradiation data fed back from the first target area localization model.
[0121] By leveraging the collaborative operation of the first and second target localization models, a balance can be achieved between high precision and high real-time performance. This satisfies the accuracy requirements of precise radiotherapy while also ensuring the real-time nature and stability of the treatment process.
[0122] 1-2, Equipment Layer
[0123] The device layer can be deployed in the aforementioned devices close to the target object, such as in the image acquisition device 101 or the radiotherapy device.
[0124] When the image acquisition device 101 is deployed at the device layer, the device layer can be used to acquire multiple modal medical image data of the target object and send the acquired multiple modal medical image data to the model layer. The device layer can also be used to acquire image data of the target object in real time and send the acquired image data to the model layer.
[0125] When the device layer is deployed on the radiotherapy equipment, the device layer can execute instructions from the model layer and feed back irradiation data and equipment operation data to the model layer.
[0126] Furthermore, to ensure normal data transmission between the device layer and the model layer, such as Figure 4 As shown, the target area localization system provided in this application embodiment may further include:
[0127] 1-3. Middleware Layer
[0128] The middleware layer can be deployed on the proximal device 103, the image acquisition device 101, or the radiotherapy device. Alternatively, the middleware layer can be deployed on a separate device located between the devices deployed in the model layer and the devices deployed in the device layer. For example, it can be deployed between the proximal device 103 and the image acquisition device 101.
[0129] In this embodiment, the middleware layer can be used to implement data transmission between the model layer and the device layer. For example, the middleware layer can forward instruction information from the model layer to the device layer, as well as image data, irradiation data, device operation data, and patient breath-holding status information from the device layer to the model layer.
[0130] The instructions may include movement adjustment commands for the radiotherapy equipment, dose distribution adjustment commands, and breath-hold control commands. Movement adjustment commands can be used to adjust the position of the radiotherapy equipment. Dose distribution adjustment commands can instruct the radiotherapy equipment to adjust the irradiation dose to different areas of the target area. Breath-hold control commands can instruct the patient to hold their breath during target irradiation. Patient breath-hold status information may include the patient's respiratory status and whether they are holding their breath. Equipment operating status may include whether the equipment is operating normally.
[0131] In this embodiment, the device housing the model layer and the device housing the device layer support different data transmission protocols. The middleware layer can convert the format of the received data / information and forward the converted data / information. This ensures that both the device housing the model layer and the device housing the device layer can transmit data normally. Figure 5The system architecture can achieve a combination of high precision (the function of the second model layer) and high real-time performance (the function of the first model layer). At the same time, the middleware layer decouples the model layer from the device layer, ensuring the flexibility and scalability of the system.
[0132] The target area localization system in the embodiments of this application has been described above.
[0133] The target area localization method in this application embodiment can be applied to the aforementioned target area localization system. The target area localization method will now be described in detail.
[0134] It should be noted that the target localization method provided in this application embodiment can be applied to each radiotherapy session in multi-fraction radiotherapy, or to a single radiotherapy session.
[0135] Figure 6 This is a flowchart illustrating a target area localization method provided in an embodiment of this application. The target area localization method provided in this application can be applied to near-field devices configured with a first target area localization model, such as... Figure 6 As shown, the method includes the following S601-S602.
[0136] S601. Before irradiating the target area of the target object, acquire the current image data of the target object.
[0137] The target object can be a patient awaiting radiotherapy or a phantom simulating a patient. The target area can be the region on the target object that needs to be irradiated. The current image data of the target object can include surface optical data and / or X-ray projection images.
[0138] In one possible implementation, before irradiating the target area of the target object, the near-end device can acquire the current image data of the target object through an image acquisition device.
[0139] S602. Input the current image data into the first target area localization model to obtain the position offset of the target area.
[0140] The target area offset can refer to the deviation between the current position of the target area and a reference position, such as the position of the target area in an irradiation scheme. The current position of the target area can be determined based on the current image data of the target object. Specifically, please refer to the description of the following embodiments.
[0141] In one possible implementation, the first target area localization model is used to determine the position offset of the target area based on the dynamic mapping relationship of the target area after acquiring the current image data of the target object.
[0142] The dynamic mapping relationship of the target area can be defined as the correspondence between the position of the target area and the previous image data as the target area moves. The movement change of the target area can refer to the change in the position of the target area caused by the physiological activities of the target object (such as involuntary movements like coughing and breathing). Movement changes can include both non-periodic and periodic movements. Non-periodic movements can include conscious movements such as coughing and convulsions, while periodic movements can include involuntary movements such as breathing, heartbeat, pulse, and gastrointestinal motility.
[0143] based on Figure 6 The proposed technical solution involves inputting the current image data of the target area into a first target area localization model before irradiating the target area. This allows the model to determine the target area's positional offset based on its dynamic mapping relationship. Since this dynamic mapping relationship changes with the target area's movement, it represents the correspondence between the target area's position and the image data. Therefore, the positional offset determined based on this dynamic mapping relationship takes into account the impact of changes in the patient's physiological activities. Consequently, based on the target area's positional offset, it is possible to accurately determine whether to irradiate the target area and the irradiation location.
[0144] It should be noted that the target region in the above embodiments can be a biological tumor target region.
[0145] In some embodiments, the method provided in this application may further include: acquiring a radiotherapy-guided image of the target object during the process of acquiring the current image data of the target object.
[0146] Radiotherapy-guided images can include the target area inside the target object. Radiotherapy-guided images are characterized by low dose and rapid imaging.
[0147] Furthermore, in S602 above, the proximal device can input the current image data of the target object and the radiotherapy guidance image into the first target area localization model to obtain the position offset of the target area.
[0148] Here, the first target localization model also has a fusion / registration function, which can fuse or register the current image data of the target object with the radiotherapy guidance image. For example, if the current image data is surface optical data and the radiotherapy guidance image is an X-ray projection image, the first target localization model can register the current surface optical data of the target object with the current X-ray projection image.
[0149] In this embodiment, since the radiotherapy guidance image of the target object can reflect the location information of the target area inside the target object, based on the current image data of the target object and the radiotherapy guidance image, there is more tissue information, which can more accurately locate the target area.
[0150] In some embodiments, such as Figure 7 As shown, the method provided in this application embodiment may further include: S701 and S702.
[0151] S701. Determine the state of the target object based on the current image data of the target area.
[0152] The target's state can include abnormal movement and / or breath-holding. Abnormal movement states can include sudden body movements, abnormal respiratory rates, and other abnormal movement states. Normal movement states can include normal breathing and heart rate.
[0153] In one possible implementation, the current image data includes surface optical data. The proximal device can then detect in real time whether the patient is experiencing any abnormal operating conditions based on the surface optical data.
[0154] In this embodiment, surface optical data can capture the amplitude, period, and phase of respiratory movements in real time, providing an indirect but effective reference for organ movement. That is, proximal devices can detect the patient's condition based on surface optical data.
[0155] Furthermore, the above-mentioned S602 can be implemented through S702.
[0156] S702. If the state of the target object meets the preset state, input the current image data into the first target area localization model.
[0157] The preset state can include a state without abnormal movement, that is, a state in which the target object does not move abnormally; the preset state can also include a state without abnormal movement and a breath-holding state.
[0158] based on Figure 7 The proposed technical solution, after acquiring the current image data of the target object, can determine the current state of the target object based on this data. If the target object is in a preset state, it indicates that the position of the target area will not be affected by abnormal movement of the target object. In this case, the current image data can be input into the first target area localization model. This allows for a more accurate determination of the target area's positional offset.
[0159] In some embodiments, after determining the position offset of the target area, the first target area positioning model provided in this application embodiment is further configured to: output first indication information for indicating irradiation of the target area when the position offset of the target area is within a preset threshold range; and output second indication information for indicating adjustment of the target area position or third indication information for indicating cessation of irradiation when the position offset of the target area is not within the preset threshold range.
[0160] The preset threshold range can be set as needed, for example, it can be 0-1 mm. That is, when the positional offset of the target area is less than or equal to 1 mm, the first indication information can be output. When the positional offset of the target area exceeds 1 mm, the second indication information can be output.
[0161] It is understandable that if the position offset of the target area is not within the preset threshold range, after adjusting the position of the target area based on the second instruction information, the first target area positioning model can redetermine the position of the target area until the offset of the adjusted target area is within the preset threshold range. The radiotherapy device then irradiates the redetermined target area based on the first instruction information.
[0162] For example, the first target area localization model can determine the current position of the target area based on the target area dynamic adjustment rule. The target area dynamic adjustment rule can be a rule governing the change in the target area's positional offset and the parameters of its shape and / or position.
[0163] Based on the technical solution of this embodiment, when the positional offset of the target area is within a preset threshold range, it indicates that the positional offset of the target area is within an acceptable range, meaning that the target area can be irradiated according to the current irradiation plan. When the positional offset of the target area is not within the preset threshold range, it indicates that the positional offset of the target area is large. If the target area continues to be irradiated according to the current irradiation plan, it may lead to poor radiotherapy results. Therefore, by adjusting the position of the target area, it can be ensured that the target area can be accurately irradiated subsequently.
[0164] Furthermore, the first target area localization model provided in this application embodiment is specifically used for: outputting first indication information for irradiating the target area when the positional offset of the target area is within a first preset threshold range; outputting second indication information for adjusting the position of the target area when the positional offset of the target area is not within the first preset threshold range but within a second threshold range; and outputting third indication information for stopping irradiation when the positional offset of the target area exceeds the second threshold range. The first threshold range is greater than or equal to the first threshold and less than or equal to the second threshold, and the second threshold range is greater than the second threshold and less than the third threshold. For example, the first preset range can be [0mm, 1mm], and the second preset range can be (1mm, 3mm). This avoids situations where irradiation continues even when the target area has significantly shifted, thus preventing damage to the radiotherapy effect.
[0165] In some embodiments, to ensure the accuracy of irradiation, the method provided in this application may further include: when the time interval between the acquisition time of the current image data of the target area and the output time of the first indication information is less than a preset time interval, the near-end device may control the irradiation device to irradiate the target area.
[0166] The preset time interval can be in the millisecond range. For example, it can be any value between 20 milliseconds and 50 milliseconds, such as 20 milliseconds, 30 milliseconds, 50 milliseconds, etc.
[0167] In one example, the near-end device can determine the time interval between the acquisition time of the current image data and the output time of the first indication information based on the timestamp carried by the current image data and the timestamp of the first indication information.
[0168] In this way, because the time interval between the image acquisition time of the image acquisition device and the time indicated for irradiating the target area is very short, on the order of milliseconds, the target area will not undergo significant positional shift during this period. Therefore, the irradiation device can accurately irradiate the target area.
[0169] In some embodiments, the method provided in this application may further include irradiating the target area while the target object is in a breath-holding state and without abnormal movement. Because the target object is in a breath-holding state and without abnormal movement, the movement of the target area within the target object is restricted. Thus, the irradiation beam can be more precisely applied to the target area, improving the accuracy of radiotherapy.
[0170] In one example, combining the above Figure 7 In one embodiment, the near-end device can determine whether the target object is in a breath-holding state or a running state based on the current image data of the target object.
[0171] In one possible implementation, if the positional offset of the target area is determined to be within a preset threshold range, and if the target object is in a breath-holding state and exhibits no abnormal operating conditions, the proximal device can send a first instruction message to the radiotherapy device. Upon receiving the first instruction message, the radiotherapy device can respond by irradiating the target area.
[0172] In another possible implementation, the near-end device can be configured with a state detection model, which can be used to detect whether the target object is in a breathless state and whether there is an abnormal operating state. This state detection model can be a first target localization model, or it can be a different model from the first target area localization model.
[0173] In one example, the near-end device can input the current image data of the target object into the state detection model, so that the state detection model can detect whether the target object is in a breathless state and whether there is an abnormal operating state.
[0174] If the target object is in a breath-holding state and the positional offset of the target area does not exceed a preset threshold range, the target area can be irradiated. If the patient is not in a breath-holding state and / or has an abnormal operating state, irradiation of the target area can be paused, or an alarm message indicating an abnormal breath-holding state can be output.
[0175] Furthermore, during the irradiation of the patient, it is possible to monitor in real time whether the irradiation is being carried out within the breath-holding window. If it is within the breath-holding window, the target area can be irradiated; if it is outside the breath-holding window, or outside the breath-holding window, the irradiation of the target area can be paused.
[0176] Based on the technical solution of this embodiment, the position of the target area will not be shifted when the target object is in a breathless state and there is no abnormal operation. Therefore, the near-end device can irradiate the target area and ensure that the target area can be accurately irradiated.
[0177] In some embodiments, the method provided in this application may further include: obtaining the dynamic mapping relationship of the target area.
[0178] The dynamic mapping relationship of the target area is determined by the second target area localization model based on the multimodal medical image data of the target object acquired during the localization phase.
[0179] In this embodiment, the dynamic position of the target area can be quickly determined using image data of the target area and optical data of the target object's surface. Furthermore, since the optical data can compensate for the insufficient temporal resolution of the image data, it can provide real-time, continuous dynamic information about the target area.
[0180] In some embodiments, the first target area localization model can be obtained by knowledge distillation of the second target area localization model.
[0181] Knowledge distillation can include at least one of the following: medical image data distillation, model parameter distillation, and irradiation strategy distillation. Medical image data distillation can refer to extracting intermediate features (such as features from the fusion of multiple modalities of medical images) from the second target area localization model and optimizing them through feature matching and a loss function. Model parameter distillation can refer to compressing the key model parameters (such as registration matrices and motion mapping functions) of the second target area localization model into a format usable by the first target area. Irradiation strategy distillation can refer to converting the strategy (such as dose optimization strategy) of the second target area localization model into lightweight rules for use by the first target area localization model.
[0182] Based on the technical solution of this embodiment, since the first target area localization model is obtained by knowledge distillation of the second target area localization model, that is, the first target area localization model has the function of the second target area localization model, and the first target area localization model is small in size, the near-end device can quickly and accurately locate the target area based on the first target area localization model without consuming a lot of storage resources.
[0183] In some embodiments, the second target area localization model can also be used to: determine the target area based on multiple modal medical image data of the target object before irradiating the target area of the target object, and generate an irradiation plan for the target area based on the target area.
[0184] The irradiation protocol includes: a dynamic target location adjustment protocol and a dose distribution protocol. Multiple modal medical imaging data include surface optical data and imaging images. Imaging images may include at least one of the following: X-ray projection images, computed tomography (CT) images, magnetic resonance imaging (MR) images, positron emission tomography (PET) images, PET-CT images, PET-MR images, single-photon emission tomography (SPECT) images, SPECT-CT images, ultrasound (US) images, digital subtraction angiography (DSA) images, and pathological section images.
[0185] In some embodiments, the method provided in this application may further include: after irradiating the target area, the near-end device may feed back irradiation data to the second target area localization model, so that the second target area localization model can be updated based on the irradiation data. Alternatively, the near-end device may feed back irradiation data to the far-end device, and the far-end device may update the second target area localization model based on the irradiation data after receiving it.
[0186] The irradiation data may include at least one of the following:
[0187] 2-1. Abnormal information during the irradiation process.
[0188] Among them, abnormal information can be used to indicate / include: abnormal respiratory status of the target object (such as a patient’s respiratory rate being too fast), abnormal operation of radiotherapy equipment, abnormal current image data of the target area, abnormal morphology of the target area, and one or more of the target areas not being completely covered by the irradiation area.
[0189] 2-2. Information on adjustments to irradiation strategies during the irradiation process.
[0190] The adjustment information for the irradiation strategy may include the adjusted position of the target area and the irradiation dose of the target area.
[0191] In one example, if the target area's positional offset is outside a preset threshold range, the near-end device can reposition the target area based on the current image data due to the offset. Simultaneously, the offset may also cause changes in the shape of heterogeneous regions and boundary regions within the target area. Since different regions correspond to different irradiation doses, the near-end device can readjust the irradiation dose for each region based on its shape.
[0192] 2-3. The correspondence between the positional offset of the target area and the irradiation dose during irradiation.
[0193] The correspondence between positional offset and irradiation dose refers to the irradiation dose adjusted after the position of the target area is adjusted. That is, after the proximal device controls the radiotherapy equipment to irradiate the target area, it can obtain the positional offset of the target area and the corresponding irradiation dose used during radiotherapy, thus establishing the correspondence between the positional offset and the irradiation dose.
[0194] In this embodiment of the application, the first target area localization model can also be updated while updating the second target area localization model. For example, the first and second target area localization models can be updated based on federated learning.
[0195] In one example, after updating the localization model of the second target area, the remote device can perform knowledge distillation on the updated model parameters and send the distilled model parameters to the near device, so that the near device can update based on the distilled model parameters.
[0196] Based on the technical solution of this embodiment, after the radiotherapy to the target area is completed, the proximal device can feed back the irradiation data to the second target area localization model, so that the second target area localization model can be updated based on the irradiation data, ensuring the accuracy of the second target area localization model.
[0197] One possible approach is to combine the above-described radiotherapy process.
[0198] During the localization phase, multiple modal medical image data of the target object can be acquired using localization devices. These multiple modal medical image data include surface optical data and at least one of the following:
[0199] X-ray projection images, CT images, MR images, PET images, PET-CT images, PET-MR images, SPECT images, SPECT-CT images, US images, DSA images, and pathological slide images.
[0200] During the design and verification phases, the second target area localization model can acquire multiple medical image data of the target object. Based on the fusion of these multiple medical image data, the target area is determined, and a dynamic mapping relationship between the target area and the target object is generated. An irradiation plan is then generated based on the target area, and the irradiation plan generated by the second target area localization model is verified. Based on this embodiment, since the medical image data of the target object can reflect the location of the target area, fusing multiple modal medical image data of the target object can more accurately locate the target area. Therefore, the irradiation plan developed for the target area can also be more precise.
[0201] During radiotherapy, before irradiating the target area of the target object, the first target area localization model obtains the dynamic mapping relationship of the target area of the target object, and uses image acquisition equipment to acquire the current image data of the target object, such as surface optical data and / or X-ray projection images. The current image data is input into the first target area localization model to determine the positional offset of the target area based on the dynamic mapping relationship of the target area of the target object.
[0202] The target area localization method involved in this application will be explained in detail below based on specific embodiments.
[0203] With the rapid development of precision medicine, tumor radiotherapy is gradually shifting from the traditional "anatomy-driven" model to a "bio-driven" model. The "anatomy-driven" model refers to using physical techniques to precisely control dose distribution based on the tumor's structure. The "bio-driven" model, on the other hand, is guided by the tumor's biological characteristics, incorporating molecular markers and functional influences to determine individualized treatment plans.
[0204] However, in clinical practice, tumor target delineation technology still faces multiple bottlenecks, urgently requiring technological innovation and breakthroughs. On the one hand, traditional target delineation mainly relies on single images (such as CT images, PET images, etc.), but a single image cannot fully integrate the structural, metabolic, and functional information of the tumor, resulting in blurred target boundaries and insufficient identification of heterogeneous regions, directly affecting the accuracy and efficacy of treatment. On the other hand, traditional target delineation is highly dependent on manual annotation, limited by the doctor's subjective experience and operating habits. Under dynamic physiological states (such as coughing, involuntary changes in body position, breathing, gastrointestinal peristalsis, heartbeat, and other involuntary movements), target identification exhibits significant inconsistencies and errors, making it difficult to meet the individualized needs of complex cases.
[0205] Furthermore, precision radiotherapy places stringent demands on the real-time responsiveness of equipment, the efficiency of algorithms, and cross-platform compatibility. However, existing AI-assisted decision-making systems largely rely on large-scale cloud models, which suffer from communication delays, poor adaptability to network outages, and insufficient hardware computing power, failing to meet the real-time and reliability requirements of dynamic treatment scenarios. Therefore, the accuracy and safety of radiotherapy face significant challenges.
[0206] Therefore, the following explanation will focus on the coordinated operation of the first and second target area localization models to achieve accurate target area identification, using the example of surface optical data and X-ray projection images to illustrate the target area localization method. Figure 8 As shown, the method includes S801 to S808.
[0207] S801: The remote device trains based on the training data to obtain the second target area localization model.
[0208] The training data includes multimodal medical image data of multiple objects, as well as the correspondence between the position of the target area and the image data of different objects that change with the movement of the target area.
[0209] Here, multimodal medical image data includes surface optical data and at least one of X-ray projection images, CT images, MR images, PET images, PET-CT images, PET-MR images, SPECT images, SPECT-CT images, ultrasound US images, DSA images, and pathological slide images.
[0210] Among these, surface optical data primarily reflects the dynamic changes of the patient's external body surface. In radiotherapy settings, surface optical data is correlated with the movement of the patient's organs. For example, in radiotherapy for abdominal tumors (such as lung or liver tumors), the patient's respiratory movements cause the target area to shift. Surface optical data can acquire real-time data reflecting organ movement, including the amplitude, period, and phase of respiratory movements.
[0211] For example, because the acquisition of surface optical data is non-invasive and does not cause additional harm to the patient, and the acquisition speed is fast, the target area can be monitored in real time using surface optical data.
[0212] For example, in breath-holding radiotherapy, surface optical data can be used to monitor in real time whether the patient is holding their breath and the stability of that breath-holding state, thus providing feedback for irradiation.
[0213] In some cases, surface optical data can be used for positioning guidance and dynamic monitoring during radiotherapy. Surface optical data can include displacement information of landmarks. This displacement information can reflect the movement of internal organs.
[0214] In one possible implementation, optical data of the target object's surface can be acquired using a camera (such as a depth camera or optical camera) or a scanner (such as a 3D optical scanner or laser scanner).
[0215] In this embodiment, the trained second target area localization model takes multiple modal medical image data of the target object as input and outputs a dynamic mapping relationship of the target area. This dynamic mapping relationship is the correspondence between the position of the target area of the target object and the image data as the target area moves. Specifically, the generation process of the second target area localization model may include the following steps:
[0216] Step 1: Data preparation.
[0217] Data preparation can include data annotation and data preprocessing. Data annotation refers to assigning the actual location of the standard target area at each time point to the corresponding frame of optical and image data. Data preprocessing can include processing the optical surface data and processing the image data.
[0218] Specifically, processing of body surface optical data can include: denoising (filtering out high-frequency noise in the optical signal), normalization (normalizing marker coordinates or point cloud data to a unified standard), and time alignment (aligning the time points of body surface optical data with those of image data). Processing of image data can include: image segmentation (extracting the target area) and feature extraction (extracting image features using a pre-defined deep learning model). Deep learning models can include ResNet and UNet.
[0219] Step 2: Model Design.
[0220] Among them, model design refers to selecting a suitable initial model or algorithm as the model architecture for the second target area localization model.
[0221] In this embodiment of the application, a deep learning model can be used as the model architecture for the second target area localization model.
[0222] Step 3: Model training.
[0223] The loss function used in model training can be mean squared error, which measures the error between the predicted and actual locations of the target area.
[0224] During the training process, the training data can be augmented, the model optimized, and evaluated to obtain the trained second target area localization model.
[0225] Data augmentation can include enhancing body surface optical data and adding to image data. Enhancing body surface optical data can include adding random noise to the data to compensate for errors in the actual data acquisition process; and randomly truncating or expanding the time series of body surface optical data to increase the model's robustness.
[0226] The model can be optimized using the Adam optimizer or other optimization algorithms suitable for deep learning. The learning rate can be set to be dynamically adjusted (e.g., using a learning rate scheduler).
[0227] Model evaluation can involve using a validation set to assess the model's prediction accuracy. Evaluation metrics can include: mean absolute error (MAE), root mean square error (RMSE), and correlation of dynamic trajectories (such as DTW distance).
[0228] S802. The near-end device trains and generates a first target area localization model, which is obtained by knowledge distillation of the second target area localization model.
[0229] The knowledge distillation results of the second target area localization model can include: multimodal medical image data fusion rules, target area boundaries and heterogeneous regions, dose optimization strategies, anomaly detection models and adjustment rules, and target area dynamic mapping relationships.
[0230] During knowledge distillation, the second target localization model can also adjust the weights of different modal features according to scenario requirements. For example, in respiratory-sensitive scenarios, surface optical data and CT images are distilled first. In heterogeneous region identification scenarios, PET images and MRI images are distilled first.
[0231] In this embodiment, the second target area localization model can also transfer distilled key information to the first target area localization model. This key information can include global target area features and dynamic motion models, such as dynamic mapping relationships within the target area. Global target area features can include anatomical boundaries, metabolic hotspots, functional zones, and heterogeneous regions. Correspondingly, the distilled content of the global target area features can include: spatial features (such as the precise contours of tumor anatomy in CT images), metabolic features (such as the SUV value distribution in PET images, reflecting metabolically active regions), functional features (such as DWI (diffusion-weighted imaging) and T2-weighted images in MRI images, capturing functional zones within the tumor), and heterogeneous features (such as high-risk sub-regions within the tumor extracted by combining multiple modalities of medical image data).
[0232] The dynamic motion model (target area dynamic mapping relationship) can include: the mapping relationship between patient surface movements (such as respiratory movements) and internal organ displacements. Correspondingly, the distillation content of the dynamic motion model can include: motion feature mapping function (the dynamic mapping relationship between surface optical landmarks and internal target area displacements generated by the second target area localization model through training), breath-holding state characteristics (such as the target area position reference and motion pattern during breath-holding), and time series characteristics (such as dynamic displacement patterns within the respiratory cycle).
[0233] In one example, the second target localization model can also be used to: perform feature alignment on multi-modal medical image data, and perform knowledge distillation on the feature-aligned multi-modal medical image data.
[0234] Feature alignment refers to aligning multiple modal medical image data, such as CT, MRI, and PET, spatially and semantically to ensure feature consistency. Correspondingly, the distillation content of multiple modal medical image data can include registration parameters (spatial registration parameters between multiple modal medical image data, such as rotation, translation, and scaling matrices) and feature alignment weights (weight allocation strategies during cross-modal feature fusion (e.g., prioritizing anatomical structures in CT and metabolic hotspots in PET)).
[0235] In another example, the second target area localization model can also be used to: generate a target area feature model based on individualized data of the target object, and perform knowledge distillation on the target area feature model.
[0236] The distillation content of the target area feature model may include: individualized target area template (target area baseline model generated before treatment of the target object) and target area dynamic adjustment rules (parametric rules for adjusting the shape and position of the target area according to the current image data of the target object).
[0237] During treatment, the second target area localization model can use the target area feature template of the target object as a benchmark and dynamically adjust it in combination with the current image data of the target object to ensure that the model parameters of the first target area localization model are always consistent with the actual state of the target object.
[0238] In another example, the second target area localization model can also generate an irradiation scheme and perform knowledge distillation on the irradiation scheme.
[0239] The irradiation protocol may include dose distribution optimization strategies. For example, it may include an optimization scheme for the dose distribution in the target area to maximize therapeutic effect and protect normal tissue. Correspondingly, the distillation content of the irradiation protocol may include: a dose distribution template (e.g., a personalized dose distribution scheme generated based on tumor heterogeneity characteristics) and dose adjustment rules (adjusting the irradiation area in real time according to the dynamic changes of the target area).
[0240] S803. For the target object, during the localization phase, the remote device acquires multimodal medical image data of the target object, inputs it into the second target area localization model, outputs the target area dynamic mapping relationship of the target object, and sends the target area dynamic mapping relationship of the target object to the near-end device. Correspondingly, the near-end device receives the target area dynamic mapping relationship of the target object from the remote device.
[0241] The multimodal medical image data includes surface optical data, as well as at least one of X-ray projection images, CT images, MR images, PET images, PET-CT images, PET-MR images, SPECT images, SPECT-CT images, ultrasound US images, DSA images, and pathological slide images.
[0242] In this embodiment, the second target area localization model can also fuse surface optical data with other types of high-quality multimodal medical image data to locate the target area.
[0243] For example, the second target localization model can fuse body surface optical data with other types of high-quality image data in the following ways.
[0244] 3-1. Fusion based on marker points.
[0245] One method based on marker points is to place optical marker points (such as reflective spheres or texture patterns) on the patient's body surface and mark the positions of the optical marker points in the image data. This allows for the mapping relationship between the marker points in the body surface optical coordinate system and the image coordinates. This method is simple, intuitive, and suitable for static or regular motion scenarios.
[0246] 3-2. Surface profile-based fusion.
[0247] One method based on surface contour fusion involves acquiring the three-dimensional contours of the patient's body surface using an optical scanning device, registering them with the contours in the image data, and then using a point cloud configuration algorithm (such as the ICP algorithm) or a surface matching algorithm to achieve contour alignment. This method can capture more body surface information and is suitable for complex surface deformations.
[0248] 3-3. Fusion based on motion models.
[0249] One approach based on motion model fusion involves: constructing a patient's motion model (e.g., a respiratory motion model) using surface optical data; establishing a correlation model between organ position and surface motion using image data; and driving the motion model with surface optical data to predict the dynamic position of the target area. This method enables real-time dynamic tracking of the target area and is suitable for periodic movements such as respiratory motion.
[0250] 3-4. Fusion based on deep learning.
[0251] One approach based on deep learning involves using a deep learning model (such as a convolutional neural network or a recurrent neural network) to jointly model surface optical data and image data. The model is inputted with the time-series data of the surface optical data and the features of the image data to obtain the dynamic location of the tumor. This method can handle complex nonlinear relationships and adapt to various motion patterns. It also offers a high degree of automation, reducing human intervention.
[0252] In one example, in a scenario involving target localization, surface optical data can compensate for some deficiencies in medical imaging data. Specifically, surface optical data offers the following advantages:
[0253] 4-1. Improve the dynamic accuracy of target area localization.
[0254] There is usually a correlation between surface optical data and organs. By combining surface optical data, the position of the target area can be dynamically adjusted to avoid target displacement caused by organ movement. For example, in radiotherapy for lung tumors, the vertical movement of landmarks in surface optical data can reflect the longitudinal displacement of the lung tumor. Similarly, in radiotherapy for liver tumors, the lateral displacement of landmarks in surface optical data can reflect the range of motion of the liver.
[0255] In addition, in clinical applications, such as during radiotherapy, the stability of a patient's breath-holding state can be monitored in real time using surface optical data to ensure that irradiation is carried out within the breath-holding window, thereby improving the accuracy of target localization.
[0256] Furthermore, by using surface optical data, it is possible to monitor in real time whether the patient exhibits abnormal movements (such as sudden body movements or unstable breath-holding states), thereby triggering a pause in irradiation or an adaptive adjustment of the irradiation program when the patient exhibits abnormal movements.
[0257] 4-2. It has good temporal resolution.
[0258] Medical image data is typically acquired intermittently, while surface optical data is acquired continuously. Therefore, surface optical data can be acquired during the acquisition intervals of medical image data, thus compensating for the insufficient temporal resolution of medical images.
[0259] S804. Before irradiating the target area of the target object, the near-end device acquires the current surface optical data and current X-ray projection image of the target object.
[0260] S805, the near-end device inputs the current surface optical data and current X-ray projection image of the target object into the first target area localization model. The first target area localization model determines the position offset of the target area based on the dynamic mapping relationship of the target area.
[0261] The first target area localization model can utilize the distillation knowledge of the second target area localization model to quickly predict the dynamic position of the target area based on body surface optical data, and adjust the irradiation direction and dose distribution of the radiotherapy equipment according to the dynamic position of the target area.
[0262] The dynamic mapping relationship of the target area can be defined as the correspondence between the position of the target area of the target object and the image data as the target area moves.
[0263] S806: The proximal device determines the radiotherapy strategy based on the positional offset of the target area.
[0264] The radiotherapy strategy may include continuing to irradiate the target area according to the original irradiation protocol, adjusting the original irradiation protocol and using the adjusted irradiation protocol to irradiate the repositioned target area, or stopping the irradiation of the target area.
[0265] In one possible implementation, when the positional offset of the target area is within a preset threshold range, the first target area localization model is used to output first indication information to instruct the target area to be irradiated. That is, the first target area localization model can instruct the target area to continue irradiating according to the original irradiation plan.
[0266] In another possible implementation, when the positional offset of the target area is not within a preset threshold range, the first target area positioning model is used to output a second indication information for indicating adjustment of the target area or a third indication information for indicating cessation of irradiation.
[0267] When the second instruction information is output, the first target area localization model can also be used to adjust the original irradiation scheme and relocate the target area.
[0268] In some examples, the primary target localization model can determine whether the target's breath-holding state meets the irradiation requirements based on current surface optical data. If the irradiation requirements are not met, irradiation can be stopped or the irradiation plan adjusted.
[0269] It should be noted that the first target area localization model runs on a near-end device, which can directly use lightweight features of body surface optical data for rapid inference without the need for high computing power to process complex image data, thus improving data processing efficiency.
[0270] S807. After the radiotherapy to the target area is completed, the proximal device sends irradiation data back to the distal device. Correspondingly, the distal device receives irradiation data from the proximal device.
[0271] S808, the remote equipment updates the second target area localization model based on irradiation data.
[0272] based on Figure 8 The proposed technical solution involves a remote device training on multimodal medical image data to construct a second target area localization model (large model). This large model is then subjected to knowledge distillation to obtain a lightweight first target area localization model (small model). This lightweight small model allows the near-end device to perform real-time target area localization. In other words, in this embodiment, the coordinated operation of the first and second target area localization models achieves precise target area localization. Furthermore, in dynamic treatment scenarios, real-time target area localization is possible with high reliability.
[0273] It should be noted that, in the embodiments of this application, image data includes clinical data and laboratory data. Clinical data may include surface optical data (such as landmark locations, body surface contours) and image data collected from patients. Laboratory data may include data from simulated human models or healthy individuals collected in a laboratory environment using motion capture devices (such as optical tracking systems) and image acquisition devices (such as CT / MRI scanners).
[0274] In this embodiment of the application, in order to ensure the validity and reliability of the data, the following requirements and principles need to be met.
[0275] 5-1. Data authenticity.
[0276] Data authenticity can be categorized into physical consistency and statistical consistency. Physical consistency means that the generated data should conform to the laws of human anatomy and physiology. For example, the movement of the body surface caused by respiration and the movement of internal organs should have a reasonable time delay and amplitude relationship, and changes in tumor location should be consistent with the respiratory cycle. Statistical consistency means that the distribution of the generated data should be consistent with the statistical distribution of real data. For example, the range of motion and frequency distribution of landmarks should match those of real patients, and the shape and location distribution of tumors in image data should be consistent with real cases.
[0277] 5-2. Data Diversity: The generated data should cover as many scenarios and individual differences as possible.
[0278] The data may include: different patient body types, breathing patterns, tumor location and size; different landmark arrangements or body surface contours; and different image acquisition conditions (such as resolution and noise level).
[0279] In one example, data diversity can be achieved by adjusting the parameters of the generative model (such as the random seed of a GAN) or by introducing random noise or perturbations.
[0280] 5-3. Accuracy of data labeling.
[0281] The accuracy of data standards refers to the requirement that the data must have accurate standards, especially the true location of the tumor. Data annotation methods can include: directly recording the true values of the generated data in the physical model; and ensuring accurate annotation through post-processing (such as expert verification) in deep learning-generated data.
[0282] 5-4. Data quality verification.
[0283] The aforementioned data needs to undergo rigorous quality verification to ensure its effective use in model training. Verification methods may include: comparing the generated data with real data to assess its authenticity (e.g., distribution similarity), training the model using the generated data, and testing its performance on real data.
[0284] 5-5. Adhere to ethical and privacy guidelines.
[0285] If the data is based on real patient data, anonymization should be ensured to avoid disclosing patient privacy. The data generation process should comply with ethical requirements to prevent the misuse of patient data.
[0286] In some embodiments of this application, the near-end device can also indirectly infer changes in the target area based on real-time monitoring of the target object's surface movement. Specifically, this may include:
[0287] Step 1: Obtain the location of the target area and surface optical data of the target object under different respiratory phases.
[0288] In one example, before irradiating the target area, 4D-CT or 4D-MRI scans can be used to obtain the location and surface optical data of the target area of the target object at different respiratory phases.
[0289] Step 2: Based on the position of the target area and surface optical data of the target object under different respiratory phases, establish a motion model of the target area.
[0290] The motion model can be a correlation model between individualized surface optical data of the target object and positional changes in the target area. For example, the motion model can be a regression analysis-based model or a machine learning-based model.
[0291] In one example, linear regression, multinomial regression, or other statistical methods can be used to fit the relationship between body surface movement (such as the location of markers on the chest and abdomen) and the positional changes of the target area. Based on this relationship, a motion model can be constructed.
[0292] In another example, machine learning algorithms such as neural networks and support vector machines can be used to train a correlation model between body surface movement and target area position changes.
[0293] In this embodiment, artificial intelligence technology can also be used to optimize the correlation model between surface optical data and the positional changes of the target area. For example, artificial intelligence technology may include deep learning, transfer learning, etc.
[0294] Deep learning can include convolutional neural networks, long short-term memory networks, etc. Transfer learning refers to using transfer learning techniques to quickly adapt new patient data to existing patient data.
[0295] In some embodiments, when an irradiation of a target area is imminent, a remote device can use X-ray projection images to acquire the current position of the target area and, based on a body surface motion model, obtain the predicted position of the target area. If there is a deviation between the current position and the predicted position of the target area, the parameters of the second target area localization model can be optimized.
[0296] Furthermore, in this embodiment, the target area localization system may also include devices such as a breathing belt and a pressure sensor, which can be used to acquire the patient's respiratory signals and other physiological parameters (such as heart rate). The target area localization model can determine the actual location of the target area based on the patient's respiratory signals, physiological parameters, and body surface motion data.
[0297] In one possible implementation, the target localization system can use respiratory signals as input variables and combine them with body surface motion data to construct a multimodal motion prediction model. Alternatively, time series analysis methods (such as LSTM neural networks) can be used to construct the dynamic relationship between respiratory signals and changes in the position of the target area.
[0298] In this embodiment, by performing surface optical monitoring and target area location prediction on the target object, the second target area positioning model can be dynamically adjusted to facilitate subsequent accurate adjustment of the irradiation direction and dose distribution, so as to ensure that the target area is always within the optimal treatment target area.
[0299] In some other embodiments, the method provided in this application, the second target area localization model, can also be used to: obtain the dynamic position of the target area based on the surface optical data and image data of the target object.
[0300] In one example, such as Figure 9 As shown, the time series of optical data and features of image data of the target object's surface can be input into the second target area localization model to obtain the dynamic position of the target area.
[0301] The time series of body surface optical data can include: the three-dimensional coordinates of landmarks (e.g., [x,y,z]) or point clouds of body surface contours. Image data can include static image data (e.g., CT images, MRI images) and dynamic image data (e.g., 4D-CT data), anatomical information of tumors (e.g., location, shape) or functional information (e.g., metabolic activity).
[0302] The time series data format for body surface optical data can be (T, N, S). T represents the time step (frame number), N represents the number of surface landmarks or point clouds, and S represents the three-dimensional coordinates. The data format for image data features is typically a three-dimensional voxel or a two-dimensional slice. For example, the data format can be (C, H, W, D). C represents the number of image channels (e.g., CT has 1 grayscale channel), and H, W, and D represent the image height, width, and depth, respectively. The dynamic position format for the target region is (T, S1). S1 represents the position of the target region at each time step.
[0303] In one possible implementation, combining Figure 9 ,like Figure 10 As shown, the second target area localization model may include multiple modules, which may include an optical data processing module, an image data processing module, a feature fusion module, and a location prediction module.
[0304] The optical data processing module can be used to process optical time series data using recurrent neural networks (RNNs), long short-term memory networks (LSTMs), or gated recurrent units (GRUs). The output of the optical data processing module is the motion characteristics of the time series, in (TF) format, where F is the feature dimension.
[0305] The image data processing module can be used to extract image features using convolutional neural networks (CNNs) or 3D-CNNs. It can also use pre-trained models (such as ResNet and UNet) to extract features from images. The output of the image data processing module is image features. The output data format is (F_I), where F_I is the dimension of the image features.
[0306] The feature fusion module can be used to fuse motion features and image features from body surface optical data. For example, motion features and image features can be directly concatenated into a single vector. Alternatively, attention mechanisms (such as Transformer) can be used to dynamically adjust the weights of optical and image features. Furthermore, deep cross networks (DCNs) or other interactive mechanisms can be used to model the non-linear relationship between the two types of features.
[0307] The location prediction module can be used with fully connected layers (MLP) or regression layers to output the dynamic location of the tumor. The output data format is (T,3), where T represents the tumor location ([x,y,z]) at each time step.
[0308] The foregoing mainly describes the solutions of the embodiments of this application from a methodological perspective. It is understood that, in order to achieve the above functions, the near-end device includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.
[0309] This application embodiment can divide the near-end device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.
[0310] like Figure 11 As shown, this application provides a target area localization device, which can be applied to a near-end device or is a near-end device. The target area localization device includes: an acquisition unit 1101 and a processing unit 1102.
[0311] The acquisition unit 1101 is used to acquire the current image data of the target object before irradiation of the target area of the target object. The processing unit 1102 is used to input the current image data into a first target area localization model, which is used to determine the position offset of the target area based on the dynamic mapping relationship of the target area; the dynamic mapping relationship of the target area is the correspondence between the position of the target area of the target object and the image data as the target area moves.
[0312] In some embodiments, the current image data includes: body surface optical data and / or X-ray projection images.
[0313] In some embodiments, the acquisition unit 1101 is further configured to: acquire the dynamic mapping relationship of the target area; the dynamic mapping relationship of the target area is determined by the second target area localization model based on the multimodal medical image data acquired by the target object during the localization phase.
[0314] In some embodiments, the second target area localization model is further used to: determine the target area based on multiple modal medical image data of the target object before irradiating the target area of the target object; and generate an irradiation plan for the target area based on the target area.
[0315] In some embodiments, the multimodal medical image data includes body surface data, and at least one of the following: X-ray projection images, CT images, MR, PET images, PET-CT images, PET-MR images, SPECT images, SPECT-CT images, US images, DSA images, and pathological slide images.
[0316] In some embodiments, the first target area localization model is obtained by knowledge distillation of the second target area localization model.
[0317] In some embodiments, knowledge distillation includes one or more of the following: medical image data distillation, model parameter distillation, and irradiation strategy distillation.
[0318] In some embodiments, the processing unit 1102 is further configured to: after the irradiation of the target area is completed, feed back the irradiation data to the second target area localization model so that the second target area localization model is updated based on the irradiation data.
[0319] In some embodiments, irradiation data may include at least one of the following: abnormal information during the irradiation process, adjustment information of the irradiation strategy during the irradiation process, and the correspondence between the positional offset of the target area and the irradiation dose during the irradiation process.
[0320] In some embodiments, the processing unit 1102 is further configured to: update the first target area localization model and the second target area localization model based on federated learning.
[0321] In some embodiments, the first target area localization model is further configured to: output first indication information for indicating that the target area be irradiated when the positional offset of the target area is within a preset threshold range.
[0322] In some embodiments, the first target area localization model is further configured to: output second indication information for adjusting the position of the target area when the position offset of the target area is not within a preset threshold range.
[0323] In some embodiments, the processing unit 1102 is further configured to: control the radiotherapy device to irradiate the target area when the time interval between the acquisition time of the current image data and the output time of the first indication information is less than a preset time interval.
[0324] In some embodiments, the acquisition unit 1101 is further configured to: acquire a radiotherapy guidance image of the target object during the process of acquiring the current image data of the target area; the processing unit 1102 is further configured to input the current image data and the radiotherapy guidance image into the first target area localization model to determine the position offset of the target area.
[0325] In some embodiments, the processing unit 1102 is further configured to: determine the state of the target object based on the current image data of the target area; wherein the state of the target object includes an abnormal motion state and / or a breath-holding state; the above-mentioned inputting the current image data into the first target area localization model includes: inputting the current image data into the first target area localization model when the state of the target object meets a preset state.
[0326] In some embodiments, the processing unit 1102 is further configured to: irradiate the target target area when the target object is in a breathless state and has no abnormal movement.
[0327] In some embodiments, the target region is a tumor biological target region.
[0328] In some embodiments, the method of this application can be applied to each fraction of a multifractionated radiotherapy.
[0329] According to an embodiment of this application, this application also provides an electronic device, including at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the target area localization method provided in this application.
[0330] According to embodiments of this application, this application also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause an electronic device to execute the target area localization method provided in this application.
[0331] According to an embodiment of this application, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the target area localization method provided in this application.
[0332] Figure 12A schematic block diagram of an example electronic device 1200 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the application described and / or claimed herein. In some embodiments, the electronic device may be as described above. Figure 1 The near-end device shown.
[0333] like Figure 12 As shown, the electronic device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1202 or a computer program loaded from a storage unit 1208 into a random access memory (RAM) 1203. The RAM 1203 may also store various programs and data required for the operation of the electronic device 1200. The computing unit 1201, ROM 1202, and RAM 1203 are interconnected via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.
[0334] Multiple components in electronic device 1200 are connected to input / output interface 1205, including: input unit 1206, such as keyboard, mouse, etc.; output unit 1207, such as various types of monitors, speakers, etc.; storage unit 1208, such as disk, optical disk, etc.; and communication unit 1209, such as network card, modem, wireless transceiver, etc. Communication unit 1209 allows electronic device 1200 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0335] The computing unit 1201 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a central processing unit, a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors, and any suitable processor, controller, microcontroller, etc. The computing unit 1201 performs the various methods and processes described above, such as the data matching method. For example, in one embodiment, the data matching method can be implemented as a computer software program tangibly included in a machine-readable medium, such as storage unit 1208. In one embodiment, part or all of the computer program can be loaded and / or installed on the electronic device 1200 via ROM 1202 and / or communication unit 1209. When the computer program is loaded into RAM 1203 and executed by the computing unit 1201, one or more steps of the data matching method described above can be performed. Alternatively, in other embodiments, the computing unit 1201 may be configured to perform a data matching method by any other suitable means (e.g., by means of firmware).
[0336] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays, application-specific integrated circuits (ASICs), application-specific standard parts (ASSPs), systems-on-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0337] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0338] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0339] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user, such as a cathode ray tube (CRT) or liquid crystal display (LCD) monitor; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0340] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.
[0341] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.
[0342] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.
[0343] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. A method of targeting a region, characterized by, include: Just before irradiating the target area of the target object, acquire the current image data of the target object; The current image data is input into the first target area localization model, which is used to determine the position offset of the target area based on the dynamic mapping relationship of the target area. The dynamic mapping relationship of the target area is: the correspondence between the position of the target area of the target object and the image data as the target area moves.
2. The method of claim 1, wherein, The current image data includes: surface optical data and / or X-ray projection images.
3. The method of claim 1, wherein, The method further includes: The dynamic mapping relationship of the target area is obtained; the dynamic mapping relationship of the target area is determined by the second target area localization model based on the multimodal medical image data of the target object acquired during the localization phase.
4. The method of claim 3, wherein, The second target area localization model is also used for: Before irradiating the target area of the target object, the target area is determined based on multiple modal medical image data of the target object; Based on the target area, an irradiation scheme is generated for the target area.
5. The method according to claim 3 or 4, characterized in that, The multimodal medical image data includes body surface optical data, and at least one of the following: X-ray projection images, computed tomography (CT) images, magnetic resonance imaging (MR) images, positron emission tomography (PET) images, PET-CT images, PET-MR images, single-photon emission tomography (SPECT) images, SPECT-CT images, ultrasound (US) images, digital subtraction angiography (DSA) images, and pathological slide images.
6. The method of claim 3, wherein, The first target area localization model is obtained by knowledge distillation of the second target area localization model.
7. The method of claim 6, wherein, The knowledge distillation includes one or more of the following: Medical image data distillation; Model parameter distillation; Irradiation strategy distillation.
8. The method of claim 3, wherein, The method further includes: After the irradiation of the target area is completed, the irradiation data is fed back to the second target area localization model so that the second target area localization model can be updated based on the irradiation data.
9. The method of claim 8, wherein, The irradiation data includes at least one of the following: Abnormal information during the irradiation process; Information on the adjustment of the irradiation strategy during the irradiation process; The relationship between the positional offset of the target area and the irradiation dose during the irradiation process.
10. The method of claim 3, wherein, The method further includes: Based on federated learning, the first target area localization model and the second target area localization model are updated.
11. The method of claim 1, wherein, The first target area localization model is also used for: If the positional offset of the target area is within a preset threshold range, a first indication message is output to indicate that the target area should be irradiated.
12. The method of claim 11, wherein, The first target area localization model is also used for: If the position offset of the target area is not within the preset threshold range, a second indication message is output to indicate the adjustment of the target area position.
13. The method according to claim 11 or 12, characterized in that, The method further includes: If the time interval between the acquisition time of the current image data and the output time of the first indication information is less than a preset time interval, the radiotherapy device is controlled to irradiate the target area.
14. The method of claim 1, wherein, The method further includes: During the acquisition of the current image data, a radiotherapy-guided image of the target object is acquired; The current image data and the radiotherapy-guided image are input into the first target area localization model to determine the positional offset of the target area.
15. The method of claim 1, wherein, The method further includes: Based on the current image data, the state of the target object is determined, and the state of the target object includes abnormal motion state and / or being in a breath-holding state; The step of inputting the current image data into the first target area localization model includes: When the state of the target object meets the preset state, the current image data is input into the first target area localization model.
16. The method of claim 1, wherein, The method further includes: Irradiate the target area when the target object is in a breathless state and does not move abnormally.
17. The method of any one of claims 1-4, wherein, The target region is a tumor biological target region.
18. The method of any one of claims 1-4, wherein, The target localization method is applied to each fraction of radiotherapy in multi-fraction radiotherapy.
19. A target area positioning system, comprising: include: Radiotherapy equipment, image acquisition equipment, proximal end equipment; The radiotherapy device is used to irradiate the target area; the image acquisition device is used to acquire image data of the target object; the proximal device is configured with a first target area localization model and is used to perform the method as described in any one of claims 1-18.
20. The system of claim 19, wherein, The target area positioning system also includes remote equipment; The remote device is equipped with a second target area localization model.
21. The system of claim 19 or 20, wherein, The target area positioning system further includes a positioning device, which is used to simulate the positioning of the target object before the target area of the target object is irradiated.
22. An electronic device, comprising: The electronic device includes: processor; A memory configured to store processor-executable instructions; The processor is configured to execute the instructions to implement the method as described in any one of claims 1-18.
