Image processing-based planning method and device for laser photocoagulation operation of fundus

Abstract

This application discloses a method and apparatus for planning fundus laser photocoagulation surgery based on image processing. The method includes: guiding feature extraction of the target fundus image based on a first mask to obtain target image features; obtaining a target mask based on the first mask, target prompt features from human-computer interaction commands, and target image features; determining a safe target area mask from the target mask; constructing a blood vessel restricted area based on a blood vessel mask of the target fundus image; obtaining a final surgical spot array based on the safe target area mask, the first mask, and preset spot layout parameters, combined with the blood vessel restricted area; and generating a standardized surgical plan document based on the final surgical spot array and the safe target area mask. This method can solve the problems of insufficient accuracy in safe target area segmentation, the need for extensive manual adjustment of the spot array, and decreased treatment coverage or uneven array caused by local blood vessel avoidance, thus lowering the operational threshold for preoperative planning of fundus laser photocoagulation surgery.

Description

Technical Field

[0001] This application relates to the fields of medical image processing and automatic planning of fundus laser photocoagulation surgery, and in particular to a method and apparatus for planning fundus laser photocoagulation surgery based on image processing. Background Technology

[0002] Retinal laser photocoagulation is an important method for treating fundus diseases such as diabetic retinopathy. Its core lies in accurately identifying the lesion area, anatomically protected areas, and the course of major blood vessels preoperatively, and generating an executable laser array while meeting clinical safety requirements, treatment coverage, and array uniformity. Current preoperative planning methods mainly include:

[0003] (1) Manual drawing: This method relies entirely on the doctor to manually draw the treatment area and prohibited area on the fundus image and place light spots point by point or area by area. This method is time-consuming and is significantly affected by the doctor's experience, image quality and operating habits, which is not conducive to forming a stable and reproducible surgical plan.

[0004] (2) Traditional computer vision-assisted segmentation: algorithms such as threshold segmentation, region growing, edge extraction, and morphological processing are used to extract the contours of lesions, optic discs, blood vessels, or other structures. Due to the common problems of uneven illumination, low contrast, hemorrhage, or exudate occlusion in fundus images, traditional algorithms have limited generalization ability to the differences in imaging from different devices and the changes in lesion morphology, requiring repeated manual parameter tuning.

[0005] (3) Simple deep learning segmentation or local spot avoidance: Existing deep learning segmentation methods can improve the ability to identify lesion areas, but without physician interaction correction and explicit surgical constraints, the macula center, optic disc, previously photocoagulated areas, or other unsuitable areas may still be mistakenly included in the treatment area. Some methods only delete spots that collide with blood vessels after spot generation, which is a local filtering strategy. This can easily lead to local treatment gaps, a significant decrease in the number of spots, or uneven spacing between adjacent spots, and cannot simultaneously take into account vascular safety, treatment coverage, and array uniformity.

[0006] Therefore, how to integrate automatic semantic segmentation, human-computer interaction correction, spot geometry arrangement based on FAZ (Foveal Avascular Zone) or the fovea region, constraints on non-vascular prohibited spot arrangement areas, and strict global vascular avoidance into the same preoperative planning process, while ensuring the accuracy of the safe target area and generating a spot array that meets the requirements of prohibited area exclusion and vascular avoidance and has uniform coverage, so as to reduce the operational threshold of preoperative planning for fundus laser photocoagulation surgery, is a technical problem that needs to be solved in this field. Summary of the Invention

[0007] This application provides a method and device for planning fundus laser photocoagulation surgery based on image processing. The purpose is to solve the problems of insufficient accuracy of safe target area segmentation, the need for a lot of manual adjustment of the spot array, and the reduction of treatment coverage or uneven array caused by local blood vessel avoidance in the preoperative planning of fundus laser photocoagulation surgery, so as to reduce the operation threshold of preoperative planning of fundus laser photocoagulation surgery.

[0008] To achieve the above objectives, this application provides the following technical solution:

[0009] A method for planning fundus laser photocoagulation surgery based on image processing includes:

[0010] Based on a pre-constructed pre-segmentation model, semantic segmentation information of the target fundus image is obtained; the semantic segmentation information includes a first mask and a second mask; the first mask is a mask for prohibited spot arrangement regions; the second mask is a mask for surgically appropriate regions.

[0011] Based on the first mask, the target fundus image is guided to perform feature extraction in order to obtain target image features;

[0012] Obtain the target prompt features of the human-computer interaction prompts;

[0013] Based on a pre-built initial mask generation model or iterative mask generation model, a mask is generated for the first mask, the target cue features, and the target image features to obtain a target mask;

[0014] Based on the first mask and the second mask, determine the safe target area mask from the target mask;

[0015] Obtain the vascular mask of the target fundus image, and construct a vascular no-entry region based on the vascular mask and the surgical spot radius;

[0016] Based on the safety target area mask, the first mask, the blood vessel restricted area and the preset spot layout parameters, a multi-layer candidate spot array is generated, and global strict blood vessel avoidance optimization is performed in the multi-layer candidate spot array to obtain the final surgical spot array.

[0017] Based on the final surgical spot array and the safe target area mask, a standardized surgical plan file that can be called by a medical terminal is generated according to a preset data protocol.

[0018] An image processing-based fundus laser photocoagulation surgery planning device includes:

[0019] A semantic segmentation unit is used to obtain semantic segmentation information of a target fundus image based on a pre-built pre-segmentation model; the semantic segmentation information includes a first mask and a second mask, wherein the first mask is a mask for prohibited light spot arrangement areas; and the second mask is a mask for surgically appropriate areas.

[0020] An image encoding unit is used to guide the target fundus image to perform feature extraction based on the first mask, so as to obtain target image features;

[0021] A prompting encoding unit is used to obtain the target prompting features of human-computer interaction prompting instructions;

[0022] The mask generation unit is used to generate a mask for the first mask, the target cue feature and the target image feature based on a pre-built initial mask generation model or an iterative mask generation model, so as to obtain a target mask;

[0023] A mask filtering unit is used to determine a security target area mask from the target mask based on the first mask and the second mask;

[0024] The vessel segmentation unit is used to obtain the vessel mask of the target fundus image and construct a vessel restricted area based on the vessel mask and the surgical spot radius.

[0025] The spot planning unit is used to generate a multi-layer candidate spot array based on the safety target area mask, the first mask, the blood vessel restricted area and preset spot layout parameters, and to perform global strict blood vessel avoidance optimization in the multi-layer candidate spot array to obtain the final surgical spot array.

[0026] The file generation unit is used to generate a standardized surgical plan file that can be called by a medical terminal based on the final surgical spot array and the safe target area mask, according to a preset data protocol.

[0027] A storage medium comprising a stored program, wherein the program is executed by a processor to perform the image processing-based fundus laser photocoagulation surgical planning method.

[0028] In the technical solution provided in this application, the first mask serves as a mask for prohibited spot arrangement regions, participating in target mask screening, safe target area determination, and spot center constraint. This ensures that treatment spots cannot be planned in anatomically protected areas such as the macula or avascular areas of the fovea, as well as at least one of the following: the optic disc area, areas of previously accumulated laser photocoagulation marks, and user-marked prohibited areas. Simultaneously, the model prior role of the first mask in applying spatial attention masking to the feature extraction process is retained, and when arrangement based on the anatomical center is required, only the FAZ or foveal region in the first mask is used to determine the array's geometric center. Vascular regions are not incorporated into the first mask; instead, independent vascular masks are obtained through a vascular segmentation model, and vascular prohibited areas are constructed according to the spot radius. Global strict vascular avoidance optimization is performed among multiple candidate spot arrays, thereby reducing the risk of major vascular coverage, local treatment voids, and uneven array spacing while ensuring that non-vascular prohibited planning areas are not covered. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 A schematic diagram of the overall process of a fundus laser photocoagulation surgery planning method based on image processing provided in this application embodiment;

[0031] Figure 2 This is a schematic diagram of a semantic segmentation information generation process provided in an embodiment of this application;

[0032] Figure 3 A schematic diagram of a target mask generation process based on automatic prompts and human-computer interaction prompts is provided for an embodiment of this application;

[0033] Figure 4 A schematic diagram illustrating a process for determining a security target area mask based on a first mask and a second mask, provided for an embodiment of this application;

[0034] Figure 5 A schematic diagram illustrating the process of generating a final surgical spot array based on a security target mask, a first mask, and a blood vessel restricted area, provided for an embodiment of this application;

[0035] Figure 6 A schematic diagram of the architecture of an image processing-based fundus laser photocoagulation surgery planning device provided in this application embodiment;

[0036] Figure 7A visualization of semantic segmentation information, a secure target mask, a blood vessel restricted area, and a surgical spot array provided in an embodiment of this application;

[0037] Figure 8 This is a schematic diagram of the interface effect of a human-computer interaction correction and surgical plan confirmation process provided in an embodiment of this application. Detailed Implementation

[0038] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0039] In this application, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0040] like Figure 1 The diagram shown is a schematic overall flow chart of a fundus laser photocoagulation surgery planning method based on image processing provided in an embodiment of this application, which includes the following steps.

[0041] S101: Obtain semantic segmentation information of the target fundus image based on a pre-built pre-segmentation model.

[0042] The target fundus image can be a color fundus photograph, a fundus angiography image, or other medical images that can be used for preoperative planning of fundus laser photocoagulation. Semantic segmentation information includes a first mask and a second mask; the first mask is a mask for prohibited spot placement areas, used to characterize non-vascular surgical avoidance areas where treatment spots should not be placed; the second mask is a mask for suitable surgical areas, used to characterize candidate treatment areas that allow the generation of the target mask and allow the placement of treatment spots.

[0043] In some examples, the prohibited spot arrangement area represented by the first mask includes at least anatomically protected areas such as the macula or the avascular area of ​​the fovea; in some embodiments, it may also include at least one of the optic disc area, previously accumulated laser photocoagulation marks, and user-marked prohibited areas. Vascular regions are not incorporated into the first mask, but rather participate in global strict vascular avoidance through a separate vascular mask and vascular prohibited areas.

[0044] It should be noted that the anatomical center mask can be used as a specific implementation of the first mask to mark anatomically protected areas such as the fovea or FAZ where spot arrangement is prohibited. When it is necessary to generate a concentric ring candidate spot array in the future, the center point of the spot arrangement is calculated only based on the FAZ or fovea region in the first mask, and does not depend on the optic disc region, the previously old laser photocoagulation spot region or the user-marked prohibited area in the first mask.

[0045] In some examples, the second mask includes areas where lesions exist and are suitable for treatment, such as non-perfusion areas, retinal neovascularization, treatment areas surrounding microaneurysms, or other areas to be treated as identified by the physician. The second mask is used to constrain the target mask and the center of the treatment spot from exceeding the appropriate surgical area.

[0046] In practical implementation, the target fundus image can be input into a pre-segmentation model to obtain a semantic label map output by the pre-segmentation model. At least one morphological post-processing step, including boundary smoothing, connected component filtering, opening operation, hole filling, area-preserving re-thresholding, or small connected component removal, is then performed on the semantic label map. A first mask and a second mask are then obtained based on the post-processed semantic label map. For example, in engineering implementation, the pre-segmentation model can be deployed using an ONNX (Open Neural Network Exchange) model. After outputting the label map, a first mask corresponding to the FAZ or fovea region and a second mask corresponding to the surgically suitable region are generated. In other implementations, the first mask can also be obtained from model output, user-labeled data, or a combination of both, to obtain non-vascular prohibited spot arrangement regions such as the optic disc region, previously old laser photocoagulation spot regions, and user-labeled prohibited areas.

[0047] Optionally, the process of obtaining semantic segmentation information of the target fundus image based on a pre-built pre-segmentation model can be found in [reference needed]. Figure 2 The steps are shown.

[0048] S102: Based on the first mask, guide the target fundus image to perform feature extraction in order to obtain the target image features.

[0049] The primary purpose of the first mask is to serve as a prohibition planning constraint for subsequent surgical spot planning. During the feature extraction stage, the first mask can also act as a spatial prior input image encoder to apply spatial attention masking to the feature extraction process, thereby suppressing feature extraction of surgical avoidance areas and reducing the probability of subsequent target masks mistakenly covering prohibited spot arrangement areas.

[0050] In some examples, the so-called spatial attention masking refers to reducing or setting the feature weights of corresponding spatial locations to zero based on the prohibited regions identified by the first mask when the image encoder performs convolution or self-attention calculations.

[0051] In one possible implementation, a suppression weight map can be generated based on the first mask, then downsampled to the same spatial size as the feature map, and the downsampled mask can be used as an attention map and multiplied element-wise with the feature map, so that the feature response located in the prohibited spot arrangement area is greatly attenuated, while the feature response located in the surgical appropriate area is preserved.

[0052] Optionally, the process of guiding feature extraction of the target fundus image based on the first mask to obtain the target image features can be as follows: inputting the first mask and the target fundus image into a pre-constructed image encoder to obtain the target image features output by the image encoder; the image encoder is used to extract features from the target fundus image, and the first mask is used as spatial prior information of non-vascular prohibited spot arrangement areas to apply spatial attention masking to the feature extraction process to suppress feature extraction of surgical avoidance areas.

[0053] S103: Obtain the target prompt feature of the human-computer interaction prompt command.

[0054] Human-computer interaction prompts refer to prompt signals such as dots, boxes, and drawings input by users through a graphical user interface, used to indicate treatment areas to be included or excluded. These prompts can be input into a pre-built prompt encoder to obtain target prompt features.

[0055] In one possible implementation, the prompt encoder can encode the user-input prompt into a high-dimensional feature vector that matches the image feature dimension through positional encoding and learnable embedding vectors, so that the subsequent mask generation model can directly utilize it.

[0056] S104: Based on a pre-built initial mask generation model or iterative mask generation model, perform mask generation on the first mask, target cue features, and target image features to obtain the target mask.

[0057] The target mask refers to the segmentation result of the candidate treatment region predicted by the model under the current human-computer interaction prompt. This target mask needs to undergo post-processing and constraint filtering before it can serve as the basis for subsequent safe target region determination.

[0058] Optionally, the construction of the pre-segmentation model, the initial mask generation model, or the iterative mask generation model includes: training an anatomically protected region segmentation model with no or weak cue input for the avascular area or fovea region of the macula, and training an interactive segmentation network model with human-computer interaction cue input for the surgically suitable area; training the interactive segmentation network model based on training samples, using at least one of point cue, previous mask input, or no cue input as input conditions during training, and using at least one of mask loss, Dice loss, cross-union prediction loss, and target category loss as training constraints; exporting the trained interactive segmentation network model and the anatomically protected region segmentation model into an open neural network exchange model that can be called by a medical terminal; the open neural network exchange model includes at least a pre-segmentation model for outputting semantic segmentation information, an initial mask generation model for generating the initial target mask, and an iterative mask generation model for receiving the previous mask input and continuously correcting it.

[0059] It should be noted that Dice loss is a loss function that directly optimizes the overlap between the segmented region and the real mask, while Intersection over Union (IoU) prediction loss is a loss function that measures the ratio of the intersection to the union between the predicted mask and the real mask.

[0060] Optionally, target mask generation can be achieved using automatic prompts and human-computer interaction prompts. For details on this implementation process, please refer to [link to relevant documentation]. Figure 3 The steps are shown.

[0061] Optionally, the process of generating a target mask based on a pre-built initial mask generation model or an iterative mask generation model, involving the first mask, target prompt features, and target image features, can be as follows: During initial mask generation, the target image features and target prompt features are input into the pre-built initial mask generation model to obtain a first candidate mask and corresponding candidate logical values; during iterative mask generation, the candidate logical values ​​obtained from the previous mask generation are scaled to a preset mask input size and input together with newly added human-computer interaction prompts into the pre-built iterative mask generation model to obtain an updated candidate mask; the candidate mask is subjected to size restoration, small connected component removal, hole filling, and a second mask constraint to obtain the target mask; for candidate masks whose overlap ratio with existing candidate schemes exceeds a preset condition, whose area ratio does not meet the preset condition, or whose second mask constraint is not met, they are recorded as invalid schemes and excluded from the subsequent automatic prompt area.

[0062] In some examples, the initial cue point can be determined based on the maximum connected region of the second mask, and this initial cue point is input into the first mask generation model to obtain the first candidate mask and candidate logical values. The first mask generation model can output a high-resolution candidate mask and a low-resolution candidate logical value; the high-resolution candidate mask is used for display and post-processing, and the low-resolution candidate logical value can be used as the previous mask input for subsequent iterative mask generation models.

[0063] In some examples, if the coverage of the current target mask over the second mask is lower than a preset coverage threshold, a new auto-suggestion point can be generated based on the largest connected region in the second mask that is not yet covered by the current target mask, and candidate mask generation can be performed again. For candidate masks that overlap with existing candidate schemes by more than a preset condition, whose area ratio does not meet the preset condition, or that do not meet the constraints of the second mask, they can be marked as invalid schemes and excluded from the subsequent auto-suggestion area.

[0064] It should be noted that the preset coverage threshold and preset conditions can be set according to clinical accuracy requirements. For example, the coverage threshold can be set to 85% or 90%, and the overlap ratio threshold can be set to 70%.

[0065] In some examples, human-computer interaction prompts include, but are not limited to: positive prompts, negative prompts, and instructions to add or delete regions input by the doctor. Positive prompts represent areas the user wishes to include in treatment, negative prompts represent areas the user wishes to exclude, and instructions to add or delete regions are used to locally correct the safe target mask or candidate spot regions. The system inputs the automatically generated prompts and / or the user-input human-computer interaction prompts into the prompt encoder to obtain target prompt features, which are then used by the mask generation model to generate or update the target mask.

[0066] In some examples, the initial mask generation model and the iterative mask generation model can be derived from a model trained on an interactive segmentation network. Training data can be converted from fundus medical images with pixel-level annotations and their corresponding masks into NPZ samples, including an array of images and an array of ground truth masks. During training, at least one of point cues, previous mask input, or no-cue input can be used as input conditions, and at least one of mask loss, Dice loss, cross-union prediction loss, and target class loss can be used as training constraints. After training, the model can be exported as an ONNX inference model for deployment in medical terminals.

[0067] S105: Determine the safe target area mask from the target mask based on the first mask and the second mask.

[0068] The safe target area mask serves as a direct regional constraint for subsequent spot planning. Specifically, a Boolean difference operation is performed between the target mask and the first mask, and the difference result is constrained within the surgically appropriate region corresponding to the second mask to obtain the safe target area mask. The safe target area mask includes the region where the target mask exists, the second mask exists, and the first mask does not exist.

[0069] Optionally, the process of determining the safe target area mask from the target mask based on the first mask and the second mask can be as follows: updating the first mask or the second mask in response to user interaction behavior related to semantic segmentation information; the user interaction behavior includes user confirmation or fine-tuning of prohibited spot arrangement areas or surgically suitable areas; performing a Boolean difference operation between the target mask and the first mask; constraining the difference result to the surgically suitable area corresponding to the second mask to obtain the safe target area mask; the safe target area mask includes areas where the target mask exists, the second mask exists, and the first mask does not exist.

[0070] Through the above processing, the non-vascular prohibited spot arrangement area corresponding to the first mask is mathematically excluded from the safe target area mask, thereby preventing subsequent treatment spot planning from falling into anatomically protected areas such as the macula or avascular areas of the fovea, as well as at least one of the optic disc area, areas with previously accumulated laser photocoagulation marks, and user-marked prohibited areas. If the doctor confirms or fine-tunes the first or second mask, the system recalculates the safe target area mask based on the updated first or second mask.

[0071] Optionally, the process of determining the secure target area mask from the target mask based on the first mask and the second mask can be found in [reference needed]. Figure 4 The steps are shown.

[0072] S106: Obtain the vascular mask of the target fundus image, and construct the vascular restricted area based on the vascular mask and the surgical spot radius.

[0073] The vascular mask is used to identify the course of major blood vessels in the fundus. The vascular region is not included in the first mask, but is used as an independent mask to construct the vascular no-entry region, which is then strictly avoided during global optimization.

[0074] In some examples, the target fundus image can be input into the blood vessel segmentation model to obtain a blood vessel probability map; the blood vessel probability map is normalized and thresholded to obtain an initial blood vessel binary map; the initial blood vessel binary map is then processed by erosion, small connected component removal, dense noise region removal and spur removal to obtain a blood vessel mask.

[0075] It should be noted that the blood vessel segmentation model can be trained based on U-Net or other convolutional neural networks suitable for medical image segmentation, and its training data consists of fundus images with blood vessel annotations. Erosion operations can eliminate minor noise, and spur removal processing can remove spurious branches in the blood vessel mask.

[0076] In some examples, the vascular mask is expanded according to the radius of the surgical spot to obtain a vascular forbidden zone. Since the vascular forbidden zone has been expanded according to the radius of the treatment spot, the center of the spot is constrained from falling into the vascular forbidden zone, which can equivalently reduce the risk of the treatment spot covering the blood vessel.

[0077] In a possible implementation, the expansion operation may employ a circular structural element with a radius equal to the preset surgical spot radius.

[0078] Optionally, the process of obtaining the vascular mask of the target fundus image and constructing the vascular forbidden region based on the vascular mask and the surgical spot radius can be as follows: input the target fundus image into the vascular segmentation model to obtain a vascular probability map; normalize and threshold the vascular probability map to obtain an initial vascular binary map; perform erosion, small connected component removal, dense noise region removal, and spur removal on the initial vascular binary map to obtain the vascular mask; dilate the vascular mask according to the surgical spot radius to obtain the vascular forbidden region; in the global strict vascular avoidance optimization, the center of the spot must not fall into the vascular forbidden region.

[0079] It should be noted that global strict vessel avoidance optimization refers to an optimization strategy used in the planning of fundus laser photocoagulation surgery to generate the final surgical spot array. The core feature of this optimization strategy is that instead of first generating a complete spot array and then locally deleting spots that conflict with the vessel-prohibited areas (i.e., non-local filtering), during the generation of multi-layer candidate spot arrays, a global layout solution is obtained across candidate rings. Under the premise of satisfying the following constraints, the center position of the spot on each candidate ring is jointly selected to obtain the overall optimal spot array.

[0080] S107: Generate a multi-layer candidate spot array based on the safety target area mask, the first mask, the blood vessel restricted area and the preset spot layout parameters, and perform global strict blood vessel avoidance optimization in the multi-layer candidate spot array to obtain the final surgical spot array.

[0081] The generation of the spot array is not a simple equidistant arrangement within the ring, but rather, under the constraint of allowing a center mask, the optimal layout is jointly selected among multiple candidate rings through a global optimization strategy.

[0082] In some examples, when the first mask contains the FAZ or fovea region, the anatomical center coordinates are determined only based on the centroid of the largest connected region of the FAZ or fovea region. The radius step size and the minimum center distance between adjacent spots are determined based on the preset spot diameter and preset spot spacing. Multiple candidate rings are generated with the anatomical center coordinates as the center, according to the radius step size, and the original candidate spot center on each candidate ring is determined based on the minimum center distance between adjacent spots.

[0083] It should be noted that the radius step size is usually determined by both the spot diameter and the spot spacing to ensure that the spots between adjacent rings can also meet the spacing requirements in the radial direction.

[0084] In some examples, an allowable center mask is generated based on the safe target area mask, the first mask, and the prohibited blood vessel region, allowing the center of the light spot to fall within the mask. The allowable center mask excludes areas outside the safe target area, the prohibited light spot arrangement area corresponding to the first mask, and the prohibited blood vessel region. If it is also necessary to ensure that the entire light spot entity is within the safe target area, the feasible region of the safe target area mask can be shrunk according to the light spot radius, and then combined with the prohibited blood vessel region to form the allowable center mask.

[0085] In the globally rigorous vessel avoidance optimization, the system does not simply generate a complete array and then delete the vessel collision spots. Instead, it jointly selects the final layout among multiple candidate rings. Specifically, for each candidate ring, candidate centers are sampled within an allowed center mask, and a set of candidate layouts for that ring is constructed. Then, the final layout is selected among multiple candidate rings through a global layout solution across candidate rings. The global layout solution has the following optimization objectives: maximizing the number of final spots as the first optimization objective, minimizing the penalty for the center distance between adjacent spots as the second optimization objective, and minimizing the movement cost relative to the original candidate spot center as the third optimization objective.

[0086] It should be noted that global layout solving across candidate rings is a specific technical means to achieve strict global vessel avoidance optimization. Global layout solving across candidate rings refers to: generating multiple layers of candidate rings with the anatomical center coordinates as the center and according to the radius step size; sampling candidate centers within the allowed center mask for each candidate ring and constructing a candidate layout set; and then jointly selecting the final layout among the candidate layout sets of multiple candidate rings through cross-candidate ring combinatorial optimization (such as a combination of dynamic programming or greedy search and local adjustment methods) to satisfy the above constraints and optimization objectives. When the vessel forbidden area or the first mask causes local spot loss, local treatment voids, or cross-ring spacing conflicts, free candidate point search and local repair can also be performed in the corresponding ring zone or local sector to restore the number of available spots and maintain array uniformity as much as possible while maintaining strict vessel avoidance.

[0087] In a possible implementation, the global layout solution can be formalized as a constrained combinatorial optimization problem and solved using a combination of dynamic programming or greedy search and local adjustment.

[0088] In some examples, when the final layout has spot loss, local treatment voids or cross-ring spacing conflicts caused by blood vessel restricted areas or the first mask, free candidate point search and local repair can be performed in the corresponding ring or local sector to restore the number of available spots and keep the center distance between adjacent spots not less than a preset relaxation threshold.

[0089] It should be noted that the preset relaxation threshold refers to the lower limit of the minimum center-to-center distance between adjacent light spots that is acceptable without causing clinical risks.

[0090] In a possible implementation, a visualization of the first mask, the safe target area mask, the blood vessel restricted area, and the surgical spot array shown by the semantic segmentation information can be found in [reference needed]. Figure 7 As shown.

[0091] Optionally, based on the safety target mask, the first mask, the blood vessel no-entry region, and the preset spot layout parameters, a multi-layer candidate spot array is generated, and global strict blood vessel avoidance optimization is performed in the multi-layer candidate spot array to obtain the final surgical spot array. For the implementation process, please refer to [link to relevant documentation]. Figure 5 The steps are shown.

[0092] S108: Based on the final surgical spot array and the safety target mask, a standardized surgical plan file is generated according to a preset data protocol and can be called by medical terminals.

[0093] The standardized surgical protocol document is a structured data file used by laser photocoagulation therapy equipment for automatic reading and execution. The standardized surgical protocol document includes at least one of the following: target image size, coordinate system information, two-dimensional coordinates of the treatment spot, spot diameter, spot spacing, safe target area mask identifier, first mask identifier, blood vessel restricted area identifier, energy level, treatment sequence, and protocol version number.

[0094] In possible implementations, the file may be stored in a structured text format such as JSON or XML, or in a structured report format defined in medical digital imaging and communication standards.

[0095] In some examples, the system displays the final surgical spot array, safe target mask, first mask, and prohibited vascular area for the physician's confirmation. If the physician confirms that everything is correct, a standardized surgical plan document is exported; if the physician disagrees with the current results, the physician is allowed to re-enter interactive prompts or modify the prohibited spot arrangement area, and the safe target mask and final surgical spot array are regenerated.

[0096] Based on the scheme shown in the embodiments of this application, the following beneficial effects can be achieved: (1) The non-vascular prohibited spot arrangement area is explicitly represented by the first mask, and the area is continuously excluded during the safe target area determination and spot planning stages, reducing the risk of treatment spots mistakenly entering the macular area, the foveal avascular area, the optic disc area, the area of ​​previously old laser photocoagulation marks, or the user-marked prohibited area; (2) The safe target area is automatically generated and controllably corrected by combining automatic prompts and human-computer interaction prompts with the first mask generation model and the iterative mask generation model, thereby improving the efficiency of preoperative planning; (3) By using independent vascular masks, vascular prohibited area construction, and global strict vascular avoidance optimization, vascular safety, treatment coverage, and array uniformity are taken into account without simply relying on local spot deletion; (4) When the first mask contains the FAZ or the foveal region, the geometric center of the candidate spot array is determined only by using the centroid of the largest connected region of that region, making the spot arrangement more in line with the fundus anatomy and clinical planning habits.

[0097] like Figure 2 The diagram shown is a schematic of a semantic segmentation information generation process provided in an embodiment of this application, including the following steps.

[0098] S201: Obtain the preprocessed target fundus image.

[0099] The target fundus image can be a color fundus photograph, a fundus angiography image, or other fundus medical images. Preprocessing methods include at least one of image size normalization, brightness or contrast enhancement, noise reduction, and channel format conversion.

[0100] In some examples, image size normalization refers to scaling the input image to a standard size required by the pre-segmentation model, such as 1024x1024 pixels. For example, bilinear or bicubic interpolation can be used for scaling to preserve image detail. Brightness or contrast enhancement can be achieved through histogram equalization or CLAHE (Contrast Limited Adaptive Histogram Equalization) to improve image quality under uneven lighting conditions. Denoising can be achieved using methods such as Gaussian filtering or median filtering to reduce random noise during image acquisition. Channel format conversion is used to unify images acquired from different devices in RGB, BGR, or other color spaces into the channel format used during model training.

[0101] S202: Input the target fundus image into the pre-built pre-segmentation model to obtain the semantic label map output by the pre-segmentation model.

[0102] The semantic label map is a pixel-level classification result map corresponding to the spatial size of the input image. Each pixel in the map is assigned a semantic category label to distinguish different anatomical structures and lesion regions. In the specific implementation, the semantic label map contains at least the necessary category information for generating the first mask and the second mask.

[0103] In some examples, the pre-segmentation model can be a deep learning model trained on fundus medical images and their pixel-level annotations. Its network structure can adopt an encoder-decoder architecture, such as convolutional neural networks based on U-Net, DeepLabV3+, etc., which are suitable for semantic segmentation of medical images. The encoder is responsible for extracting multi-scale visual features from the input image, while the decoder gradually restores these features to the original image resolution and outputs the class probability. Finally, the semantic label map is obtained by taking the maximum value pixel by pixel.

[0104] In some examples, the pre-segmentation model, after training, can be exported as an ONNX inference model for deployment on medical terminals. The semantic label map output by the pre-segmentation model is used to generate at least the first and second masks. Inference using the ONNX model ensures efficient forward inference and cross-platform compatibility without relying on a specific deep learning framework.

[0105] In one possible implementation, the pre-segmentation model receives a preprocessed target fundus image as input, performs one forward inference, and outputs a single-channel semantic label map. Different values ​​in the map represent different structures or regions such as the background, macular region, avascular region of the fovea, optic disc region, region of previous photocoagulation, and suitable surgical region.

[0106] S203: Perform morphological post-processing on the semantic label graph.

[0107] Morphological post-processing includes at least one of boundary smoothing, connected component filtering, opening operation, hole filling, area preservation re-thresholding, or small connected component removal, in order to reduce the impact of noisy regions, isolated mis-segmented regions, and boundary spikes on subsequent planning.

[0108] It should be noted that opening operation refers to an operation that involves erosion followed by dilation, which can effectively remove small noise points and smooth region boundaries. Hole filling is used to fill holes inside segmented regions, making the mask continuous and complete. Small connected component removal refers to deleting isolated regions with pixel areas smaller than a preset threshold, which can be set according to the clinically acceptable minimum lesion size.

[0109] S204: Obtain the first mask and the second mask based on the semantic label map after morphological post-processing.

[0110] The first mask is a mask for prohibited spot placement areas, and the second mask is a mask for surgically suitable areas. The first mask is used to exclude non-vascular prohibited spot placement areas in subsequent safe target area determination and spot planning. If the first mask includes the FAZ or fovea region, the anatomical center coordinates are calculated only from the centroid of the largest connected region of the FAZ or fovea region.

[0111] It should be noted that, through the pre-segmentation model and morphological post-processing, the first mask and the second mask can be obtained as semantic segmentation information of the target fundus image.

[0112] The processes shown in S201-S204 above generate pixel-level constraint information required for subsequent segmentation correction and spot planning through pre-segmentation model and morphological post-processing, so that non-vascular prohibited spot arrangement areas, surgically suitable areas and FAZ or macular fovea areas can be called in the same planning process.

[0113] like Figure 3 The diagram shown is a schematic of a target mask generation process based on automatic prompts and human-computer interaction prompts provided in an embodiment of this application, including the following steps.

[0114] S301: Generate automatic prompting points based on the second mask.

[0115] Automatic cue points are signals automatically generated by the system based on the current segmentation state to guide the mask generation model in expanding or correcting the target mask. The automatic cue point generation strategy aims to improve the coverage of the target mask over the second mask (the surgically appropriate area) and reduce the workload of doctors manually adding cue points.

[0116] In some examples, before generating the target mask for the first time, the system determines the initial cue point based on the largest connected region of the second mask. For example, the centroid coordinates of the largest connected region in the second mask can be used as the initial cue point to guide the first mask generation model to generate candidate masks in that region.

[0117] In some examples, if one or more candidate target masks have already been generated, the largest uncovered connected region in the second mask is determined based on the coverage of the current candidate target mask over the second mask, and a new auto-hint point is generated within that region. For example, the Boolean difference between the second mask and the current target mask can be calculated to obtain the mask of the uncovered region, and then the centroid of the largest connected region of the uncovered region mask can be extracted as the new auto-hint point.

[0118] It should be noted that if the area of ​​the largest connected region in the uncovered area is lower than the preset minimum prompt area threshold, such as being smaller than the area covered by the spot diameter, the system can stop generating new automatic prompt points to avoid generating invalid prompts in clinically meaningless fragmented areas.

[0119] S302: Receives user input of human-computer interaction prompts.

[0120] Among them, human-computer interaction prompts refer to the prompt signals actively input by doctors through a graphical interface to correct the target mask segmentation results. The human-computer interaction prompts input by the user and the automatic prompts generated by the system together constitute the prompt input of the mask generation model.

[0121] In some examples, users can enter positive or negative prompts, add or delete regions. Positive prompts are used to increase the target treatment area, while negative prompts are used to decrease the target treatment area or exclude incorrectly included areas. Add and delete regions can be used to locally correct candidate safe target area masks or candidate spot regions.

[0122] In a possible implementation, the user can directly place positive or negative cue points on the display interface of the target fundus image by clicking with a mouse or tapping with a stylus. The system can assign a first preset label value (e.g., 1) to positive cue points and a second preset label value (e.g., 0) to negative cue points to distinguish different types of cue signals.

[0123] In another possible implementation, users can also draw closed outlines or irregular doodle areas using the lasso tool or brush tool, which the system will then parse into region-level commands to add or delete regions.

[0124] S303: Input the automatic prompting point and / or human-computer interaction prompting command into the prompting encoder to obtain the target prompting feature.

[0125] A cue encoder is a learnable module that maps discrete cue signals (such as point coordinates or label types) into high-dimensional feature representations compatible with the image feature space. The cue encoder converts point cues or other interactive cues into feature representations compatible with the target image features for use by the mask generation model.

[0126] In some examples, the cue encoder is implemented based on positional encoding and learnable embedding vectors. For positive and negative cue points, the cue encoder first generates the corresponding positional encoding based on the pixel coordinates of the cue point, and then looks up the corresponding semantic embedding vector from the learnable embedding vector table according to the label type (positive or negative) of the cue point. The positional encoding and the semantic embedding vector are added or concatenated to obtain the target cue feature of the cue point.

[0127] In a possible implementation, if multiple prompts exist simultaneously, the target prompt features of each prompt point can be averaged or summed to obtain the global target prompt features. For instructions to add or delete a region, the region mask can first be converted into a dense set of coordinates, and then encoded in a manner similar to that used for point prompts.

[0128] In possible implementations, the effects of human-computer interaction and surgical plan confirmation processes can be found in [reference needed]. Figure 8 As shown.

[0129] S304: Generate candidate masks based on target image features and target cue features.

[0130] Here, the candidate mask refers to the segmentation output used to represent the treatment region, predicted by the mask generation model based on image features and cue features. Depending on whether it's the first generation or an iterative correction, either the initial mask generation model or the iterative mask generation model is invoked to obtain the candidate mask.

[0131] It should be noted that the first mask is not directly used as input to the initial mask generation model or the iterative mask generation model. Instead, it guides the feature extraction of the target fundus image through step S102, generating target image features containing spatial prior information. The mask generation model relies solely on the target image features and human-computer interaction prompt features (as well as the previous candidate logical values ​​during iteration) for inference.

[0132] During the initial mask generation, the target image features and target cue features are input into the initial mask generation model to obtain candidate masks and candidate logical values. The initial mask generation model relies solely on the target image features extracted by the image encoder and the target cue features generated by the cue encoder, and does not include iterative information from previous masks.

[0133] In one possible implementation, the initial mask generation model can concatenate the target image features and target cue features along the channel dimension, then perform upsampling and pixel-by-pixel classification through a decoder network to output a high-resolution candidate mask corresponding to the spatial size of the input image, while simultaneously outputting a low-resolution candidate logical value for use as the previous mask input in the next iteration.

[0134] During iterative mask generation, the candidate logic values ​​obtained from the previous mask generation are scaled to the preset mask input size and input into the iterative mask generation model together with the newly added human-computer interaction prompts to obtain the updated candidate mask.

[0135] It should be noted that the candidate logical values ​​are intermediate representations in a low-resolution feature space, preserving the probability distribution information of the previous segmentation results. Scaling the candidate logical values ​​to the preset mask input size aligns them with the spatial dimensions of the image features.

[0136] In a possible implementation, the iterative mask generation model concatenates the target image features, target cue features, and scaled candidate logical values ​​along the channel dimension. After processing by a decoder network, it outputs an updated candidate mask and new candidate logical values. This iterative mechanism allows the model to make local corrections based on previous segmentation results each time a doctor adds or modifies a cue instruction, rather than starting from scratch, thereby improving the interaction response speed.

[0137] S305: Perform post-processing and constraint filtering on the candidate mask.

[0138] Post-processing and constraint filtering refer to a series of morphological refinements and logical checks on the candidate masks output by the mask generation model to eliminate segmentation artifacts, fill internal holes, ensure that the candidate masks do not exceed the appropriate surgical area, and filter out invalid schemes that do not meet clinical requirements, so as to finally obtain the target mask.

[0139] It should be noted that the candidate mask is subjected to size restoration, small connected component removal, hole filling, and second mask constraints to obtain the target mask. Candidate masks that overlap with existing candidate schemes by more than a preset condition, whose area ratio does not meet the preset condition, or that do not meet the second mask constraints are marked as invalid schemes and excluded from the subsequent automatic suggestion area.

[0140] In some examples, size restoration is performed on the candidate mask. Because the mask generation model may downsample the feature map during inference, the size of its output candidate mask may be smaller than the size of the original target fundus image. In a possible implementation, the candidate mask can be upsampled to the same spatial resolution as the target fundus image using bilinear interpolation or nearest-neighbor interpolation to ensure the coordinate accuracy of subsequent spot planning.

[0141] In some examples, small connected component removal is performed on the candidate mask after size restoration. Small connected component removal involves traversing all connected regions with positive values ​​in the candidate mask, calculating the pixel area of ​​each connected region, and deleting isolated fragment regions whose pixel area is less than a preset area threshold. In one possible implementation, the preset area threshold can be set as a multiple of the spot area.

[0142] In some examples, a second mask constraint is applied to the candidate mask. The second mask constraint involves performing a pixel-by-pixel logical AND operation between the candidate mask and the second mask, retaining only the portion of the candidate mask that falls within the surgically appropriate area.

[0143] In some examples, for the candidate masks obtained after the above post-processing, constraint filtering needs to be performed to determine whether they are valid solutions. The constraint filtering conditions include: whether the overlap ratio with existing candidate solutions exceeds a preset condition, whether the area ratio of the target mask relative to the second mask meets a preset condition, and whether the constraints of the second mask are met.

[0144] The processes shown in S301-S305 above use automatic prompts and user interaction prompts to drive the generation of the target mask, enabling the system to automatically cover the main surgical appropriate areas while allowing doctors to quickly correct mis-segmented areas, missed areas, and clinically prohibited areas.

[0145] like Figure 4 The diagram shown is a flowchart illustrating a process for determining a security target mask based on a first mask and a second mask, according to an embodiment of this application. The process includes the following steps.

[0146] S401: In response to user interaction with semantic segmentation information, update the first mask or the second mask.

[0147] User interaction includes user confirmation or fine-tuning of prohibited spot placement areas or suitable surgical areas. Updating the first mask will simultaneously affect subsequent safe target area masks and spot planning results; updating the second mask will simultaneously affect the effective range of the target mask and the allowed spot center area.

[0148] In some examples, user interactions include confirming or fine-tuning prohibited areas or suitable surgical areas. For prohibited areas, doctors can check if the first mask completely covers the macula, avascular fovea, optic disc, areas with previous laser photocoagulation marks, and other non-vascular areas that need to be avoided. If omissions or oversegmentation are found, they can be completed or erased using interface tools. For suitable surgical areas, doctors can check if the second mask accurately represents non-perfusion areas, areas surrounding retinal neovascularization, or other areas to be treated, and locally modify the boundaries of the second mask based on clinical judgment.

[0149] It should be noted that updating the first mask will simultaneously affect the subsequent safe target area mask and spot planning results, because the updated first mask will exclude newly added prohibited spot arrangement areas through Boolean difference operations; updating the second mask will simultaneously affect the effective range of the target mask and the allowed spot center area, because the target mask must be constrained within the range of the updated second mask.

[0150] S402: Perform a Boolean difference operation between the target mask and the first mask.

[0151] Boolean difference operation refers to performing a logical difference operation on two binary masks at the pixel level, removing the part that overlaps with the first mask from the target mask, and ensuring that the prohibited spot arrangement area does not enter the subsequent safe target area.

[0152] It should be noted that Boolean difference operations can be used to remove the part of the target mask that overlaps with the first mask, so that the prohibited spot arrangement area does not enter the safe target area.

[0153] S403: Constrain the difference results within the surgically appropriate area corresponding to the second mask to obtain a safe target mask.

[0154] Constraining the difference result within the second mask means performing a pixel-by-pixel logical AND operation between the difference mask obtained in S402 and the second mask, so that the safe target area mask only includes regions that simultaneously meet the following three conditions: the target mask predicts the treatment area, the first mask does not prohibit it, and the second mask identifies it as a suitable surgical area.

[0155] It should be noted that the safe target area mask includes the region where the target mask exists, the second mask exists, and the first mask does not exist, so that the subsequent treatment spot array must not be planned to the non-vascular prohibited spot arrangement region corresponding to the first mask.

[0156] The processes shown in S401-S403 above enable the primary function of the first mask to be the prohibition planning constraint for surgical spot planning, and provide a basis for the subsequent generation of the allowable center mask.

[0157] like Figure 5 The diagram shown is a flowchart illustrating a process for generating a final surgical spot array based on a security target mask, a first mask, and a blood vessel restricted area, according to an embodiment of this application. The process includes the following steps.

[0158] S501: Determine the radius step size and the minimum center distance between adjacent light spots based on the preset light spot layout parameters.

[0159] The preset spot layout parameters include spot diameter and spot spacing. In some examples, the preset spot layout parameters may also include at least one of the following: number of spot array layers, maximum planning radius, safety distance threshold, minimum coverage requirement, and local repair relaxation threshold.

[0160] S502: If the first mask contains the FAZ or the fovea region, determine the coordinates of the anatomical center based on the centroid of the largest connected region of the FAZ or the fovea region.

[0161] Among them, the anatomical center coordinates refer to the reference point of the center of the circle used to generate multiple concentric candidate rings. Its clinical significance is that the spot array is arranged in a ring with the geometric center of the avascular area of ​​the fovea or the fovea region as a reference, which is in line with the clinical practice of arranging the spot around anatomical landmarks in fundus laser photocoagulation surgery.

[0162] In some examples, if the first mask contains both the FAZ or the fovea region and other prohibited light spot arrangement areas (such as the optic disc area, the area of ​​old laser photocoagulation marks, or the user-marked prohibited area), the anatomical center coordinates are extracted only from the FAZ or the fovea region, and the other prohibited light spot arrangement areas are not included in the center point calculation.

[0163] S503: Using the coordinates of the anatomical center as the center, generate multiple candidate rings according to the radius step size, and determine the original candidate spot center on each candidate ring based on the minimum center distance between adjacent spots.

[0164] Among them, the multi-layer candidate ring refers to a series of concentric rings generated with the anatomical center coordinates as the center and the radius as an integer multiple of the radius step size. Each candidate ring has original candidate spot centers distributed on it. These original candidate spot centers are initially arranged without considering the constraints of blood vessel avoidance and prohibited areas.

[0165] It should be noted that the anatomical center coordinates, as the center of the multi-layer candidate ring, are one of the auxiliary functions of the first mask and do not change the main function of the first mask as a mask for prohibited spot arrangement areas; the visual disc area, the area of ​​old laser photocoagulation marks or the user-marked prohibited area in the first mask do not participate in the calculation of the spot arrangement center point.

[0166] S504: Generate an allowable center mask that allows the center of the light spot to fall into the target area based on the safe target area mask, the first mask, and the blood vessel restricted area.

[0167] The allowable center mask is a binary mask used to define the set of pixel positions in the final surgical spot array where the center of the spot can legally fall. The spot center is only a legal placement point if it falls within a pixel position where the allowable center mask is true.

[0168] It should be noted that the center mask is allowed to exclude the non-vascular prohibited spot arrangement area and the vascular prohibited area corresponding to the first mask. Specifically, the center mask is allowed to exclude the area outside the safe target area, the prohibited spot arrangement area corresponding to the first mask, and the vascular prohibited area obtained by expanding the spot radius.

[0169] S505: For each candidate ring, sample candidate centers within the allowed center mask and construct a candidate layout set.

[0170] The candidate layout set refers to the set of all possible valid spot center arrangement schemes generated for each candidate ring. This set is constructed as follows: using the original candidate spot centers generated by S503 as a reference, and under the constraint of allowing center masks, a local position search is performed on each original candidate spot center to generate multiple candidate arrangement schemes.

[0171] In some examples, for an original candidate spot center on a candidate ring, if the coordinates fall within the allowed center mask, it is directly used as a candidate center; if the original candidate spot center does not fall within the allowed center mask, a local search can be performed in its neighboring area according to a preset search radius and a preset search step size to find the nearest pixel coordinates that fall within the allowed center mask as the candidate center.

[0172] S506: By solving the global layout across candidate rings, the final layout is selected from the candidate layout sets of multiple candidate rings to obtain the final surgical spot array.

[0173] The global layout solution across candidate rings involves a joint optimization search across multiple candidate ring sets to select the final spot center arrangement scheme for each candidate ring. All selected candidate ring arrangement schemes together constitute the final surgical spot array. Unlike the independent ring-by-ring solution, the global layout solution considers the mutual constraints between different candidate rings, such as the spot spacing constraints between adjacent rings.

[0174] It should be noted that the global layout solution takes maximizing the final number of light spots as the first optimization objective, minimizing the center-to-center distance penalty between adjacent light spots as the second optimization objective, and minimizing the movement cost relative to the original candidate light spot center as the third optimization objective.

[0175] The first optimization objective is to maximize the treatment coverage and the number of effective spots in the final arrangement. The second optimization objective is to ensure the uniformity of the final array by penalizing the spacing between adjacent spots that is too small or too large. The third optimization objective is to make the final layout as close as possible to a regular geometric arrangement based on the anatomical center by penalizing the offset of the spot center relative to the original candidate position.

[0176] S507: When the restricted area of ​​blood vessels or the first mask causes local spot loss, local treatment voids or cross-ring spacing conflicts, perform free candidate point search and local repair in the corresponding ring or local sector to restore the number of available spots as much as possible and maintain array uniformity.

[0177] Local spot loss refers to a situation where, within a local sector of a candidate ring, due to the obstruction of a prohibited area for blood vessels or the first mask, there are no valid pixels falling into the allowed center mask within the original candidate spot center and its adjacent areas, resulting in a treatment hole with no spot coverage in that sector. Cross-ring spacing conflict refers to a situation where, after solving the global layout, the spacing between spots in adjacent candidate rings is too small or too large.

[0178] In some examples, free candidate point search refers to finding the coordinate positions where one or more light spots can be added within the treatment cavity area by random sampling or grid search, provided that the allowable center mask constraint and the adjacent spot spacing constraint are satisfied.

[0179] In one possible implementation, one or more candidate points can be generated near the geometric center of the treatment cavity area, and the candidate points can be checked in turn to see if they fall into the allowed center mask and whether the distance between the candidate points and the surrounding existing light spots meets the minimum center distance requirement between adjacent light spots.

[0180] The process described in S501-S507 above differs from the local filtering method that first generates a complete light spot array and then deletes light spots that conflict with blood vessels. Instead, it performs global optimization between candidate rings, which can strictly avoid prohibited light spot arrangement areas and prohibited blood vessel areas while taking into account treatment coverage, adjacent spacing and array uniformity.

[0181] like Figure 6 The diagram shown is a structural schematic of an image processing-based fundus laser photocoagulation surgery planning device provided in an embodiment of this application, which includes the following units.

[0182] The semantic segmentation unit 100 is used to obtain semantic segmentation information of the target fundus image based on a pre-constructed pre-segmentation model. The semantic segmentation information includes a first mask and a second mask. The first mask is a mask for the prohibited light spot arrangement area, and the second mask is a mask for the surgically appropriate area.

[0183] Optionally, the semantic segmentation unit 100 is specifically used for: acquiring a preprocessed target fundus image; inputting the target fundus image into a pre-constructed pre-segmentation model to obtain a semantic label map output by the pre-segmentation model; performing morphological post-processing on the semantic label map; and obtaining a first mask and a second mask based on the morphologically post-processed semantic label map. The first mask is used to exclude non-vascular prohibited spot arrangement regions in subsequent safe target region determination and spot planning. If the first mask includes the avascular region of the fovea or the fovea region, the anatomical center coordinates for subsequent spot arrangement are determined only from the largest connected region of the avascular region of the fovea or the fovea region.

[0184] Image coding unit 200 is used to guide feature extraction of the target fundus image based on the first mask in order to obtain the target image features.

[0185] Optionally, the image encoding unit 200 is specifically used to: input the first mask and the target fundus image into a pre-constructed image encoder to obtain the target image features output by the image encoder; the image encoder is used to extract features from the target fundus image, and uses the first mask as spatial prior information of the non-vascular prohibited spot arrangement area to apply spatial attention masking to the feature extraction process to suppress feature extraction of the surgical avoidance area.

[0186] The prompt encoding unit 300 is used to obtain the target prompt features of the human-computer interaction prompt instructions.

[0187] The mask generation unit 400 is used to generate a mask based on a pre-built initial mask generation model or an iterative mask generation model, using a first mask, target cue features, and target image features to obtain a target mask.

[0188] Optionally, the construction of the pre-segmentation model, the initial mask generation model, or the iterative mask generation model includes: converting fundus medical images with pixel-level annotations and their corresponding masks into training samples; the training samples include at least an image array and a real mask array; training an anatomically protected region segmentation model with no or weak cue input for the avascular area or fovea region of the macula, and training an interactive segmentation network model with human-computer interaction cue input for the surgically suitable area; training the interactive segmentation network model based on the training samples, using at least one of point cue, previous mask input, or no cue input as input conditions during training, and using at least one of mask loss, Dice loss, cross-union prediction loss, and target category loss as training constraints; exporting the trained interactive segmentation network model and the anatomically protected region segmentation model into an open neural network exchange model that can be called by a medical terminal; the open neural network exchange model includes at least a pre-segmentation model for outputting semantic segmentation information, an initial mask generation model for generating the initial target mask, and an iterative mask generation model for receiving the previous mask input and continuously correcting it.

[0189] Optionally, the mask generation unit 400 is specifically used for: during the initial mask generation, inputting the target image features and target prompt features into a pre-built initial mask generation model to obtain a first candidate mask and the corresponding candidate logical value; during iterative mask generation, scaling the candidate logical value obtained from the previous mask generation to a preset mask input size, and inputting it together with the newly added human-computer interaction prompt instruction into a pre-built iterative mask generation model to obtain an updated candidate mask; performing size restoration, small connected component removal, hole filling, and second mask constraint on the candidate mask to obtain the target mask; and recording candidate masks whose overlap ratio with existing candidate schemes exceeds a preset condition, whose area ratio does not meet the preset condition, or whose second mask constraint is not met as invalid schemes and excluding them from the subsequent automatic prompt area.

[0190] The mask filtering unit 500 is used to determine the safe target area mask from the target mask based on the first mask and the second mask.

[0191] Optionally, the mask filtering unit 500 is specifically used to: update the first mask or the second mask in response to user interaction behavior of semantic segmentation information; the user interaction behavior includes user confirmation or fine-tuning of prohibited spot arrangement areas or surgically suitable areas; perform Boolean difference operation on the target mask and the first mask; constrain the difference result to the surgically suitable area corresponding to the second mask to obtain a safe target area mask; the safe target area mask includes the area where the target mask exists, the second mask exists, and the first mask does not exist.

[0192] The vessel segmentation unit 600 is used to obtain the vessel mask of the target fundus image and construct the vessel restricted area based on the vessel mask and the surgical spot radius.

[0193] Optionally, the vessel segmentation unit 600 is specifically used for: inputting the target fundus image into the vessel segmentation model to obtain a vessel probability map; normalizing and thresholding the vessel probability map to obtain an initial vessel binary map; performing erosion, small connected component removal, dense noise region removal, and spur removal processing on the initial vessel binary map to obtain a vessel mask; dilating the vessel mask according to the surgical spot radius to obtain a vessel forbidden region; and in the global strict vessel avoidance optimization, the center of the spot must not fall into the vessel forbidden region.

[0194] The spot planning unit 700 is used to generate a multi-layer candidate spot array based on the safety target area mask, the first mask, the blood vessel no-entry area and the preset spot layout parameters, and to perform global strict blood vessel avoidance optimization in the multi-layer candidate spot array to obtain the final surgical spot array.

[0195] Optionally, the spot planning unit 700 is specifically used for: determining the radius step size and the minimum center distance between adjacent spots based on preset spot layout parameters; when the first mask includes the avascular region or fovea region of the fovea, determining the anatomical center coordinates based on the centroid of the largest connected region of the avascular region or fovea region of the fovea; generating multiple candidate rings with the anatomical center coordinates as the center and according to the radius step size, and determining the original candidate spot center on each candidate ring based on the minimum center distance between adjacent spots; generating a target area mask, the first mask, and the vessel-prohibited area based on the target area mask, the first mask, and the vessel-prohibited area, and determining the target area mask, the target area mask, and the target area mask. An allowable center mask is used; where the allowable center mask excludes non-vascular prohibited spot arrangement areas and vascular prohibited areas corresponding to the first mask; for each candidate ring, candidate centers are sampled within the allowable center mask, and a candidate layout set for the candidate ring is constructed; through global layout solution across candidate rings, the final layout is selected among the candidate layout sets of multiple candidate rings to obtain the final surgical spot array; the global layout solution takes maximizing the number of final spots as the first optimization objective, minimizing the center distance penalty between adjacent spots as the second optimization objective, and minimizing the movement cost relative to the original candidate spot center as the third optimization objective.

[0196] The file generation unit 800 is used to generate a standardized surgical plan file that can be called by a medical terminal based on the final surgical spot array and the safety target mask, according to a preset data protocol.

[0197] Each of the above units explicitly excludes non-vascular prohibited spot arrangement areas through the first mask, constrains the appropriate surgical area through the second mask, corrects the target mask through human-computer interaction prompts, and generates the final surgical spot array through independent vascular masks, vascular prohibited areas, and global strict vascular avoidance optimization, thereby improving the safety, coverage, and controllability of preoperative planning for fundus laser photocoagulation.

[0198] This application also provides a computer-readable storage medium including a stored program, wherein the program executes the image processing-based fundus laser photocoagulation surgery planning method provided in this application.

[0199] This application also provides an electronic device, including a processor, a memory, and a bus. The processor and the memory are connected via the bus. The memory is used to store a program, and the processor is used to run the program. When the program runs, it executes the image processing-based fundus laser photocoagulation surgery planning method provided in this application.

[0200] While several specific implementation details are included in the foregoing discussion, these should not be construed as limiting the scope of this application. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0201] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method for planning a fundus laser photocoagulation surgery procedure based on image processing, characterized in that, include: Based on a pre-constructed pre-segmentation model, semantic segmentation information of the target fundus image is obtained; the semantic segmentation information includes a first mask and a second mask; the first mask is a mask for prohibited spot arrangement regions; the second mask is a mask for surgically appropriate regions. Based on the first mask, the target fundus image is guided to perform feature extraction in order to obtain target image features; Obtain the target prompt features of the human-computer interaction prompts; Based on a pre-built initial mask generation model or iterative mask generation model, a mask is generated for the first mask, the target cue features, and the target image features to obtain a target mask; Based on the first mask and the second mask, determine the safe target area mask from the target mask; Obtain the vascular mask of the target fundus image, and construct a vascular no-entry region based on the vascular mask and the surgical spot radius; Based on the safety target area mask, the first mask, the blood vessel restricted area and the preset spot layout parameters, a multi-layer candidate spot array is generated, and global strict blood vessel avoidance optimization is performed in the multi-layer candidate spot array to obtain the final surgical spot array. Based on the final surgical spot array and the safe target area mask, a standardized surgical plan file that can be called by a medical terminal is generated according to a preset data protocol.

2. The method according to claim 1, characterized in that, Based on a pre-built pre-segmentation model, semantic segmentation information of the target fundus image is obtained, including: Obtain the preprocessed target fundus image; The target fundus image is input into a pre-constructed pre-segmentation model to obtain a semantic label map output by the pre-segmentation model; Perform morphological post-processing on the semantic tag graph; Based on the semantic tag map after morphological post-processing, a first mask and a second mask are obtained; wherein, the first mask is used to exclude non-vascular prohibited spot arrangement areas in the subsequent safe target area determination and spot planning; if the first mask includes the avascular area of ​​the fovea or the fovea region, the anatomical center coordinates for subsequent spot arrangement are determined only from the largest connected region of the avascular area of ​​the fovea or the fovea region.

3. The method according to claim 1, characterized in that, The construction of the pre-segmentation model, the initial mask generation model, or the iterative mask generation model includes: Medical images of the fundus with pixel-level annotations and their corresponding masks are converted into training samples; the training samples include at least an image array and a real mask array. For the avascular area or fovea region of the macula, we train an anatomical protection region segmentation model with no or weak prompting input, and for the surgically appropriate area, we train an interactive segmentation network model with human-computer interaction prompting input. Based on the training samples, the interactive segmentation network model is trained. During the training process, at least one of point prompts, previous masked input, or no prompt input is used as the input condition, and at least one of mask loss, Dice loss, cross-union prediction loss, and target category loss is used as the training constraint. The trained interactive segmentation network model and the anatomical protection region segmentation model are exported as an open neural network exchange model that can be called by a medical terminal. The open neural network exchange model includes at least a pre-segmentation model for outputting the semantic segmentation information, an initial mask generation model for generating the initial target mask, and an iterative mask generation model for receiving the previous mask input and making continuous corrections.

4. The method according to claim 1, characterized in that, Based on the first mask, feature extraction is performed on the target fundus image to obtain target image features, including: The first mask and the target fundus image are input into a pre-constructed image encoder to obtain the target image features output by the image encoder. The image encoder is used to extract features from the target fundus image and uses the first mask as spatial prior information for non-vascular prohibited spot arrangement areas to apply spatial attention masking to the feature extraction process, so as to suppress feature extraction of surgical avoidance areas.

5. The method according to claim 1, characterized in that, Based on a pre-built initial mask generation model or iterative mask generation model, a mask is generated from the first mask, the target cue features, and the target image features to obtain a target mask, including: During the initial mask generation, the target image features and the target cue features are input into the pre-constructed initial mask generation model to obtain the first candidate mask and the corresponding candidate logical value; During iterative mask generation, the candidate logical values ​​obtained from the previous mask generation are scaled to the preset mask input size and input together with the newly added human-computer interaction prompts into the pre-built iterative mask generation model to obtain the updated candidate mask. The candidate mask is subjected to size restoration, small connected component removal, hole filling, and second mask constraint to obtain the target mask; Candidate masks that overlap with existing candidate solutions by more than the preset condition, whose area ratio does not meet the preset condition, or that do not meet the second mask constraint are recorded as invalid solutions and excluded from the subsequent automatic prompt area.

6. The method according to claim 1, characterized in that, Determining a secure target area mask from the target mask based on the first mask and the second mask includes: In response to user interaction with the semantic segmentation information, the first mask or the second mask is updated; the user interaction includes user confirmation or fine-tuning of prohibited spot arrangement areas or suitable surgical areas; Perform a Boolean difference operation between the target mask and the first mask; The difference result is constrained within the surgically appropriate area corresponding to the second mask to obtain a safe target area mask; the safe target area mask includes the area where the target mask exists, the second mask exists, and the first mask does not exist.

7. The method according to claim 1, characterized in that, Obtain the vascular mask of the target fundus image, and construct a vascular forbidden region based on the vascular mask and the surgical spot radius, including: The target fundus image is input into the blood vessel segmentation model to obtain a blood vessel probability map; The blood vessel probability map is normalized and thresholded to obtain an initial binary blood vessel map; The initial binary image of blood vessels is subjected to erosion, small connected region removal, dense noise region removal and spiking removal to obtain a blood vessel mask; The blood vessel mask is expanded according to the radius of the surgical spot to obtain the blood vessel restricted area; in the global strict blood vessel avoidance optimization, the center of the spot must not fall into the blood vessel restricted area.

8. The method according to claim 1, characterized in that, Based on the safety target mask, the first mask, the blood vessel restricted area, and preset spot layout parameters, a multi-layer candidate spot array is generated, and global strict blood vessel avoidance optimization is performed in the multi-layer candidate spot array to obtain the final surgical spot array, including: Based on the preset spot layout parameters, determine the radius step size and the minimum center distance between adjacent spots; If the first mask includes the avascular region of the fovea or the fovea region, the coordinates of the anatomical center are determined based on the centroid of the largest connected region of the avascular region of the fovea or the fovea region. Using the coordinates of the anatomical center as the center, generate multiple candidate rings according to the radius step size, and determine the original candidate spot center on each candidate ring based on the minimum center distance between adjacent spots; Based on the safety target area mask, the first mask, and the blood vessel restricted area, an allowable center mask is generated that allows the center of the light spot to fall into the mask; wherein, the allowable center mask excludes the non-blood vessel restricted light spot arrangement area corresponding to the first mask and the blood vessel restricted area. For each candidate ring, sample the candidate center within the allowed center mask and construct a candidate layout set for the candidate ring; By solving the global layout across candidate rings, the final layout is selected from the candidate layout sets of multiple candidate rings to obtain the final surgical spot array; the global layout solution takes maximizing the number of final spots as the first optimization objective, minimizing the center distance penalty between adjacent spots as the second optimization objective, and minimizing the movement cost relative to the original candidate spot center as the third optimization objective.

9. A device for planning fundus laser photocoagulation surgery based on image segmentation masking, human-computer interaction prompts, and blood vessel avoidance, characterized in that, include: A semantic segmentation unit is used to obtain semantic segmentation information of a target fundus image based on a pre-built pre-segmentation model; the semantic segmentation information includes a first mask and a second mask, wherein the first mask is a mask for prohibited light spot arrangement areas; and the second mask is a mask for surgically appropriate areas. An image encoding unit is used to guide the target fundus image to perform feature extraction based on the first mask, so as to obtain target image features; A prompting encoding unit is used to obtain the target prompting features of human-computer interaction prompting instructions; The mask generation unit is used to generate a mask for the first mask, the target cue feature and the target image feature based on a pre-built initial mask generation model or an iterative mask generation model, so as to obtain a target mask; A mask filtering unit is used to determine a security target area mask from the target mask based on the first mask and the second mask; The vessel segmentation unit is used to obtain the vessel mask of the target fundus image and construct a vessel restricted area based on the vessel mask and the surgical spot radius. The spot planning unit is used to generate a multi-layer candidate spot array based on the safety target area mask, the first mask, the blood vessel restricted area and preset spot layout parameters, and to perform global strict blood vessel avoidance optimization in the multi-layer candidate spot array to obtain the final surgical spot array. The file generation unit is used to generate a standardized surgical plan file that can be called by a medical terminal based on the final surgical spot array and the safe target area mask, according to a preset data protocol.

10. A storage medium, characterized in that, The storage medium includes a stored program, wherein the program is executed by a processor to perform the image processing-based fundus laser photocoagulation surgery planning method according to any one of claims 1-8.

Images