Methods and apparatus for predicting anterior segment structure, electronic equipment and storage media

Abstract

This application provides a method, apparatus, electronic device, and storage medium for predicting anterior segment structure. The method includes: encoding a preoperative anterior segment structure image and the ICL implantation direction using an encoder module to obtain anterior segment structure image features; generating an iris image based on the anterior segment structure image features using a first auxiliary decoder, a second auxiliary decoder, and a geometric information sharing module to obtain an iris segmentation image output by the first auxiliary decoder and an iris contour and lens upper surface image output by the second auxiliary decoder; decoding the anterior segment structure image features based on a main decoder and the first auxiliary decoder to obtain a postoperative anterior segment structure image; and displaying the postoperative anterior segment structure image, iris segmentation image, and iris contour and lens upper surface image under the ICL implantation direction. This application embodiment can achieve postoperative anterior segment structure prediction with high accuracy.

Description

Technical Field

[0001] This application relates to the fields of deep learning and image processing technology, and in particular to a method and apparatus for predicting anterior segment structure, an electronic device, and a storage medium. Background Technology

[0002] Implantable collamer lenses (ICLs) are soft, elastic gel lenses made of collagen polymer materials. ICL surgery is an ophthalmic procedure that involves implanting an ICL (Implantable Collamer Lens) into the eye for refractive correction, such as hyperopia (farsightedness) and myopia (nearsightedness). When planning ICL surgery, it's necessary to predict postoperative parameters such as the anterior segment structure before surgery, including the arch height and anterior chamber angle, to determine the appropriate ICL size and implantation direction. This is crucial for reducing postoperative complications and achieving better postoperative recovery.

[0003] Related techniques generally employ regression models to predict postoperative anterior chamber angle status or postoperative arch height, such as using preoperative trabecular iris angle and scleral projection angle to predict postoperative anterior chamber angle and other angle-related parameters. These ICL postoperative prediction techniques typically predict single postoperative parameters (such as anterior chamber angle and arch height) based on preoperative anterior segment structural parameters. While these methods are essentially numerical prediction techniques, anterior segment structural prediction is relatively complex and cannot measure the overall postoperative anterior segment structural status using a single parameter; therefore, they cannot achieve true prediction of postoperative anterior segment structure. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, electronic device, and storage medium for predicting anterior segment structure, which aims to achieve high accuracy in predicting anterior segment structure.

[0005] To achieve the above objectives, a first aspect of this application proposes a method for predicting anterior segment structure images, the method comprising:

[0006] Obtain preoperative anterior segmental images of ICL and ICL implantation direction;

[0007] The preoperative anterior segment structure image of the ICL and the ICL implantation orientation information are input into a pre-trained anterior segment structure image generation model. The anterior segment structure image generation model includes an encoder module, a decoder module, and a geometric information sharing module. The decoder module includes a main decoder, a first auxiliary decoder, and a second auxiliary decoder.

[0008] The encoder module encodes the preoperative anterior segment structure image of the ICL and the ICL implantation direction to obtain the anterior segment structure image features;

[0009] Based on the first auxiliary decoder, the second auxiliary decoder and the geometric information sharing module, the anterior segment structure image features are used to generate an iris image, resulting in an iris segmentation image output by the first auxiliary decoder and an iris contour and lens upper surface image output by the second auxiliary decoder.

[0010] Based on the main decoder and the first auxiliary decoder, the anterior segment structure image features are decoded to obtain the anterior segment structure image after ICL surgery;

[0011] The image displays the anterior segment structure of the ICL after surgery, the iris segmentation image, and the iris contour and lens upper surface image under the ICL implantation direction.

[0012] In some embodiments, the first auxiliary decoder includes a first upsampling layer, a first stitching layer, and a first fusion layer; the second auxiliary decoder includes a second upsampling layer, a second stitching layer, and a second fusion layer; the anterior segment structure image generation model further includes a first output layer and a second output layer; and the step of generating an iris image based on the features of the anterior segment structure image using the first auxiliary decoder, the second auxiliary decoder, and the geometric information sharing module to obtain an iris segmentation image output by the first auxiliary decoder and an iris contour and lens upper surface image output by the second auxiliary decoder includes:

[0013] The anterior segment structure image features are upsampled by the first upsampling layer to obtain a reconstructed image of the first anterior segment structure features;

[0014] The anterior segment structure image features are upsampled by the second upsampling layer to obtain a reconstructed image of the second anterior segment structure features;

[0015] Geometric information is extracted from the reconstructed images of the first and second anterior segment structures using a geometric information sharing module to obtain geometric information sharing results.

[0016] The first feature image is obtained by stitching together the reconstructed image of the anterior segment structure features and the anterior segment structure image features through the first stitching layer;

[0017] The second stitching layer stitches together the reconstructed image of the second anterior segment structure features and the anterior segment structure image features to obtain the stitched result of the second feature image;

[0018] The first image fusion result is obtained by fusing the first feature image stitching result and the geometric information extraction result through the first fusion layer.

[0019] The second fusion layer fuses the second feature image stitching result and the geometric information extraction result to obtain the second image fusion result.

[0020] The first image fusion result is upsampled by the first output layer to output an iris segmentation image;

[0021] The second image fusion result is upsampled by the second output layer to output the iris contour and the upper surface of the lens image.

[0022] In some embodiments, the geometric information sharing module includes convolutional layers, average pooling layers, max pooling layers, and activation layers. The geometric information sharing module extracts geometric information from the first anterior segment structural feature reconstruction image and the second anterior segment structural feature reconstruction image to obtain the geometric information sharing result, including:

[0023] The first anterior segment structural feature reconstruction image and the second anterior segment structural feature reconstruction image are stitched together to obtain the third feature image stitching result;

[0024] Based on the convolutional layer, feature extraction is performed on the stitched result of the third feature image to obtain the geometric feature extraction result;

[0025] Based on the max pooling layer, channel max pooling is performed on the stitching result of the third feature image to obtain the max pooling result;

[0026] Based on the average pooling layer, channel average pooling is performed on the stitching result of the third feature image to obtain the average pooling result;

[0027] The max pooling result, the average pooling result, and the geometric feature extraction result are concatenated to obtain the fourth feature image concatenation result.

[0028] The activation layer performs a nonlinear transformation on the stitching result of the fourth feature image to obtain the geometric information sharing result.

[0029] In some embodiments, before acquiring the preoperative anterior segment structure image of ICL, the method further includes: training the anterior segment structure image generation model, wherein the training process specifically includes:

[0030] Acquire an image sample set and ICL implantation direction, wherein the image sample set includes multiple anterior segment structure image samples, reference iris segmentation images, and reference iris contour and lens images;

[0031] The anterior segment structure image sample is input and the ICL implantation direction is fed into the anterior segment structure image generation model for image prediction, to obtain the postoperative anterior segment structure prediction image, iris segmentation prediction image and iris contour and lens prediction image under the ICL implantation direction;

[0032] The predicted anterior segment structure image after ICL surgery is input into a preset anterior segment structure prediction discriminator for image discrimination, and the image discrimination result is obtained. The adversarial loss is determined based on the image discrimination result.

[0033] A first loss is determined based on the iris segmentation prediction image and the reference iris segmentation image, and a second loss is determined based on the iris contour and lens prediction image and the reference iris contour and lens image.

[0034] The target loss is determined based on the first loss, the second loss, and the adversarial loss, and the model parameters of the anterior segment structure image generation model are updated based on the target loss.

[0035] In some embodiments, determining the first loss based on the iris segmentation prediction image and the reference iris segmentation image includes:

[0036] Obtain the preset weighted map of the iris root position;

[0037] The binary cross-entropy loss is determined based on the iris segmentation prediction image and the reference iris segmentation image;

[0038] The first loss is obtained by adjusting the weights of the binary cross-entropy loss based on the weighted map of the iris root position.

[0039] In some embodiments, acquiring preoperative anterior segmental images of ICL includes:

[0040] The brightness of the input preoperative anterior segment structure image of ICL is adjusted according to a preset brightness adjustment curve to obtain a preoperative anterior segment structure brightness adaptation image of ICL. There are multiple brightness adjustment curves.

[0041] Multiple preoperative anterior segment structure brightness adaptation images with different brightness levels of the ICL were stitched together to obtain a multi-brightness stitched image.

[0042] Based on a preset attention module, feature weights are assigned to the multi-brightness stitched image to obtain the preoperative anterior segment structure image of the ICL.

[0043] In some embodiments, the preoperative anterior segment image of the ICL also includes the corresponding ICL size, and the method further includes:

[0044] The preoperative anterior segment structure image of the ICL is input into the anterior segment structure image generation model to generate a postoperative anterior segment structure image of the ICL at the corresponding ICL size.

[0045] Recommendations for ICL implantation direction are made based on the postoperative anterior segmental structure images of ICL and the corresponding ICL implantation direction.

[0046] ICL size recommendations are made based on the anterior segment structure images and corresponding ICL sizes after ICL surgery.

[0047] To achieve the above objectives, a second aspect of the present application provides an anterior segment structure image prediction device, which is applied to the method described in the first aspect.

[0048] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0049] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0050] This application presents a method, device, electronic device, and storage medium for predicting anterior segment structure images after ICL surgery, providing a visualized method for predicting anterior segment structure after ICL surgery. It acquires preoperative anterior segment structure images and the ICL implantation direction, and uses a pre-trained anterior segment structure image generation model to predict the corresponding postoperative anterior segment structure image, iris segmentation image, and iris contour and lens upper surface image after ICL surgery, based on the selected ICL implantation direction. The input and output of the generation model are both images. Compared to related technologies that can only perform numerical prediction of anterior segment structure parameters and can only predict single postoperative parameters (such as arch height and anterior chamber angle), the prediction method provided in this application can more comprehensively and accurately predict the anterior segment structure after ICL surgery, which is helpful for analyzing complex anterior segment structure states. Furthermore, this application obtains anterior segment structure image features through an encoder module. In addition to setting a main decoder to decode the anterior segment structure image features and generate anterior segment structure images after ICL surgery, a first auxiliary decoder and a second auxiliary decoder are also set. The main decoder is combined with the first auxiliary decoder. The first auxiliary decoder can guide the main decoder to focus on important anterior segment structures, such as the iris position, thereby improving the accuracy of postoperative image prediction. The first and second auxiliary decoders can output iris segmentation images, iris contour and lens upper surface images, respectively, which helps to observe important structures of the anterior segment and ultimately improve the prediction accuracy of the generated model. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the anterior segment structure after ICL surgery provided in the embodiments of this application;

[0052] Figure 2 This is a flowchart of the anterior segment structure prediction method provided in the embodiments of this application;

[0053] Figure 3 This is a schematic diagram of the AS-OCT modal images provided in the embodiments of this application;

[0054] Figure 4 This is a schematic diagram of the input or output image provided in the embodiments of this application;

[0055] Figure 5 This is a schematic diagram of the multi-brightness adaptation module provided in an embodiment of this application;

[0056] Figure 6 This is a schematic diagram of the framework of the anterior segment structure image generation model provided in the embodiments of this application;

[0057] Figure 7 This is a schematic diagram of the geometric information sharing module provided in an embodiment of this application;

[0058] Figure 8 This is a training block diagram of the anterior segment structure image generation model provided in the embodiments of this application;

[0059] Figure 9 This is a schematic diagram of the anterior segment structure prediction device provided in the embodiments of this application;

[0060] Figure 10 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0062] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0064] Before introducing the anterior segment structure prediction method provided in this application, the relevant background of ICL surgery will be explained. Please refer to... Figure 1 , Figure 1 This is a schematic diagram of the anterior segment structure after ICL surgery, provided in an embodiment of this application. The anterior segment structure mainly includes the ciliary muscle, cornea, iris, and lens. ICL includes the lens body and ICL antennae. ICL surgery is a vision correction surgery performed by an ophthalmologist to implant an ICL lens into the eye. During the surgery, the ICL lens is pushed into the posterior chamber. After implantation, the ICL is located in front of the lens and behind the iris.

[0065] Postoperative vault is a crucial parameter in ICL surgery planning. Vault is defined as the maximum distance between the posterior surface of the ICL and the anterior surface of the lens, or, alternatively, by connecting two SS points and drawing a perpendicular bisector; the distance between these intersections with both the posterior and anterior surfaces of the ICL is the vault. Insufficient vault may lead to secondary cataracts, while excessive vault may cause angle stenosis and closed glaucoma. The anterior chamber angle is the angle between the cornea and iris; a narrowed anterior chamber angle can cause impaired aqueous humor flow, increased intraocular pressure, and optic nerve damage. Therefore, appropriately selecting the ICL size and correct implantation direction helps avoid postoperative complications.

[0066] Among related technologies, some solutions employ regression or ensemble models, using preoperative anterior segment structural parameters such as sulcus-to-sulcus (STS), white-to-white (WTW), and anterior chamber depth (ACD) to predict postoperative arch height and recommend ICL size. Other solutions use regression models to predict postoperative anterior chamber angle status, for example, using preoperative trabecular iris angle (TIA) and scleral spur angle (SSA) to predict postoperative anterior chamber angle (ACA) and other angle-related parameters.

[0067] However, related technologies can only predict single postoperative parameters, such as arch height and anterior chamber angle, based on some preoperative anterior segment structural parameters. This approach is actually a numerical prediction technique. However, the anterior segment structure is relatively complex, and it is impossible to measure the overall anterior segment structure state after ICL using a single parameter, resulting in low accuracy of anterior segment structure prediction.

[0068] Based on this, embodiments of this application provide a method, apparatus, electronic device, and storage medium for predicting anterior segment structures, aiming to improve the accuracy of anterior segment structure prediction. It should be noted that the anterior segment structure prediction method provided in this application can accurately predict and display anterior segment structure images after ICL surgery, and can analyze preoperative anterior segment structure images to provide corresponding recommended ICL size and recommended ICL implantation direction.

[0069] As those skilled in the art will understand, the ICL size is therefore determined based on the width of the human eye and other relevant factors. The width of the human eye is essentially fixed, and the commonly used ICL sizes include 12.1mm, 12.6mm, 13.2mm, and 13.7mm. The ICL implantation direction includes horizontal and vertical implantation.

[0070] The anterior segment structure prediction method, apparatus, electronic device, and storage medium provided in this application are specifically described through the following embodiments. First, the anterior segment structure prediction method in this application is described.

[0071] The anterior segment structure prediction method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the anterior segment structure prediction method, but is not limited to the above forms.

[0072] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0073] Figure 2 This is an optional flowchart of the anterior segment structure prediction method provided in the embodiments of this application. Figure 2 The method may include, but is not limited to, steps 201 to 206.

[0074] Step 201: Obtain preoperative anterior segmental images of ICL and ICL implantation direction;

[0075] Step 202: Input the preoperative anterior segment structure image and ICL implantation direction into the pre-trained anterior segment structure image generation model;

[0076] Step 203: Encode the preoperative anterior segment structure image and ICL implantation direction using the encoder module to obtain the anterior segment structure image features;

[0077] Step 204: Based on the first auxiliary decoder, the second auxiliary decoder and the geometric information sharing module, generate an iris image from the anterior segment structure image features to obtain the iris segmentation image output by the first auxiliary decoder and the iris contour and lens upper surface image output by the second auxiliary decoder.

[0078] Step 205: Decode the anterior segment structure image features based on the main decoder and the first auxiliary decoder to obtain the anterior segment structure image after ICL surgery;

[0079] Step 206 displays postoperative images of the anterior segment structure, iris segmentation, and iris contour and upper surface of the lens in the direction of ICL implantation.

[0080] In step 201 of some embodiments, the preoperative anterior segment image acquired for ICL can be any modality, including anterior segment-optical coherence tomography (AS-OCT) images and ultrabiomicroscopy (UBM) images; no limitation is imposed here. This application uses AS-OCT images for illustration; please refer to [link to relevant documentation]. Figure 3 , Figure 3 This is a schematic diagram of the AS-OCT modal image provided in the embodiments of this application. Figure 3 The image on the left is a preoperative OCT image of the anterior segment before ICL lens implantation. The positions of the cornea, iris, and lens are marked in the left image. Figure 3 The image on the right is an AS-OCT modal image of the anterior segment structure after ICL lens implantation. It can be seen that the structure of the anterior segment has changed accordingly after the lens is implanted.

[0081] Specifically, the preoperative anterior segment images of ICL include the corresponding ICL implantation direction, i.e., the predicted anterior segment images after ICL surgery under the current ICL implantation direction. Please refer to... Figure 4 , Figure 4 This is a schematic diagram of the input or output image provided in the embodiments of this application. The input image is a preoperative anterior segment structure image of ICL, and the output image is a postoperative anterior segment structure image of ICL. The input preoperative anterior segment structure image of ICL can be unidirectional or multidirectional, and the predicted postoperative anterior segment structure image of ICL can also be unidirectional or multidirectional.

[0082] For example, by inputting a unidirectional preoperative anterior segment image of ICL, the model can be configured to output either multidirectional or unidirectional postoperative anterior segment images of ICL, depending on the requirements. Similarly, by inputting a multidirectional preoperative anterior segment image of ICL, the model can also be configured to output either multidirectional or unidirectional postoperative anterior segment images of ICL, depending on the requirements.

[0083] Figure 4In the diagram, each solid black line represents an anterior segment image at a corresponding angle. This angle is the angle between the projection of a section parallel to the sagittal axis onto the coronal plane and the transverse section. A single-direction image is a cross-sectional view corresponding to a single angle. The black lines represent sections, which are planes parallel to the sagittal axis. The cross-sectional view seen when cut through this section is the preoperative anterior segment image of the ICL or the postoperative anterior segment image of the ICL generated from the model used in this application embodiment. The leftmost image represents a single-angle image, the middle images represent multiple images centered on the insertion direction (taking horizontal ICL implantation as an example) and evenly spaced before and after, and the rightmost image represents images from all directions.

[0084] In some embodiments, acquiring preoperative anterior segmental images of ICL includes:

[0085] The brightness of the preoperative anterior segmental structure image of ICL is adjusted according to the preset brightness adjustment curve to obtain the brightness adaptation image of the preoperative anterior segmental structure of ICL. There are multiple brightness adjustment curves.

[0086] Multiple preoperative anterior segment structure brightness adaptation images with different brightness levels were stitched together to obtain multi-brightness stitched images;

[0087] Based on a preset attention module, feature weights are assigned to the multi-brightness stitched images to obtain the anterior segment structure image before ICL surgery.

[0088] Specifically, please see Figure 5 , Figure 5 This is a schematic diagram of the multi-brightness adaptation module provided in this application embodiment. Five brightness adjustment curves are preset to adjust the brightness of the input preoperative anterior segment structure image for ICL surgery, resulting in a preoperative anterior segment structure brightness-adapted image. The preoperative anterior segment structure brightness-adapted images under different brightness levels can be stitched together to obtain a multi-brightness stitched image. Further, feature weights are assigned to the multi-brightness stitched image based on a preset attention module to obtain the preoperative anterior segment structure image for ICL surgery. The attention module is a Convolutional Block Attention Module (CBAM), including a channel attention module and a spatial attention module. The channel attention module calculates the importance of each channel to distinguish features between different channels. The spatial attention module calculates the spatial importance of each pixel to better capture the spatial structure in the image. This application embodiment, through multiple brightness adjustments and attention module processing, enables the anterior segment structure image generation model to adapt to images of different brightness levels.

[0089] In step 202 of some embodiments, the preoperative anterior segment image of the ICL and the ICL implantation orientation information are input into a pre-trained anterior segment image generation model. The ICL implantation orientation information can be text information, and the input text information can also include ICL size information. Specifically, the pre-trained anterior segment image generation model may include an encoder module, a decoder module, and a geometric information sharing module. The decoder module may include a main decoder, a first auxiliary decoder, and a second auxiliary decoder.

[0090] Please refer to Figure 6 , Figure 6 This is a schematic diagram of the framework of the anterior segment structure image generation model provided in this application embodiment. Specifically, the anterior segment structure image generation model may include an encoder module and a decoder module, and the composition of the anterior segment structure image generation model will be described in detail later.

[0091] In step 203 of some embodiments, the encoder module can be used to encode the preoperative anterior segment structure image of ICL, extract image features, and obtain anterior segment structure image features. Specifically, the encoder module may include multiple downsampling modules, three of which are used as an example here. Each downsampling module may include a convolutional layer, an activation function layer, and a normalization layer. Downsampling reduces image resolution and speeds up the computation process. The anterior segment structure image generation model provided in this application adopts a skip connection design, that is, the first downsampling module is not concatenated with the decoder module. The decoding result output by the main decoder is concatenated with the features output by the corresponding downsampling modules in the encoder module, respectively, to achieve the effect of fusing multi-scale information and improve the accuracy of anterior segment structure prediction.

[0092] In step 204 of some embodiments, an iris image is generated based on the anterior segment structure image features using a first auxiliary decoder, a second auxiliary decoder, and a geometric information sharing module, resulting in an iris segmentation image output by the first auxiliary decoder and an iris contour and lens upper surface image output by the second auxiliary decoder, including but not limited to the following steps:

[0093] The anterior segment structure image features are upsampled by the first upsampling layer to obtain the reconstructed image of the first anterior segment structure features;

[0094] The anterior segment structure image features are upsampled by the second upsampling layer to obtain the reconstructed image of the second anterior segment structure features;

[0095] Geometric information is extracted from the reconstructed images of the first and second anterior segment structures using the geometric information sharing module, resulting in geometric information sharing results.

[0096] The first feature image is obtained by stitching together the reconstructed image of the anterior segment structure features and the anterior segment structure image features through the first stitching layer.

[0097] The second stitching layer stitches together the reconstructed image of the second anterior segment structure features and the anterior segment structure image features to obtain the stitched result of the second feature image.

[0098] The first fusion layer fuses the first feature image stitching result and the geometric information extraction result to obtain the first image fusion result.

[0099] The second fusion layer fuses the second feature image stitching result and the geometric information extraction result to obtain the second image fusion result.

[0100] The first image fusion result is upsampled by the first output layer to output an iris segmentation image;

[0101] The second image fusion result is upsampled by the second output layer to output the iris contour and the upper surface of the lens image.

[0102] Specifically, continue to refer to Figure 6 The decoder module can include three branches: a main decoder branch, a first auxiliary decoder branch, and a second auxiliary decoder branch (not labeled in the figure). The main decoder branch can include multiple main decoders, the first auxiliary decoder branch can include multiple first auxiliary decoders, and the second auxiliary decoder branch can include multiple second auxiliary decoders. The main decoder can include an upsampling module and two splicing layers. The first auxiliary decoder can include a first upsampling layer, a first splicing layer, and a first fusion layer; the second auxiliary decoder can include a second upsampling layer, a second splicing layer, and a second fusion layer. The number of upsampling layers, splicing layers, and fusion layers is not limited and can be set according to actual needs. This embodiment uses two main decoders, two first auxiliary decoders, and two second auxiliary decoders as an example. The first and second auxiliary decoders can guide the anterior segment structure image generation model to focus on important anterior segment structures, such as the iris root, through different network parameter settings.

[0103] Furthermore, the anterior segment structure image features can be upsampled through the first upsampling layer of the first auxiliary decoder to obtain a reconstructed image of the first anterior segment structure features. The reconstructed image of the first anterior segment structure features and the anterior segment structure image features output by the third downsampling module in the encoder module are then concatenated through the first concatenation layer of the first auxiliary decoder to obtain the first feature image concatenation result. In addition, the reconstructed image of the first anterior segment structure features serves as one of the inputs to the geometric information sharing module.

[0104] Then, the anterior segment structure image features can be upsampled using a second upsampling layer to obtain a second anterior segment structure feature reconstruction image. This second anterior segment structure feature reconstruction image is then combined with the anterior segment structure image features output from the third downsampling module in the encoder module through a second stitching layer to obtain the second feature image stitching result. Furthermore, the second anterior segment structure feature reconstruction image serves as another input to the geometric information sharing module.

[0105] In some embodiments, geometric information is extracted from the reconstructed images of the first and second anterior segment structural features using a geometric information sharing module to obtain geometric information sharing results, including but not limited to the following steps:

[0106] The first and second anterior segment structural feature reconstruction images are stitched together to obtain the third feature image stitching result.

[0107] Based on the convolutional layer, feature extraction is performed on the stitched result of the third feature image to obtain the geometric feature extraction result;

[0108] The third feature image stitching result is obtained by performing channel max pooling on the max pooling layer;

[0109] The average pooling result is obtained by performing channel average pooling on the stitching result of the third feature image based on the average pooling layer.

[0110] The results of max pooling, average pooling, and geometric feature extraction are concatenated to obtain the fourth feature image concatenation result.

[0111] By performing a nonlinear transformation on the stitching result of the fourth feature image through the activation layer, a geometric information sharing result is obtained.

[0112] The geometric information sharing module extracts geometric information from the reconstructed images of the first and second anterior segment structures, resulting in geometric information sharing results. These results help the generative model pay more attention to image information with geometric features to distinguish different shapes or structures in the image, such as the location of the iris root.

[0113] Please refer to Figure 7 , Figure 7This is a schematic diagram of the geometric information sharing module provided in this application embodiment. The geometric information sharing module includes convolutional layers, max pooling layers, average pooling layers, and activation layers. The convolutional layers include 1x3 convolutional kernels and 3x1 convolutional kernels. It can be understood that the number of convolutional layers, max pooling layers, average pooling layers, and activation layers in the geometric information sharing module can be set according to actual needs. For example, the number of activation layers and pooling layers can be four, five, or seven, etc., and there is no limitation here. Specifically, the reconstructed image of the first anterior segment structure features and the reconstructed image of the second anterior segment structure features can be stitched together to obtain the third feature image stitching result. The third feature image stitching result can be a feature map obtained by stitching together the feature map output by the first upsampling layer in the first auxiliary decoder and the feature map output by the second upsampling layer in the second auxiliary decoder. The feature map can be represented by a three-dimensional array, which can include three dimensions: height, width, and number of channels. The first auxiliary decoder branch focuses on the features of the iris root endpoint, while the second auxiliary decoder branch focuses on the features of the contours of the iris and the upper surface of the lens.

[0114] Furthermore, the stitched result of the third feature image is input into a convolutional layer. A 1x3 convolutional kernel extracts the horizontal features from the stitched feature image, and a 3x1 convolutional kernel extracts the vertical features. The horizontal and vertical features are then fused pixel-wise to obtain the geometric feature extraction result. This embodiment strengthens the feature representation through two asymmetric convolutional kernels and fuses features of different shapes, which helps to capture the geometric structure in the feature image.

[0115] The stitched result of the third feature image is input into both a max pooling layer and an average pooling layer. In the max pooling layer, channel max pooling is performed on the stitched result to obtain the max pooling result. In the average pooling layer, channel average pooling is performed on the stitched result to obtain the average pooling result. This method helps extract important information from different channels in the feature image while reducing the impact of noise.

[0116] Furthermore, the max-pooling result from the max-pooling layer, the average-pooling result from the average-pooling layer, and the geometric feature extraction result from the convolutional layer are concatenated to obtain the fourth feature image concatenation result. Concatenating feature maps extracted from different sources enhances the model's understanding and representation of images. An activation layer performs a non-linear transformation on the feature maps obtained in the preceding steps—that is, on the fourth feature image concatenation result—to enhance the geometric information in the feature maps or form new feature maps, resulting in a geometric information sharing outcome. The activation function used in the activation layer is the sigmoid function.

[0117] The geometric information sharing result output from the activation layer is multiplied pixel-by-pixel with the reconstructed image of the first anterior segment structure feature to obtain the updated image of the first anterior segment structure feature. Similarly, the geometric information sharing result output from the activation layer is multiplied pixel-by-pixel with the reconstructed image of the second anterior segment structure feature to obtain the updated image of the second anterior segment structure feature. The updated image of the first anterior segment structure feature is then concatenated back to the first auxiliary decoder branch, i.e., input to the first fusion layer of the first auxiliary decoder. Likewise, the updated image of the second anterior segment structure feature is concatenated back to the second auxiliary decoder branch, i.e., input to the second fusion layer of the second auxiliary decoder. It is understood that if there are multiple first auxiliary decoders, second auxiliary decoders, and geometric information sharing modules, the aforementioned steps can be repeated multiple times.

[0118] In some embodiments, the reconstructed image of the first anterior segment structure features and the anterior segment structure image features are stitched together using a first stitching layer to obtain a first feature image stitching result; the reconstructed image of the second anterior segment structure features and the anterior segment structure image features are stitched together using a second stitching layer to obtain a second feature image stitching result. For details, please refer to... Figure 6 The updated image of the first anterior segment structure features output by the geometric information sharing module is input to the first fusion layer of the first decoder. The other input of the first fusion layer is the stitching result of the first feature image. The first fusion layer multiplies the updated image of the first anterior segment structure features and the stitching result of the first feature image pixel by pixel to obtain the first image fusion result.

[0119] Furthermore, the updated image of the second anterior segment structure features output by the geometric information sharing module is input to the second fusion layer of the second decoder. Another input to the second fusion layer is the stitched result of the second feature image. The second fusion layer multiplies the updated image of the second anterior segment structure features and the stitched result of the second feature image pixel-by-pixel to obtain the second image fusion result. The first image fusion result output by the first fusion layer is input to the second stitching layer of the main decoder and stitched with the feature map obtained from the main decoder branch. This ensures that when generating anterior segment structure images after ICL surgery, the focus is on the iris root position, improving the model's predictive performance.

[0120] In step 205 of some embodiments, the anterior segment structure image features are decoded based on the main decoder and the first auxiliary decoder to obtain an anterior segment structure image after ICL surgery. The main decoder includes an upsampling layer and two stitching layers. The upsampling module is used to upsample the anterior segment structure image features output by the encoder module, and the upsampled result is stitched together with the anterior segment structure image features output by the third downsampling module in the encoder module based on the stitching layer in the main decoder. The feature information lost during downsampling is recovered through feature stitching. Further, the feature stitching result output by the first stitching layer and the updated image of the first anterior segment structure features output by the geometric information sharing module are input into the second stitching layer of the main decoder to guide the main decoder to generate an anterior segment structure feature map focusing on the position of the iris root.

[0121] In some embodiments, an iris image is generated based on the anterior segment structure image features obtained from a first auxiliary decoder, a second auxiliary decoder, and a geometric information sharing module. Specifically, the anterior segment structure image generation model may further include a first output layer corresponding to the first auxiliary decoder branch. The first output layer is also an upsampling layer, and the output of the first auxiliary decoder is upsampled through the first output layer to output a high-resolution image of the iris root position. The anterior segment structure image generation model may further include a second output layer corresponding to the second auxiliary decoder branch. The second output layer is also an upsampling layer, and the output of the second auxiliary decoder is upsampled through the second output layer to output a high-resolution iris contour and lens image. In addition, the main branch containing the main decoder may also include a corresponding target output layer, which is used to upsample the anterior segment structure feature map output by the main decoder to output a high-resolution anterior segment structure image after ICL surgery.

[0122] In some embodiments, before acquiring preoperative anterior segment images of ICL, the method may further include: training an anterior segment image generation model, wherein the training process specifically includes:

[0123] Acquire an image sample set and ICL implantation orientation. The image sample set includes multiple anterior segment structure image samples.

[0124] Anterior segment structure image samples and ICL implantation direction are input into anterior segment structure image generation model for image prediction, resulting in postoperative anterior segment structure prediction image, iris segmentation prediction image, and iris contour and lens prediction image of ICL implantation direction.

[0125] The predicted images of the anterior segment structure after ICL surgery are input into a preset anterior segment structure prediction discriminator for image discrimination, and the image discrimination results are obtained. The adversarial loss is determined based on the image discrimination results.

[0126] A first loss is determined based on the iris segmentation prediction image and the reference iris segmentation image, and a second loss is determined based on the iris contour and lens prediction image and the reference iris contour and lens image.

[0127] The target loss is determined based on the first loss, the second loss, and the adversarial loss, and the model parameters of the anterior segment structure image generation model are updated based on the target loss.

[0128] Specifically, please see Figure 8 , Figure 8 This is a training block diagram of the anterior segment structure image generation model provided in this application embodiment. An image sample set and ICL implantation direction are acquired. The image sample set includes multiple anterior segment structure image samples for training the anterior segment structure image generation model. These multiple anterior segment structure image samples include paired pre-ICL anterior segment structure image samples and post-ICL anterior segment structure image samples. The pre-ICL anterior segment structure image samples are input into the anterior segment structure image generation model for image prediction, resulting in a predicted post-ICL anterior segment structure image, an iris segmentation prediction image, and an iris contour and lens prediction image under the corresponding ICL implantation direction. The iris contour and lens prediction image is a predicted image of the iris contour and the upper surface of the lens. This application embodiment uses a generative adversarial network (GAN) for post-operative anterior segment structure prediction. The GAN consists of a generator and a discriminator. The anterior segment structure image generation model is the generator, responsible for generating post-operative anterior segment structure images, and the anterior segment structure prediction discriminator is responsible for determining whether the image is a real image or a generated image. The predicted anterior segment structure image after ICL surgery is input into a pre-defined anterior segment structure prediction discriminator for image discrimination. The discriminator compares the generated predicted anterior segment structure image with the corresponding ICL anterior segment structure image sample to obtain the image discrimination result. Then, adversarial loss is determined based on the image discrimination result. As the adversarial loss decreases, the generator can generate data more similar to the target domain data, and the discriminator can improve its ability to distinguish between generated data and original data. The adversarial loss is denoted as L. adv (G,D), the specific expression is as follows:

[0129] L adv (G,D)=E[log(D(x))]+E[log(1-D(G(x)))]

[0130] Where x represents the input preoperative anterior segment image sample of ICL, G represents the generator, D represents the discriminator, G(x) represents the postoperative anterior segment image of ICL generated by the anterior segment image generation model based on the input data x, and D(x) represents the discrimination result of the anterior segment structure prediction discriminator on the input data x. The judgment result of D(x) can include: the input data x is real data, or the input data x is generated data; D(G(x)) represents the discrimination result of the anterior segment structure prediction discriminator on the input data G(x). The judgment result of D(G(x)) can include: the input data G(x) is real data, or the input data G(x) is generated data.

[0131] Furthermore, the reference iris segmentation image can be used as the output label of the anterior segment structure image generation model, i.e., the iris segmentation image label. The first loss is determined based on the iris segmentation prediction image and the reference iris segmentation image. The reference iris contour and lens image is used as another output label of the anterior segment structure image generation model. Then, the second loss can be determined based on the iris contour and lens prediction image and the reference iris contour and lens image. Here, the iris contour and lens prediction image represents the iris contour and lens upper surface image, and the reference iris contour and lens image represents the reference iris contour and lens upper surface image, i.e., the iris contour and lens upper surface image label.

[0132] In some embodiments, determining a first loss based on an iris segmentation prediction image and a reference iris segmentation image includes:

[0133] Obtain the preset weighted map of the iris root position;

[0134] The binary cross-entropy loss is determined based on the iris segmentation prediction image and the reference iris segmentation image;

[0135] The first loss is obtained by adjusting the weights of the binary cross-entropy loss based on the weighted map of the iris root position.

[0136] Specifically, the iris segmentation prediction image is represented as x I ′ The reference iris segmentation image is represented as y I According to x I ′ and y I Determine the binary cross-entropy loss L BCE (x I ′ ,y I Obtain the preset iris root position weight map W. HThe iris root position weight map is a pseudo-color image that demonstrates how iris root position information is applied. The pseudo-color image contains two Gaussian peaks, and the center of the Gaussian peaks (displayed as yellow dots) represents the iris root position. This image is used as a weight map to increase the weight of the iris root position in the first auxiliary decoder branch, thereby improving the accuracy of iris prediction based on endpoints.

[0137] According to the weighted map of the iris root position W H For binary cross-entropy loss L BCE (x I ′ ,y I After weight adjustment, the first loss is obtained. in, This indicates pixel-by-pixel multiplication, and the first loss represents the loss function of the iris region prediction auxiliary branch (first auxiliary decoder branch). Further, based on the iris contour and the lens prediction image x... ′ C and a preset reference iris profile and lens image y C The second loss L is determined as input. Dice (x ′ C ,y C The second loss function is the Dice similarity coefficient loss function, which represents the loss function of the iris contour branch (second auxiliary decoder branch).

[0138] Based on the foregoing steps, the target loss L in this embodiment of the application is as follows:

[0139]

[0140] Among them, L adv (G,D) represents the resistance loss. Indicates the first loss, L Dice (x ′ C ,y C ) represents the second loss, λ I and λ C The preset weight values ​​are used; the higher the weight value, the greater the influence of that branch on the network. As the target loss L decreases, the model parameters of the anterior segment structure image generation model are updated based on the target loss L, gradually improving the prediction accuracy of the anterior segment structure prediction model.

[0141] In some embodiments, in addition to inputting the preoperative anterior segment structure image and ICL implantation direction into the anterior segment structure image generation model, the input information may also include the ICL size. The method provided in this application embodiment may also include:

[0142] The preoperative anterior segment structure image of ICL is input into the anterior segment structure image generation model to generate the postoperative anterior segment structure image of ICL at the corresponding ICL size.

[0143] Recommendations for ICL implantation direction are made based on postoperative anterior segmental images of ICL and the corresponding ICL implantation direction.

[0144] ICL size recommendations are made based on postoperative anterior segment images and corresponding ICL dimensions.

[0145] Specifically, this application provides three recommended ICL sizes and ICL implantation orientations. The anterior segment structure prediction model provided in this application can take two ICL implantation orientations, four ICL sizes, and preoperative anterior segment structure images as input, and output eight combinations of postoperative anterior segment structure images. These eight combinations are individual combinations of ICL implantation orientations and ICL sizes. Based on the postoperative anterior segment structure image under each combination, the postoperative anterior segment structure image can be unidirectional or multidirectional, which can be set by the user, and displays the final recommended ICL implantation orientation and size scheme; or it can output the recommended ICL size and recommended ICL implantation orientation corresponding to the postoperative anterior segment structure image, and display the recommended ICL size and recommended ICL implantation orientation; or it can only output the text corresponding to the recommended ICL size and recommended ICL implantation orientation.

[0146] Please see Figure 9 This application also provides an anterior segment structure prediction device 900, which can implement the above-described anterior segment structure prediction method. The device includes:

[0147] Acquisition unit 910 is used to acquire preoperative anterior segmental structure images and ICL implantation orientation; input unit 920 is used to input the preoperative anterior segmental structure images and ICL implantation orientation into a pre-trained anterior segmental structure image generation model, which includes an encoder module, a decoder module, and a geometric information sharing module. The decoder module includes a main decoder, a first auxiliary decoder, and a second auxiliary decoder; encoding unit 930 is used to encode the preoperative anterior segmental structure images and ICL implantation orientation through the encoder module to obtain anterior segmental structure image features; generation unit 920 is used to generate the generation unit 930. 940 is used to generate an iris image based on the features of the anterior segment structure image using the first auxiliary decoder, the second auxiliary decoder, and the geometric information sharing module, to obtain an iris segmentation image output by the first auxiliary decoder and an iris contour and lens upper surface image output by the second auxiliary decoder; decoding unit 950 is used to decode the features of the anterior segment structure image based on the main decoder and the first auxiliary decoder to obtain an anterior segment structure image after ICL surgery; display unit 960 is used to display the anterior segment structure image, iris segmentation image, and iris contour and lens upper surface image after ICL surgery in the ICL implantation direction.

[0148] The specific implementation of this anterior segment structure prediction device is basically the same as the specific embodiment of the anterior segment structure prediction method described above, and will not be repeated here.

[0149] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described anterior segment structure prediction method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0150] Please see Figure 10 , Figure 10 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0151] The processor 1001 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0152] The memory 1002 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1002 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1002 and is called and executed by the processor 1001 using the anterior segment structure prediction method of the embodiments of this application.

[0153] Input / output interface 1003 is used to implement information input and output;

[0154] The communication interface 1004 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0155] Bus 1005 transmits information between various components of the device (e.g., processor 1001, memory 1002, input / output interface 1003, and communication interface 1004);

[0156] The processor 1001, memory 1002, input / output interface 1003 and communication interface 1004 are connected to each other within the device via bus 1005.

[0157] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described anterior segment structure prediction method.

[0158] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0159] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0160] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0161] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0162] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0163] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0164] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0165] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0166] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0167] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0168] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0169] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for predicting anterior segment structure, characterized in that, The method includes: Obtain preoperative anterior segmental images of ICL and ICL implantation direction; The preoperative anterior segment structure image of the ICL and the ICL implantation direction are input into a pre-trained anterior segment structure image generation model. The anterior segment structure image generation model includes an encoder module, a decoder module, and a geometric information sharing module. The decoder module includes a main decoder, a first auxiliary decoder, and a second auxiliary decoder. The encoder module encodes the preoperative anterior segment structure image of the ICL and the ICL implantation direction to obtain the anterior segment structure image features; Based on the first auxiliary decoder, the second auxiliary decoder and the geometric information sharing module, the anterior segment structure image features are used to generate an iris image, resulting in an iris segmentation image output by the first auxiliary decoder and an iris contour and lens upper surface image output by the second auxiliary decoder. Based on the main decoder and the first auxiliary decoder, the anterior segment structure image features are decoded to obtain the anterior segment structure image after ICL surgery; The image displays the anterior segment structure of the ICL after surgery, the iris segmentation image, and the iris contour and lens upper surface image under the ICL implantation direction.

2. The prediction method according to claim 1, characterized in that, The first auxiliary decoder includes a first upsampling layer, a first stitching layer, and a first fusion layer; the second auxiliary decoder includes a second upsampling layer, a second stitching layer, and a second fusion layer; the anterior segment structure image generation model further includes a first output layer and a second output layer; the step of generating an iris image based on the features of the anterior segment structure image using the first auxiliary decoder, the second auxiliary decoder, and the geometric information sharing module, to obtain an iris segmentation image output by the first auxiliary decoder and an iris contour and lens upper surface image output by the second auxiliary decoder, includes: The anterior segment structure image features are upsampled by the first upsampling layer to obtain a reconstructed image of the first anterior segment structure features; The anterior segment structure image features are upsampled by the second upsampling layer to obtain a reconstructed image of the second anterior segment structure features; Geometric information is extracted from the reconstructed images of the first and second anterior segment structures using a geometric information sharing module to obtain geometric information sharing results. The first feature image is obtained by stitching together the reconstructed image of the anterior segment structure features and the anterior segment structure image features through the first stitching layer; The second stitching layer stitches together the reconstructed image of the second anterior segment structure features and the anterior segment structure image features to obtain the stitched result of the second feature image; The first image fusion result is obtained by fusing the first feature image stitching result and the geometric information extraction result through the first fusion layer. The second fusion layer fuses the second feature image stitching result and the geometric information extraction result to obtain the second image fusion result. The first image fusion result is upsampled by the first output layer to output an iris segmentation image; The second image fusion result is upsampled by the second output layer to output the iris contour and the upper surface of the lens image.

3. The prediction method according to claim 2, characterized in that, The geometric information sharing module includes convolutional layers, average pooling layers, max pooling layers, and activation layers. The geometric information sharing module extracts geometric information from the reconstructed images of the first and second anterior segment structures to obtain geometric information sharing results, including: The first anterior segment structural feature reconstruction image and the second anterior segment structural feature reconstruction image are stitched together to obtain the third feature image stitching result; Based on the convolutional layer, feature extraction is performed on the stitched result of the third feature image to obtain the geometric feature extraction result; Based on the max pooling layer, channel max pooling is performed on the stitching result of the third feature image to obtain the max pooling result; Based on the average pooling layer, channel average pooling is performed on the stitching result of the third feature image to obtain the average pooling result; The max pooling result, the average pooling result, and the geometric feature extraction result are concatenated to obtain the fourth feature image concatenation result. The activation layer performs a nonlinear transformation on the stitching result of the fourth feature image to obtain the geometric information sharing result.

4. The prediction method according to claim 1, characterized in that, Before acquiring the preoperative anterior segment structure image of ICL, the method further includes: training the anterior segment structure image generation model, the training process specifically including: Acquire an image sample set and ICL implantation direction, wherein the image sample set includes multiple anterior segment structure image samples, reference iris segmentation images, and reference iris contour and lens images; The anterior segment structure image sample and the ICL implantation direction are input into the anterior segment structure image generation model for image prediction, to obtain the postoperative anterior segment structure prediction image, iris segmentation prediction image and iris contour and lens prediction image under the ICL implantation direction; The predicted anterior segment structure image after ICL surgery is input into a preset anterior segment structure prediction discriminator for image discrimination, and the image discrimination result is obtained. The adversarial loss is determined based on the image discrimination result. A first loss is determined based on the iris segmentation prediction image and the reference iris segmentation image, and a second loss is determined based on the iris contour and lens prediction image and the reference iris contour and lens image. The target loss is determined based on the first loss, the second loss, and the adversarial loss, and the model parameters of the anterior segment structure image generation model are updated based on the target loss.

5. The prediction method according to claim 4, characterized in that, The step of determining the first loss based on the iris segmentation prediction image and the reference iris segmentation image includes: Obtain the preset weighted map of the iris root position; The binary cross-entropy loss is determined based on the iris segmentation prediction image and the reference iris segmentation image; The first loss is obtained by adjusting the weights of the binary cross-entropy loss based on the weighted map of the iris root position.

6. The prediction method according to claim 1, characterized in that, The acquisition of preoperative anterior segmental images for ICL surgery includes: The brightness of the input preoperative anterior segment structure image of ICL is adjusted according to a preset brightness adjustment curve to obtain a preoperative anterior segment structure brightness adaptation image of ICL. There are multiple brightness adjustment curves. Multiple preoperative anterior segment structure brightness adaptation images with different brightness levels of the ICL were stitched together to obtain a multi-brightness stitched image. Based on a preset attention module, feature weights are assigned to the multi-brightness stitched image to obtain the preoperative anterior segment structure image of the ICL.

7. The method according to claim 1, characterized in that, The preoperative anterior segmental image of the ICL also includes the corresponding ICL size, and the method further includes: The preoperative anterior segment structure image of the ICL is input into the anterior segment structure image generation model to generate a postoperative anterior segment structure image of the ICL at the corresponding ICL size. Recommendations for ICL implantation direction are made based on the postoperative anterior segmental structure images of ICL and the corresponding ICL implantation direction. ICL size recommendations are made based on the anterior segment structure images and corresponding ICL sizes after ICL surgery.

8. A device for predicting the structure of the anterior segment, characterized in that, The device includes: The acquisition unit is used to acquire preoperative anterior segmental images of ICL and the ICL implantation direction. The input unit is used to input the preoperative anterior segment structure image of the ICL and the ICL implantation direction into a pre-trained anterior segment structure image generation model. The anterior segment structure image generation model includes an encoder module, a decoder module and a geometric information sharing module. The decoder module includes a main decoder, a first auxiliary decoder and a second auxiliary decoder. The encoding unit is used to encode the preoperative anterior segment structure image of the ICL and the ICL implantation direction through the encoder module to obtain the anterior segment structure image features; The generation unit is used to generate an iris image based on the anterior segment structure image features of the first auxiliary decoder, the second auxiliary decoder and the geometric information sharing module, to obtain an iris segmentation image output by the first auxiliary decoder and an iris contour and lens upper surface image output by the second auxiliary decoder. A decoding unit is used to decode the anterior segment structure image features based on the main decoder and the first auxiliary decoder to obtain an anterior segment structure image after ICL surgery. The display unit is used to display the postoperative anterior segment structure image of the ICL, the iris segmentation image, and the iris contour and lens upper surface image under the ICL implantation direction.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the anterior segment structure prediction method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the anterior segment structure prediction method according to any one of claims 1 to 7.

Images