Tracking-based anterior segment image processing method and system, device and medium

Abstract

Embodiments of the present application provide an ocular anterior segment image processing method and system based on tracking, equipment and medium. The method comprises: acquiring ocular anterior segment image data collected by an ocular anterior segment of a target object, the ocular anterior segment comprising an ocular anterior segment structure; performing modal classification on the ocular anterior segment image data to obtain a target modal; if the target modal indicates that the ocular anterior segment image data at least comprises a two-dimensional static ocular anterior segment image, performing segmentation and positioning on the two-dimensional static ocular anterior segment image to obtain segmentation data and positioning data of the ocular anterior segment structure; if the target modal indicates that the ocular anterior segment image data comprises a two-dimensional static ocular anterior segment image and a motion ocular anterior segment image video, performing tracking based on the segmentation data, the positioning data, and the motion ocular anterior segment image video to obtain a motion trajectory of the ocular anterior segment structure; and performing parameter quantification based on the segmentation data, the positioning data, and the motion trajectory of the ocular anterior segment structure to obtain ocular anterior segment parameter data. The present application can improve the accuracy of the evaluation of the ocular anterior segment structure.

Description

Technical Field

[0001] This application relates to the field of image processing, and more particularly to a tracking-based anterior segment image processing method and system, electronic device, and storage medium. Background Technology

[0002] Existing computer-aided diagnosis (CAD) systems in ophthalmology can process anterior segment image data, but they are primarily designed for two-dimensional static image data (such as data acquired from fundus photography, corneal topography, and anterior eye photography). While two-dimensional static image data is easy to process, it contains less information, and errors may exist between the captured two-dimensional static image data due to factors such as shooting angle and lighting conditions. These errors can lead to inaccurate assessments of anterior segment structures based on planar static image data, thus affecting the doctor's diagnosis. Summary of the Invention

[0003] The main objective of this application is to propose a tracking-based anterior segment image processing method and system, electronic device and storage medium, which aims to improve the accuracy of anterior segment structure assessment, thereby providing doctors with more accurate judgment criteria.

[0004] To achieve the above objectives, a first aspect of this application proposes a tracking-based anterior segment image processing method, the method comprising:

[0005] Acquire anterior segment image data of a target object by acquiring the anterior segment of the target object using a target acquisition method; the target acquisition method includes at least one of two-dimensional image acquisition and video acquisition, and the anterior segment includes the anterior segment structure;

[0006] Modality classification is performed on the anterior segment image data to obtain a target modality; the target modality indicates that the anterior segment image data includes at least one of two-dimensional static anterior segment image images and moving anterior segment image videos.

[0007] If the target modality indicates that the anterior segment image data includes at least a two-dimensional static anterior segment image, the two-dimensional static anterior segment image is segmented and located to obtain segmentation data and location data of the anterior segment structure.

[0008] If the target modality indicates that the anterior segment image data includes two-dimensional static anterior segment image images and moving anterior segment image videos, then the anterior segment structure is tracked based on the segmentation data, the localization data, and the moving anterior segment image videos to obtain the anterior segment structure motion trajectory;

[0009] Perform parameter quantification based on the segmentation data, the positioning data, and the motion trajectory of the anterior segment structure to obtain anterior segment parameter data.

[0010] In some embodiments, the segmentation and positioning of the two-dimensional static anterior segment image are performed to obtain segmentation data and positioning data of the anterior segment structure, including:

[0011] Encode the two-dimensional static anterior segment image into a high-dimensional hidden space through a first encoder to obtain first encoding features;

[0012] Perform element weighting on elements of the first encoding features through a first optimization model to obtain first weighted features;

[0013] Perform decoding processing on the first weighted features through a first decoder to obtain the segmentation data;

[0014] Input the two-dimensional static anterior segment image, the segmentation data, and the first weighted features into a first positioning module for positioning processing to obtain the positioning data.

[0015] In some embodiments, the motion anterior segment image video includes a motion process of the anterior segment structure in a target environment;

[0016] The tracking of the anterior segment structure based on the segmentation data, the positioning data, and the motion anterior segment image video to obtain a motion trajectory of the anterior segment structure includes:

[0017] Extract every adjacent two image frames in a time sequence order from all image frames in the motion anterior segment image video to obtain a first image frame and a second image frame;

[0018] Determine position change information of the anterior segment based on the segmentation data, the positioning data, the first image frame, and the second image frame;

[0019] Obtain the motion trajectory of the anterior segment structure based on each position change information.

[0020] In some embodiments, the determination of the position change information of the anterior segment based on the segmentation data, the positioning data, the first image frame, and the second image frame includes:

[0021] Perform encoding processing on the first image frame through a second encoder to obtain second encoding features;

[0022] Perform encoding processing on the second image frame through a third encoder to obtain third encoding features;

[0023] Perform element weighting on elements of the second encoding features through a second optimization module to obtain second weighted features;

[0024] The third optimization module is used for element weighting on the elements of the third encoded feature, to obtain a third weighted feature;

[0025] The second decoder is used for motion field estimation on the second weighted feature and the third weighted feature, to obtain a motion field;

[0026] The second positioning module is used for position tracking processing on the motion field, the segmentation data, and the positioning data, to determine position change information of the anterior segment.

[0027] In some embodiments, the second optimization module is used for element weighting on the elements of the second encoded feature, to obtain a second weighted feature, including:

[0028] The second optimization module is used for correlation calculation on the elements of the second encoded feature by using an attention mechanism, to obtain an attention weight;

[0029] The second weighted feature is obtained by multiplying calculation on the attention weight and the elements of the second encoded feature.

[0030] In some embodiments, the target acquisition mode further includes three-dimensional image acquisition, and if the target modality indicates that the anterior segment image data further includes a three-dimensional static anterior segment image, after the two-dimensional static anterior segment image is segmented and positioned to obtain segmentation data and positioning data of the anterior segment structure, the method further includes:

[0031] A three-dimensional reconstruction is performed based on the three-dimensional static anterior segment image, to obtain an anterior segment three-dimensional model;

[0032] The anterior segment three-dimensional model is segmented and calibrated based on the segmentation data and the positioning data, to obtain an anterior segment structure three-dimensional model;

[0033] A volume calculation is performed on the anterior segment structure three-dimensional model, to obtain an anterior segment volume parameter, which is one of the anterior segment parameter data.

[0034] In some embodiments, after the parameter quantization is performed based on the segmentation data, the positioning data, and the anterior segment structure motion trajectory to obtain anterior segment parameter data, the method further includes:

[0035] A disease prediction is performed based on the parameter data, to obtain a disease prediction result, the parameter data including static parameters and dynamic parameters, including:

[0036] A preset index is compared with the dynamic parameters, to obtain a comparison result;

[0037] performing analysis processing based on the comparison result, to obtain a first prediction result;

[0038] performing prediction based on the dynamic parameter and the static parameter, to obtain a second prediction result;

[0039] performing summarization on the first prediction result and the second prediction result, to obtain the disease prediction result.

[0040] To achieve the above object, a second aspect of the embodiment of the present application provides an ocular anterior segment image processing system based on tracking, which comprises:

[0041] a data acquisition module configured to acquire ocular anterior segment image data of an ocular anterior segment of a target object collected in a target collection manner; the target collection manner comprises at least one of two-dimensional image collection and video collection, and the ocular anterior segment comprises an ocular anterior segment structure;

[0042] a modality determination module configured to perform modality classification on the ocular anterior segment image data, to obtain a target modality; the target modality indicates that the ocular anterior segment image data comprises at least one of a two-dimensional static ocular anterior segment image and a motion ocular anterior segment video;

[0043] a first processing module configured to, if the target modality indicates that the ocular anterior segment image data comprises at least a two-dimensional static ocular anterior segment image, perform segmentation and positioning on the two-dimensional static ocular anterior segment image, to obtain segmentation data and positioning data of the ocular anterior segment structure;

[0044] a second processing module configured to, if the target modality indicates that the ocular anterior segment image data comprises a two-dimensional static ocular anterior segment image and a motion ocular anterior segment video, perform tracking on the ocular anterior segment structure based on the segmentation data, the positioning data, and the motion ocular anterior segment video, to obtain an ocular anterior segment structure motion trajectory;

[0045] a parameter quantification module configured to perform parameter quantification based on the segmentation data, the positioning data, and the ocular anterior segment structure motion trajectory, to obtain ocular anterior segment parameter data.

[0046] To achieve the above object, a third aspect of the embodiment of the present application provides an electronic device, which comprises a memory and a processor; the memory stores a computer program; and the processor implements the method of the first aspect when executing the computer program.

[0047] To achieve the above object, a fourth aspect of the embodiment of the present application provides a computer readable storage medium, which stores a computer program; and the computer program is executed by a processor to implement the method of the first aspect.

[0048] This application proposes a tracking-based anterior segment image processing method, system, electronic device, and storage medium. It acquires anterior segment image data obtained by target acquisition of the anterior segment of a target object. The target acquisition method includes at least one of two-dimensional image acquisition and video acquisition, providing a data foundation for subsequent processing. Furthermore, the anterior segment image data is modally classified to obtain target modalities, facilitating the selection of different processing methods for different modalities, thus improving flexibility and adaptability. Further, if the target modality indicates that the anterior segment image data includes at least two-dimensional static anterior segment images, the two-dimensional static anterior segment images are segmented and localized to obtain segmentation data and localization data of the anterior segment structure. If the target modality indicates that the anterior segment image data includes two-dimensional static anterior segment images and moving anterior segment video, the anterior segment structure is tracked based on the segmentation data, localization data, and moving anterior segment video to obtain the anterior segment structure's motion trajectory. Information obtained from the two-dimensional static anterior segment images can be used as prior knowledge to assist subsequent tracking processing, simplifying processing steps, improving processing efficiency, and saving computational resources and time. Furthermore, by quantifying parameters based on segmentation data, localization data, and the anterior segment structure's motion trajectory, anterior segment parameter data is obtained. This allows for the introduction of parameters related to the anterior segment's motion process, providing data showcasing structural changes from multiple angles and at multiple times, avoiding errors caused by factors such as shooting angle and single time period in two-dimensional static image data. This approach improves the accuracy of anterior segment structure assessment by acquiring more accurate and comprehensive parameter data, thus providing doctors with more accurate diagnostic criteria. Attached Figure Description

[0049] Figure 1 This is a flowchart of the tracking-based anterior segment image processing method provided in the embodiments of this application;

[0050] Figure 2 yes Figure 1 Flowchart of step S103;

[0051] Figure 3 This is a schematic diagram of the processing procedure of the anterior segment segmentation and localization model applied in the tracking-based anterior segment image processing method provided in the embodiments of this application;

[0052] Figure 4 yes Figure 1 Flowchart of step S104;

[0053] Figure 5 yes Figure 4 Flowchart of step S402;

[0054] Figure 6is a processing process schematic diagram of an anterior segment structure tracking model applied in the tracking-based anterior segment image processing method provided by the embodiment of the present application;

[0055] Figure 7 is Figure 5 a flowchart of step S503 in the method;

[0056] Figure 8 is Figure 1 another flowchart of step S103 in the method;

[0057] Figure 9 is Figure 1 another flowchart of step S105 in the method;

[0058] Figure 10 is a flowchart of a computer-aided diagnosis system applied in the tracking-based anterior segment image processing method provided by the embodiment of the present application;

[0059] Figure 11 is a flowchart of an iris state prediction in the tracking-based anterior segment image processing method provided by the embodiment of the present application;

[0060] Figure 12 is a structural schematic diagram of the tracking-based anterior segment image processing system provided by the embodiment of the present application;

[0061] Figure 13 is a hardware structural schematic diagram of the electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0063] It should be noted that although the functional modules are divided in the system schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from the module division in the system or the order in the flowchart. The terms "first", "second", etc. in the specification and claims and the above drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0065] First, the several terms involved in the present application are analyzed:

[0066] Computer Aided Diagnosis (CAD) refers to automatically processing and analyzing ophthalmic image data by a computer, calculating various structure parameters, and providing preliminary diagnosis suggestions for doctors to assist doctors in disease diagnosis, thereby improving the diagnosis efficiency. At present, the CAD system has been widely used in the field of ophthalmic medicine, such as glaucoma, cataract, diabetic retinopathy, etc. The use of the CAD system greatly improves the diagnosis efficiency and accuracy of ophthalmologists, and makes an important contribution to early screening and timely detection of diseases.

[0067] Artificial intelligence (AI): It is a new technical science of studying, developing, simulating, extending and expanding human intelligence, and its application system. Artificial intelligence is a branch of computer science. Artificial intelligence attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. The research in this field includes robots, language recognition, image recognition, natural language processing and expert systems. Artificial intelligence can simulate the information process of human consciousness and thinking. Artificial intelligence is also the theory, method, technology and application system of using digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0068] The existing computer-aided diagnosis system for ophthalmic medical treatment can process anterior segment image data, but is mainly designed for two-dimensional static image data (such as fundus photography, corneal topography, and data obtained by anterior segment photography). Although two-dimensional static image data is easy to process, it contains less information, and there may be errors between the two-dimensional static image data taken under the influence of factors such as shooting angle and light condition, which will lead to inaccurate evaluation of the anterior segment structure based on the two-dimensional static image data, thereby affecting the doctor's diagnosis.

[0069] Therefore, the embodiment of the present application provides an anterior segment image processing method and system based on tracking, an electronic device and a storage medium, which aims to improve the evaluation accuracy of the anterior segment structure, thereby providing more accurate judgment basis for doctors.

[0070] The anterior segment image processing method and system based on tracking, the electronic device and the storage medium provided by the embodiment of the present application are specifically explained by the following embodiments. First, the anterior segment image processing method based on tracking in the embodiment of the present application is described.

[0071] The embodiments of the present application can acquire and process related data based on artificial intelligence technology. The artificial intelligence (AI) is a theory, method, technology and application system for simulating, extending and expanding human intelligence by using a digital computer or a machine controlled by a digital computer, perceiving an environment, acquiring knowledge and using the knowledge to obtain optimal results.

[0072] The artificial intelligence basic technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. The artificial intelligence software technology mainly includes computer vision technology, robot technology, biometric identification technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0073] The tracking-based anterior segment image processing method provided by the embodiments of the present application relates to the field of image processing. The tracking-based anterior segment image processing method provided by the embodiments of the present application can be applied in a terminal, can be applied in a server end, and can also be software running in a terminal or a server end. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc.; the server end can be configured as an independent physical server, can be configured as a server cluster or a distributed system composed of multiple physical servers, can also be configured as a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and big data and artificial intelligence platform; and the software can be an application for implementing the tracking-based anterior segment image processing method, etc., but is not limited to the above forms.

[0074] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0075] It should be noted that in various specific embodiments of the present application, when relevant processing needs to be performed on data related to the identity or characteristics of the object, such as object information, object behavior data, object historical data, and object location information, the permission or consent of the object is obtained first, and the collection, use, and processing of such data comply with relevant laws, regulations, and standards. In addition, when the embodiments of the present application need to obtain sensitive personal information of the object, the separate permission or separate consent of the object is obtained through a pop-up window or a jump to a confirmation page, and after obtaining the separate permission or separate consent of the object, the necessary object-related data for enabling the embodiments of the present application to normally operate is obtained.

[0076] Please refer to Figure 1 , Figure 1 is an optional flowchart of an anterior segment image processing method based on tracking provided by the embodiments of the present application, Figure 1 The method in the flowchart can include but is not limited to steps S101 to S105:

[0077] Step S101, acquiring anterior segment image data obtained by collecting the anterior segment of a target object in a target collection manner; the target collection manner includes at least one of two-dimensional image collection and video collection, and the anterior segment includes an anterior segment structure;

[0078] Step S102, performing modality classification on the anterior segment image data to obtain a target modality; the target modality indicates that the anterior segment image data includes at least one of a two-dimensional static anterior segment image and a motion anterior segment video;

[0079] Step S103, if the target modality indicates that the anterior segment image data includes at least a two-dimensional static anterior segment image, performing segmentation and positioning on the two-dimensional static anterior segment image to obtain segmentation data and positioning data of the anterior segment structure;

[0080] Step S104, if the target modality indicates that the anterior segment image data includes a two-dimensional static anterior segment image and a motion anterior segment video, performing tracking on the anterior segment structure based on the segmentation data, the positioning data, and the motion anterior segment video to obtain a motion trajectory of the anterior segment structure;

[0081] Step S105, performing parameter quantification based on the segmentation data, the positioning data, and the motion trajectory of the anterior segment structure to obtain anterior segment parameter data.

[0082] The steps S101 to S105 shown in the embodiments of the present application, by acquiring the anterior segment image data collected by the target acquisition manner to the target object, wherein the target acquisition manner includes at least one of two-dimensional image acquisition and video acquisition, which can provide data basis for subsequent processing. Further, the anterior segment image data is modality classified to obtain the target modality, so as to select different processing methods for different modalities, improve flexibility and adaptability. Further, if the target modality indicates that the anterior segment image data at least includes two-dimensional static anterior segment image, the two-dimensional static anterior segment image is segmented and positioned to obtain segmentation data and positioning data of the anterior segment structure; if the target modality indicates that the anterior segment image data includes two-dimensional static anterior segment image and motion anterior segment image video, the anterior segment structure is tracked based on the segmentation data, positioning data and motion anterior segment image video to obtain the anterior segment structure motion trajectory, which can use the information obtained from the two-dimensional static anterior segment image as prior knowledge to assist subsequent tracking processing, simplify the processing steps, improve the processing efficiency, and save the calculation resources and time. Further, the segmentation data, positioning data and anterior segment structure motion trajectory are used for parameter quantization to obtain the anterior segment parameter data, which can obtain the data showing the structural change information of the anterior segment from multiple angles and multiple times by introducing the parameters related to the motion process of the anterior segment structure, avoiding the error caused by the shooting angle, single time and other conditions of the two-dimensional static image data. This way improves the accuracy of the evaluation of the anterior segment structure by obtaining more accurate and comprehensive parameter data, thereby providing more accurate judgment basis for doctors.

[0083] In step S101 of some embodiments, the anterior segment of the eye is defined as the front one-third of the eye, which includes the conjunctiva, cornea, anterior chamber, iris, pupil, ciliary body, and lens, etc. The anterior segment structures constitute the optical path of the eye. The target object is the object to be collected. The target collection mode refers to the way of acquiring the image data of the anterior segment of the target object, which includes a two-dimensional image collection mode and a video collection mode. The two-dimensional image collection mode collects two-dimensional images, which can also be referred to as planar static images. Specifically, the two-dimensional image collection mode includes at least one of the following modes: Anterior Segment Optical Coherence Tomography (AS-OCT), slit lamp microscope imaging, Ultrasound Biomicroscopy (UBM), etc. The video collection mode collects two-dimensional videos and three-dimensional videos, which can also be referred to as planar motion images and stereoscopic motion images, respectively. Specifically, the video collection mode includes at least one of the following modes: high-speed camera, dynamic slit lamp video imaging, dynamic optical coherence tomography, ultrasound video imaging, etc. Therefore, the anterior segment image data is the image and / or video data obtained by imaging the anterior segment of the target object. The anterior segment image data can be AS-OCT data or UBM data.

[0084] In step S102 of some embodiments, the target modality is the modality corresponding to the anterior segment image data, which is classified according to the type of the anterior segment image data. The type of the anterior segment image data includes two-dimensional static anterior segment image and motion anterior segment image video. The target modality is used to indicate that the anterior segment image data includes at least one of the two-dimensional static anterior segment image and the motion anterior segment image video. By modality classification, it can be determined which types of anterior segment image data are currently input, so as to determine the subsequent processing mode for these data. After determining the modality, the two-dimensional image can be resized to adapt to subsequent processing, or the video can be preprocessed to improve the quality of the video.

[0085] In step S103 of some embodiments, the segmentation data of the anterior segment structure refers to the region information of the anterior segment structure, which includes information such as the boundary and shape of the anterior segment structure. Specifically, the segmentation data can be obtained by segmenting the region corresponding to the anterior segment structure from the original image, or can be obtained by labeling the region corresponding to the anterior segment structure in the original image. The positioning data refers to the position information of the segmented anterior segment structure, which can be at least one of the center point coordinates of each anterior segment structure, the four vertex coordinates of the rectangular region formed after the structure is framed with a rectangular frame, the boundary coordinates along the structure boundary, or the position coordinates of the key points (i.e. anatomical key points) in the anterior segment structure that have anatomical significance. If the target modality indicates that the anterior segment image data at least includes two-dimensional static anterior segment image, the two-dimensional static anterior segment image will be segmented and positioned to obtain the segmentation data and positioning data of the anterior segment structure.

[0086] Referring to Figure 2 In step S103 of some embodiments, the anterior segment image processing method based on tracking can include but is not limited to steps S201 to S204:

[0087] Step S201, encoding the two-dimensional static anterior segment image into a high-dimensional hidden space by a first encoder to obtain first encoding features;

[0088] Step S202, performing element weighting on the elements of the first encoding features by a first optimization module to obtain first weighted features;

[0089] Step S203, decoding the first weighted features by a first decoder to obtain segmentation data;

[0090] Step S204, inputting the two-dimensional static anterior segment image, the segmentation data, and the first weighted features into a first positioning module for positioning processing to obtain positioning data.

[0091] Figure 3 is a schematic diagram of the processing process of the anterior segment segmentation and positioning model, and now Figure 3 The above steps S201 to S204 will be described in detail. The anterior segment segmentation and positioning model is used for segmentation and positioning processing of the two-dimensional static anterior segment image, so as to obtain the segmentation data and positioning data of the anterior segment structure in the two-dimensional static anterior segment image. The anterior segment segmentation and positioning model includes a first encoder, a first optimization module, a first decoder, and a first positioning module.

[0092] In step S201 of some embodiments, the high-dimensional hidden space refers to a compact representation in a high-dimensional space generated after encoding or feature extraction on the two-dimensional static anterior segment image, each dimension in the space representing a certain feature or attribute of the two-dimensional static anterior segment image, but these features are usually implicit, i.e., not directly interpretable or visible. The first encoded feature is a feature vector, which includes high-level semantic features and structural position features. Among them, the high-level semantic features can be the class information of specific structures in the image. The structural position feature can be the spatial distribution information of the anterior segment structure in the image. The first encoder is used to encode the input two-dimensional static anterior segment image into a high-dimensional hidden space, thereby obtaining the first encoded feature. The first encoder can be a convolutional neural network, an autoencoder, or a variational autoencoder, which is not limited here. In a specific embodiment, the first encoder includes a convolutional layer, a down-sampling layer, and a multi-layer perceptron, which encodes the two-dimensional static anterior segment image into a high-dimensional hidden space through a series of convolution, down-sampling, and multi-layer perceptron operations, and automatically extracts high-level semantic features and structural position features in the high-dimensional hidden space.

[0093] In step S202 of some embodiments, the first weighted feature is obtained by weighting the input first encoded feature. Specifically, the first optimization module adjusts the weights of each element in the first encoded feature, so that the weight corresponding to the more valuable element in the first encoded feature is higher, and the weight corresponding to the lower value element in the first encoded feature is lower, and then the adjusted weight is multiplied by each element in the first encoded feature to obtain the first weighted feature. The first optimization module can be a multi-head self-attention mechanism, or can be composed of a fully connected layer, or can be composed of a weighted loss function, which is not limited here.

[0094] In step S203 of some embodiments, the first decoder is used to decode the input first weighted feature, thereby obtaining the segmentation data. The specific process of decoding is to convert the high-dimensional weighted feature into a pixel-level segmentation image to display the structural information in the image. The first decoder can be an inverse convolutional neural network, or can be composed of an up-sampling layer, which is not limited here. In a specific embodiment, the first decoder includes an inverse convolutional layer, an up-sampling layer, and a multi-layer perceptron, which generates segmentation results of each anterior segment structure by a series of inverse convolution, up-sampling, and multi-layer perceptron operations on the high-level semantic features and structural position features in the high-dimensional hidden space, to segment the iris, cornea, anterior chamber, pupil, and other structures.

[0095] In step S204 of some embodiments, the first positioning module is configured to locate the anterior segment structure in the two-dimensional static anterior segment image, so as to obtain positioning data including position information and the like. The first positioning module can be a small coding-decoding pair, a small convolutional neural network, or a local attention mechanism. In an embodiment in which the first positioning module is a small coding-decoding pair, the small coding-decoding pair further encodes the input two-dimensional static anterior segment image, the segmentation data, and the first weighted feature, then fuses the encoded features, and finally decodes the fused features to reconstruct the spatial position of the anterior segment structure, so as to obtain the positioning data. In a specific embodiment, based on the two-dimensional static anterior segment image and the first weighted feature, the key points of the anterior segment structure are classified and positioned by taking the segmentation data as a constraint condition, so as to obtain the positions of important anatomical key points such as the iris root point, the pupil margin point, and the scleral spur point.

[0096] Through steps S201 to S204 described above, the two-dimensional static anterior segment image can be converted into first encoded features in a high-dimensional hidden space by the first encoder, and then the features are weighted and optimized by the first optimization module to strengthen key information and suppress noise. Then, the first decoder decodes the optimized features to generate accurate segmentation data, which identifies the boundaries of the anterior segment structure. Finally, by inputting the original image, the segmentation data, and the weighted feature into the positioning module, the position coordinates of each anterior segment structure are further accurately determined, which improves the accuracy and efficiency of segmentation and positioning.

[0097] In step S104 of some embodiments, the anterior segment structure motion trajectory refers to the motion path of the anterior segment structure in the motion anterior segment image video, and is constructed based on the position change information of a plurality of adjacent image frames. If the target modality indicates that the anterior segment image data includes two-dimensional static anterior segment images and motion anterior segment image videos, the two-dimensional static anterior segment images are first segmented and positioned to obtain segmentation data and positioning data of the anterior segment structure, and then the anterior segment structure is tracked based on the segmentation data, the positioning data, and the motion anterior segment image video to obtain the anterior segment structure motion trajectory. In other embodiments, if the target modality indicates that the anterior segment image data only includes motion anterior segment image videos, some image frames are first selected from the motion anterior segment image videos, and then segmentation data and positioning data of the anterior segment structure are obtained based on the image frames.

[0098] Please refer to Figure 4, the motion ocular anterior segment image video includes the movement process of the ocular anterior segment structure in the target environment, which can be an environment changing between a dark room and a bright room, or other environments that enable the ocular anterior segment to be stimulated by light. Specifically, the environment changing between the dark room and the bright room can be an environment of one dark room-bright room-dark room change. In step S104 of some embodiments, the tracking-based ocular anterior segment image processing method can include but is not limited to including steps S401 to S403:

[0099] Step S401, extracting, from all image frames in the motion ocular anterior segment image video, every two adjacent image frames in a time sequence order to obtain a first image frame and a second image frame;

[0100] Step S402, determining position change information of the ocular anterior segment based on the segmentation data, the positioning data, the first image frame and the second image frame;

[0101] Step S403, obtaining a movement trajectory of the ocular anterior segment structure based on each position change information.

[0102] It should be noted that the iris movement is the main manifestation of the ocular anterior segment movement, and the size of the pupil is controlled by muscles such as the sphincter and the dilator, so as to adjust the amount of light entering the eye. When the environment is relatively dark, the iris contracts, the pupil enlarges, and the light entering the eye increases; when the environment is relatively bright, the iris relaxes, the pupil shrinks, and the light entering the eye decreases. The severity of diseases such as narrow angle, closed angle, closed angle glaucoma, open angle glaucoma and other eye diseases will affect the ability of iris movement. When a person with such diseases enters a bright room from a dark room, the range and speed of iris relaxation may decrease. Therefore, the strength of the iris movement can reflect the severity of the eye disease, assist doctors in disease diagnosis and evaluation, and help patients to be detected and treated early. Therefore, when the motion ocular anterior segment image video includes the movement process of the ocular anterior segment structure in the target environment, it is especially capable of tracking the iris.

[0103] In step S401 of some embodiments, the first image frame is located before the second image frame in the time sequence order of the motion ocular anterior segment image video, and the two image frames are continuous.

[0104] In step S402 of some embodiments, the position change information refers to the spatial displacement or deformation of the ocular anterior segment structure between the image frames.

[0105] Please refer to Figure 5 In step S402 of some embodiments, the tracking-based ocular anterior segment image processing method can include but is not limited to including steps S501 to S506:

[0106] Step S501, encoding the first image frame by a second encoder to obtain second encoding features;

[0107] Step S502, the second image frame is encoded by a third encoder to obtain a third encoding feature;

[0108] Step S503, the elements of the second encoding feature are element-weighted by a second optimization module to obtain a second weighted feature;

[0109] Step S504, the elements of the third encoding feature are element-weighted by a third optimization module to obtain a third weighted feature;

[0110] Step S505, the second weighted feature and the third weighted feature are motion field estimated by a second decoder to obtain a motion field;

[0111] Step S506, the motion field, the segmentation data, and the positioning data are position tracked by a second positioning module to determine the position change information of the anterior segment.

[0112] Figure 6 is a schematic diagram of a processing process of an anterior segment structure tracking model, and now combined with Figure 6 The steps S501 to S506 are described in detail. The anterior segment structure tracking model is used to track the anterior segment structure in a moving anterior segment image video, so as to obtain the motion trajectory of the anterior segment structure in the moving anterior segment image video. The anterior segment structure tracking model includes a second encoder, a third encoder, a second optimization module, a third optimization module, a second decoder, and a second positioning module.

[0113] In steps S501 and S502 of some embodiments, the second encoder and the third encoder are substantially similar to the first encoder. The second encoding feature and the third encoding feature are substantially similar to the first encoding feature. The difference is that the second encoder is used to encode the first image frame to obtain the second encoding feature; and the third encoder encodes the second image frame to obtain the third encoding feature.

[0114] In steps S503 and S504 of some embodiments, the second optimization module and the third optimization module are substantially similar to the first optimization module. The second weighted feature and the third weighted feature are substantially similar to the first weighted feature. The difference is that the second optimization module element-weights the elements of the second encoding feature to obtain the second weighted feature; and the third optimization module element-weights the elements of the third encoding feature to obtain the third weighted feature.

[0115] Please refer to Figure 7 In step S503 of some embodiments, the tracking-based anterior segment image processing method can include but is not limited to steps S701 to S702:

[0116] Step S701, the second optimization module uses an attention mechanism to calculate the correlation between each element of the second encoded feature, and obtains an attention weight;

[0117] Step S702, based on the multiplication calculation of the attention weight and the element of the second encoded feature, the second weighted feature is obtained.

[0118] In step S701 of some embodiments, the second optimization module applies an attention mechanism to calculate the correlation of all elements of the second encoded feature. This means that each feature element is compared with other feature elements to evaluate their similarity or relevance. Among them, the correlation calculation method includes dot product or cosine similarity, which can quantify the similarity between features. In this process, the attention mechanism will obtain the attention weight. The attention weight is specifically in the form of a vector or a matrix, which reflects the importance of each element in the encoded feature. This step focuses on the feature information related to the current task by using the attention mechanism, so that the model is more efficient and accurate in processing complex data.

[0119] In step S702 of some embodiments, the second optimization module will multiply the attention weight with each element of the second encoded feature. This process can be element-by-element multiplication, that is, each element will be multiplied by its corresponding attention weight. This not only preserves the original semantic and structural information in the second encoded feature, but also highlights those features considered important under the attention weight. Through this step, the model can more accurately focus on the features closely related to the task, while weakening or ignoring secondary feature information.

[0120] Through the above steps S701 to S702, the model can obtain the important part of the image feature through correlation calculation, so as to capture key details in a large amount of information and improve the accuracy and efficiency of subsequent processing.

[0121] In step S505 of some embodiments, the motion field refers to the motion vector field of each point in the image between two image frames. Therefore, the motion field describes the movement of the anterior segment structure in the image space, which is usually represented as the movement speed and direction of each pixel. The second decoder is used for motion field estimation of the input second weighted feature and third weighted feature, which is designed based on the matching and similarity analysis between the second weighted feature and the third weighted feature.

[0122] In step S506 of some embodiments, the second positioning module is configured to track the position of the anterior segment structure, specifically based on the motion field, the segmentation data, and the positioning data, to determine the position change information of the anterior segment structure. The position change information of the anterior segment structure can be displacement information of the entire anterior segment structure, or displacement information of key points in the anterior segment structure with anatomical significance, wherein the displacement information includes information such as a starting point of movement, an end point of movement, a movement speed, and a movement direction. In a specific embodiment, the segmentation data and the positioning data are used as prior constraints for key points, which includes inputting the positioning data as a weight, the segmentation data as a range constraint, and the motion field into the second positioning module to obtain the position change information corresponding to the key points of the anterior segment structure.

[0123] Through the above steps S501 to S506, by processing the two adjacent image frames (the first image frame and the second image frame), first, the high-dimensional features of each image frame are extracted by the second encoder to obtain the second encoding features. These features contain the structural information of the anterior segment at different time points. Next, the second optimization module is used to weight these encoding features, highlight important features and reduce redundant information, thereby improving the representativeness of key features. Subsequently, the second decoder is used to perform motion field estimation on the weighted features to generate a motion field describing the motion changes of the anterior segment in the time sequence. This process not only improves the accuracy of the motion field, but also provides a reliable basis for subsequent dynamic analysis. Finally, the second positioning module combines the segmentation data and the positioning data for accurate tracking processing, thereby accurately identifying the displacement and changes of the anterior segment between adjacent frames and reflecting the dynamic motion state of the anterior segment in real time, providing important motion analysis information for clinical diagnosis and treatment.

[0124] In step S403 of some embodiments, based on the position change information obtained every two frames and the time sequence information between the image frames, the starting point of movement and the end point of movement in all the position change information are sequentially spliced to obtain the motion trajectory of the anterior segment structure. The motion trajectory of the anterior segment structure includes the corresponding motion trajectory of each anterior segment structure, and in particular, the motion trajectory of the iris can be obtained.

[0125] Through the above steps S401 to S403, by introducing the motion process of the anterior segment structure under stimulation, it is possible to obtain more structural information and physiological information of the entire anterior segment structure from more angles, thereby facilitating clinicians to make accurate diagnoses.

[0126] In another embodiment, position tracking processing can also be performed on each frame of the moving ocular anterior segment image video at the same time, so as to obtain the positions of the corresponding ocular anterior segment structure or key points of the ocular anterior segment structure of each frame, and then the positions are spliced in sequence to obtain the motion trajectory of the ocular anterior segment structure or the key points of the ocular anterior segment structure in the moving ocular anterior segment image video.

[0127] It should be noted that if the moving ocular anterior segment image video is a planar motion image, a single angle image frame will be obtained; if the moving ocular anterior segment image video is a stereoscopic motion image, multiple angle image frames will be obtained, and the multiple angles can be in the horizontal direction and the vertical direction, and can also include other directions.

[0128] Referring to Figure 8 After step S103 of some embodiments, the target acquisition mode further includes three-dimensional image acquisition. If the target modality indicates that the ocular anterior segment image data further includes a three-dimensional static ocular anterior segment image, after the two-dimensional static ocular anterior segment image is segmented and positioned to obtain segmentation data and positioning data of the ocular anterior segment structure, the tracking-based ocular anterior segment image processing method can include but is not limited to steps S801 to S803:

[0129] Step S801, three-dimensional reconstruction is performed based on the three-dimensional static ocular anterior segment image to obtain an ocular anterior segment three-dimensional model;

[0130] Step S802, segmentation calibration is performed on the ocular anterior segment three-dimensional model based on the segmentation data and the positioning data to obtain an ocular anterior segment structure three-dimensional model;

[0131] Step S803, volume calculation is performed on the ocular anterior segment structure three-dimensional model to obtain an ocular anterior segment volume parameter, which is one of the ocular anterior segment parameter data.

[0132] In step S801 of some embodiments, the three-dimensional static ocular anterior segment image is static three-dimensional image data of the ocular anterior segment. The three-dimensional image acquisition mode acquires a three-dimensional image, which can also be referred to as a stereoscopic static image. Specifically, the three-dimensional image acquisition mode includes at least one of three-dimensional AS-OCT and anterior segment optical biometry and the like. The ocular anterior segment three-dimensional model is a three-dimensional virtual model of the ocular anterior segment obtained by three-dimensional reconstruction, which can restore the three-dimensional geometric shape of the ocular anterior segment. The three-dimensional reconstruction can be implemented by three-dimensional reconstruction algorithms such as surface reconstruction and voxel reconstruction, which are not limited herein. In a specific implementation, the three-dimensional static ocular anterior segment image can be preprocessed by image registration, denoising and the like to ensure the clarity and alignment of the image; then a depth map is generated by a stereoscopic matching algorithm; finally, the ocular anterior segment three-dimensional model is reconstructed by triangulation method using the depth map and the image.

[0133] In step S802 of some embodiments, the anterior segment structure three-dimensional model is a three-dimensional virtual model of the anterior segment structure, which can restore the three-dimensional geometry of the anterior segment structure. Specifically, after obtaining the segmentation data and the positioning data, the anterior segment three-dimensional model is aligned and registered by the segmentation data (i.e. the shape of each anterior segment structure and other related information) and the positioning data (i.e. the position of each anterior segment structure and other related information, such as the position information of the anatomical key points of the anterior segment structure), and then the three-dimensional model is segmented according to the segmentation information by using an image segmentation algorithm to obtain the three-dimensional model of the cornea, iris, lens and other anterior segment structures.

[0134] In another embodiment, if the segmentation data and the positioning data are not obtained from the two-dimensional static anterior segment image in advance, the segmentation data and the positioning data of the anterior segment structure can be obtained from the three-dimensional static anterior segment image.

[0135] In step S803 of some embodiments, the anterior segment parameter data is a parameter related to the anterior segment, and the anterior segment volume parameter is a parameter related to the volume of the anterior segment structure, which is one of the anterior segment parameter data. The anterior segment volume parameter can include the iris volume, the anterior chamber volume, the trabecular iris space volume (TISV) and other anterior chamber angle volume parameters, and can also include other volume parameters. Specifically, the anterior segment structure three-dimensional model is calculated by voxel or area integration to obtain the anterior segment volume parameter.

[0136] Through the above steps S801 to S803, three-dimensional information reflecting the static structure can be obtained. Compared with the information obtained from the two-dimensional image, the three-dimensional information is more comprehensive and the parameters are more accurate, thereby improving the accuracy of the evaluation of the anterior segment structure.

[0137] In step S105 of some embodiments, the parameter quantification refers to converting the information into measurable parameter data, specifically calculating various parameters (e.g., thickness, area, movement speed, etc.) of the anterior segment structure according to the segmentation result, the positioning data, and the movement trajectory data. Therefore, since the anterior segment parameter data is obtained based on the two-dimensional anterior segment image and the moving anterior segment video, the anterior segment parameter data includes both static parameters and dynamic parameters. The static parameters refer to parameters related to the anterior segment measured at a certain time point. The static parameters can include Iris Thickness (IT), Iris Area (I-Area), Central Corneal Thickness, Anterior Chamber Area (ACA), Angle Opening Distance (AOD), Angle Recess Area (ARA), Trabecular Iris Space (TISA), Pupil Distance, etc. The aforementioned anterior segment volume parameter is also a static parameter. The dynamic parameters refer to movement change parameters related to the anterior segment measured within a certain time range. The dynamic parameters are obtained according to the movement trajectory of the anterior segment structure, and can include parameters such as the change parameter of the pupil size, the movement parameter of the iris position, etc. This step, on the basis of obtaining the static parameters, introduces parameters related to the movement process of the anterior segment structure under stimulation, which can reflect the movement ability and other physiological information of the anterior segment structure, so that the obtained anterior segment parameter data is more comprehensive.

[0138] In a specific embodiment, the dynamic parameters include Iris Relaxation Velocity, Iris Contraction Velocity, and Pupil Diameter Change Velocity. The Iris Relaxation Velocity is calculated based on formula (1), the Iris Contraction Velocity is calculated based on formula (2), and the Pupil Diameter Change Velocity is calculated based on formula (3). Formulas (1)-(3) are as follows.

[0139]

[0140] Please refer to Figure 9After step S105 of some embodiments, the tracking-based anterior segment image processing method can include, but is not limited to, predicting a disease based on the anterior segment parameter data to obtain a disease prediction result. The disease prediction process can include, but is not limited to, steps S901 to S904:

[0141] In step S901, the preset index is compared with the dynamic parameter to obtain a comparison result.

[0142] In step S902, the comparison result is analyzed and processed to obtain a first prediction result.

[0143] In step S903, the dynamic parameter and the static parameter are used to predict a second prediction result.

[0144] In step S904, the first prediction result and the second prediction result are summarized to obtain a disease prediction result.

[0145] In step S901 of some embodiments, the preset index is a standard or threshold value preset for evaluating the health status of the anterior segment. The preset index and the dynamic parameter are in a corresponding relationship, and the comparison result is obtained by comparing the dynamic parameter with the corresponding preset index. The comparison result is used to indicate the relationship between the preset index and the dynamic parameter. In some embodiments, a disease risk prediction classification model can be pre-trained to obtain a first prediction result based on one-hot encoding and a VGG (Visual Geometry Group) classifier. Specifically, the disease categories are processed by one-hot encoding, for example, different diseases such as glaucoma, iris disease, and neurodegenerative disease are represented by one-hot vectors; based on the pre-trained VGG deep learning classifier, the disease categories after one-hot encoding are used as labels, and the dynamic parameters and static parameters of the target object are input for training and classification prediction; the VGG classifier classifies and predicts the possibility of different iris diseases (such as glaucoma, iritis, and neurodegenerative disease) in the target object.

[0146] In step S902 of some embodiments, the first prediction result is a result obtained based on the dynamic parameter about whether the target object has an anterior segment disease. The analysis and processing can include statistical analysis or machine learning model implementation.

[0147] In a specific embodiment, when predicting a disease, the iris hardness can be predicted. The iris hardness has a corresponding preset index, which can be a health threshold. If the comparison result indicates that the dynamic parameter is higher than the health threshold, it means that the target object has a higher risk of iris disease and neurodegenerative disease. Therefore, the first prediction result indicates that the target object has a risk of iris disease and neurodegenerative disease.

[0148] In step S903 of some embodiments, the second prediction result is a result about whether the target subject has an anterior segment disease based on the dynamic parameters and the static parameters. The prediction can be implemented by a pre-trained regression-based risk prediction model. Specifically, the static parameters and the dynamic parameters are jointly input into the regression-based risk prediction model, which performs binary classification prediction on glaucoma and abnormal risks, and outputs the probabilities (risk values) of glaucoma and abnormal diseases, to obtain a prediction result for the probabilities of glaucoma and abnormal risks, which can be a complementary result of the VGG classifier prediction result. Therefore, the second prediction result can indicate whether the target subject has a glaucoma and abnormal risk.

[0149] In step S904 of some embodiments, after obtaining the first prediction result and the second prediction result, the two results are summarized to obtain a disease prediction result. In a specific implementation, the disease prediction result can indicate whether the target subject has an iris disease and a neurodegenerative disease risk, and whether the target subject has a glaucoma and abnormal risk.

[0150] Through the above steps S901 to S904, a comprehensive disease prediction result can be formed by summarizing the first prediction result and the second prediction result, which effectively improves the accuracy of disease prediction and provides an important basis for the formulation of clinical decision and the management of target subjects. At the same time, based on multiple dimensions of image parameters, doctors can more accurately perform disease diagnosis and evaluation.

[0151] In other embodiments, physiological medical history information of the target subject can be obtained at the same time as the anterior segment image data, which includes gender, age, intraocular pressure, eye axis, optometry results, angle state, and related medical history of the target subject. After obtaining the quantized anterior segment parameter data, the anterior segment parameter data and the physiological medical history information can be used for disease clustering. Specifically, the anterior segment parameter data and the physiological medical history information can be used as inputs of a model, and k- nearest neighbor and multilayer perceptron algorithms can be used to predict the states of iris hardness, abnormal risk, glaucoma, and neurodegenerative disease of the target subject.

[0152] Please refer to Figure 10 , Figure 10is a flowchart of a computer-aided diagnosis system provided by the tracking-based anterior segment image processing method of the embodiments of the present application. First, the anterior segment image data is imported into the system. Then, the imported anterior segment image data is modality selected to determine the target modality, and if the target modality indicates that the imported anterior segment image data includes multiple modalities, the target modality and the processing order of each modality in the target modality are determined. For two-dimensional image data, the system will perform two-dimensional or three-dimensional modality detection and size adjustment processing; for video data, the system will perform two-dimensional and three-dimensional motion modality detection and image quality preprocessing. Next, data preprocessing is performed, which includes structure segmentation and structure positioning of the data, the specific implementation process of which is similar to that of step S103 and will not be repeated here. Then, in the case where the target modality indicates that the anterior segment image data includes two-dimensional static anterior segment image and motion anterior segment image video, the anterior segment structure is tracked, the iris motion trajectory is predicted based on the motion tracking of the anterior segment structure, and the iris motion trajectory is obtained, the specific implementation process of which is similar to that of step S104 and will not be repeated here. Next, the clinical parameter quantification is performed based on the obtained structure segmentation information, structure positioning information, and anterior segment structure motion trajectory (especially the iris motion trajectory), the specific implementation process of which is similar to that of step S105 and will not be repeated here. Finally, the multi-modality fusion disease assessment is performed based on the obtained parameters, the specific implementation process of which is similar to that of steps S901 to S904 and will not be repeated here. In combination with the foregoing description of the model or algorithm, the system uses different artificial intelligence algorithms based on neural networks in multiple key steps such as data preprocessing, anterior segment structure tracking, and iris motion trajectory prediction, and improves the algorithm performance through multiple neural network optimization strategies (such as increasing the optimization module). Compared with the traditional algorithm, the artificial intelligence algorithm used by the system has higher precision and robustness. Moreover, the system can be not bound to any device.

[0153] It should be noted that, in the above description of the embodiments of the present application, the steps of the method are described in a specific sequence, but the sequence of the steps is not limited to the sequence described in the embodiments of the present application. The sequence of the steps can be changed, and the sequence of the steps can be changed according to the actual situation. Figure 10In the data preprocessing of the application, when the modality includes two-dimensional images and / or three-dimensional images, after structural segmentation and structural positioning are performed to obtain segmentation data and positioning data, static parameters can be obtained based on the segmentation data and the positioning data, and then disease clustering can be performed based on the static parameters and / or physiological medical history information of the target object. On this basis, when the modality also includes video, the steps of anterior segment structure motion tracking, iris motion trajectory prediction, and clinical parameter quantification are performed, at this time, the static parameters have been obtained in the previous steps, therefore, in the step of clinical parameter quantification, only dynamic parameters can be obtained, and the static parameters obtained will not be quantified repeatedly, and finally, multi-modal fusion disease evaluation is obtained based on the results of disease clustering and the results obtained based on dynamic parameter analysis and / or static parameter analysis. When the modality does not include video, the steps of anterior segment structure motion tracking, iris motion trajectory prediction, and clinical parameter quantification are not performed, and finally, disease evaluation is obtained based on the results of disease clustering.

[0154] In some embodiments, the computer-aided diagnosis system can include a visualization operation interface, which can specifically include an angle of the anterior chamber monitoring interface, an iris motion tracking interface, and an iris abnormality early warning interface.

[0155] Please refer to Figure 11 , Figure 11 is a process schematic diagram for iris state prediction in the tracking-based anterior segment image processing method provided by the application. First, multi-source data is imported, which includes images and videos related to the anterior segment. Then, structural segmentation and positioning are performed based on the images, and dynamic tracking of the anterior segment structure is performed based on the videos. Then, clinical parameter measurement (i.e., clinical parameter quantification) is performed based on the segmentation and positioning results and the tracking results to obtain anterior segment parameter data. Then, the iris state is predicted based on the anterior segment parameter data to obtain prediction results, which include predictions of iris hardness, iris reaction speed, and disease risk, wherein the disease risk prediction includes predictions of iris structural abnormalities and iris motion abnormalities, and this step is a computer-aided diagnosis. Finally, the computer can perform disease risk prediction based on the prediction results to obtain conclusions and severity assessments of diseases such as angle narrowing, angle apposition closure, angle synechia, and neurodegenerative iris abnormalities, which can assist doctors in disease diagnosis. Of course, doctors can also perform disease diagnosis based on the prediction results.

[0156] It should be noted that, in order to enhance the accuracy, generalization and robustness of the algorithm used by the system, the computer-aided diagnosis system based on the tracking-based anterior segment image processing method introduces structural reparameterization, squeeze-and-excitation module, global-local attention module and pyramid convolution module for algorithm optimization and improvement for each corresponding neural network algorithm of each step, and adds corresponding anatomical clinical prior knowledge constraints and uncertainty analysis steps, which can better obtain high-level semantic features and structural position information in the anterior segment image data, mine the correlation between clinical parameters and different diseases, and improve the confidence of the assisted diagnosis conclusion and severity rating.

[0157] Referring to Figure 12 The embodiment of the present application also provides an anterior segment image processing system based on tracking, which can implement the anterior segment image processing method based on tracking. The system comprises:

[0158] A data acquisition module 1201 is configured to acquire anterior segment image data of an anterior segment of a target object collected in a target collection manner. The target collection manner comprises at least one of two-dimensional image collection and video collection. The anterior segment comprises an anterior segment structure.

[0159] A modality determination module 1202 is configured to perform modality classification on the anterior segment image data to obtain a target modality. The target modality indicates that the anterior segment image data comprises at least one of a two-dimensional static anterior segment image and a motion anterior segment image video.

[0160] A first processing module 1203 is configured to, if the target modality indicates that the anterior segment image data comprises at least the two-dimensional static anterior segment image, perform segmentation and positioning on the two-dimensional static anterior segment image to obtain segmentation data and positioning data of the anterior segment structure.

[0161] A second processing module 1204 is configured to, if the target modality indicates that the anterior segment image data comprises the two-dimensional static anterior segment image and the motion anterior segment image video, perform tracking on the anterior segment structure based on the segmentation data, the positioning data and the motion anterior segment image video to obtain a motion trajectory of the anterior segment structure.

[0162] A parameter quantization module 1205 is configured to perform parameter quantization based on the segmentation data, the positioning data and the motion trajectory of the anterior segment structure to obtain anterior segment parameter data.

[0163] The specific implementation of the tracking-based anterior segment image processing system is basically the same as the specific implementation of the tracking-based anterior segment image processing method described above, and will not be repeated here.

[0164] The embodiments of the present application also provide an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor implements the tracking-based anterior segment image processing method described above when executing the computer program. The electronic device can be any intelligent terminal, such as a tablet computer or a vehicle-mounted computer.

[0165] Please refer to Figure 13 , Figure 13 The hardware structure of the electronic device of another embodiment is illustrated, which includes:

[0166] The processor 1301 can be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is used to execute related programs to implement the technical solutions provided by the embodiments of the present application.

[0167] The memory 1302 can be implemented in the form of a ROM (Read-Only Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory). The memory 1302 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 1302 and are called and executed by the processor 1301 to implement the tracking-based anterior segment image processing method of the embodiments of the present application.

[0168] The input / output interface 1303 is used to realize information input and output.

[0169] The communication interface 1304 is used to realize the communication interaction between the device and other devices. The communication can be realized by a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.).

[0170] The bus 1305 transmits information between various components (such as the processor 1301, the memory 1302, the input / output interface 1303, and the communication interface 1304) of the device.

[0171] The processor 1301, the memory 1302, the input / output interface 1303, and the communication interface 1304 are connected to each other through a bus 1305 to realize communication connection between devices inside.

[0172] The application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the tracking-based anterior segment image processing method.

[0173] The memory is a non-transitory computer readable storage medium, which can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0174] The tracking-based anterior segment image processing method, the tracking-based anterior segment image processing system, the electronic device, and the storage medium provided by the application can obtain anterior segment image data collected by a target collection manner on a target object, wherein the target collection manner includes at least one of two-dimensional image collection and video collection, and can provide a data basis for subsequent processing. Further, the anterior segment image data is classified by modalities to obtain a target modality, so as to select different processing methods for different modalities, improve flexibility and adaptability. Further, if the target modality indicates that the anterior segment image data at least includes a two-dimensional static anterior segment image, the two-dimensional static anterior segment image is segmented and positioned to obtain segmentation data and positioning data of an anterior segment structure; if the target modality indicates that the anterior segment image data includes a two-dimensional static anterior segment image and a motion anterior segment image video, the anterior segment structure is tracked based on the segmentation data, the positioning data, and the motion anterior segment image video to obtain an anterior segment structure motion trajectory, which can use information obtained from the two-dimensional static anterior segment image as prior knowledge to assist subsequent tracking processing, simplify the processing steps, improve the processing efficiency, and save the computing resources and time. Further, the segmentation data, the positioning data, and the anterior segment structure motion trajectory are used for parameter quantization to obtain anterior segment parameter data, which can introduce parameters related to the motion process of the anterior segment structure to obtain data that exhibits the structural change information of the anterior segment from multiple angles and multiple times, and avoid errors caused by the shooting angle, single time, and the like of the two-dimensional static image data. This way improves the accuracy of the evaluation of the anterior segment structure by obtaining more accurate and comprehensive parameter data, thereby providing more accurate judgment basis for doctors.

[0175] The embodiments described in the specification are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0176] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than those shown in the figures, or combine certain steps, or different steps.

[0177] The system embodiments described above are merely illustrative, and the units described as separate components can or can not be physically separated, that is, can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0178] Those skilled in the art can understand that all or some steps in the above disclosed method, functional modules / units in the system, and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0179] The terms "first", "second", "third", "fourth" and the like (if any) in the specification of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0180] It should be understood that, in the application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases of only A, only B and A and B existing at the same time, wherein A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b and c can be single or multiple.

[0181] In several embodiments provided in the application, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiments described above are only illustrative, for example, the division of the above units is only a logical function division, and actual implementation can have another division mode, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. The coupling or direct coupling or communication connection between the displayed or discussed each other can be through some interface, indirect coupling or communication connection between systems or units, which can be electrical, mechanical or other forms.

[0182] The units described above as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. According to actual needs, some or all of the units can be selected to achieve the purpose of the embodiment.

[0183] In addition, each functional unit in each embodiment of the application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0184] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0185] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not limited to the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A method for processing an anterior segment image based on tracking, characterized in that, The method comprises: acquiring anterior segment image data of an anterior segment of a target object collected in a target collection manner; the target collection manner comprises two-dimensional image collection, video collection, and three-dimensional image collection, and the anterior segment comprises an anterior segment structure; performing modality classification on the anterior segment image data to obtain a target modality; the target modality indicates that the anterior segment image data comprises a two-dimensional static anterior segment image, a motion anterior segment image video, and a three-dimensional static anterior segment image; if the target modality indicates that the anterior segment image data is a two-dimensional static anterior segment image, performing segmentation and positioning on the two-dimensional static anterior segment image to obtain segmentation data and positioning data of the anterior segment structure; if the target modality indicates that the anterior segment image data is a motion anterior segment image video, tracking the anterior segment structure based on the segmentation data, the positioning data, and the motion anterior segment image video to obtain an anterior segment structure motion trajectory; if the target modality indicates that the anterior segment image data is a three-dimensional static anterior segment image, performing three-dimensional reconstruction based on the three-dimensional static anterior segment image to obtain an anterior segment three-dimensional model; performing segmentation and calibration on the anterior segment three-dimensional model based on the segmentation data and the positioning data to obtain an anterior segment structure three-dimensional model; and performing volume calculation on the anterior segment structure three-dimensional model to obtain an anterior segment volume parameter; performing parameter quantification based on the segmentation data, the positioning data, and the anterior segment structure motion trajectory, and combining the anterior segment volume parameter to obtain anterior segment parameter data.

2. The method of claim 1, wherein, The segmentation and positioning of the two-dimensional static anterior segment image to obtain the segmentation data and the positioning data of the anterior segment structure comprises: encoding the two-dimensional static anterior segment image into a high-dimensional hidden space through a first encoder to obtain first encoding features; performing element weighting on elements of the first encoding features through a first optimization model to obtain first weighted features; performing decoding processing on the first weighted features through a first decoder to obtain the segmentation data; inputting the two-dimensional static anterior segment image, the segmentation data, and the first weighted features into a first positioning module for positioning processing to obtain the positioning data.

3. The method of claim 1, wherein, The motion anterior segment image video comprises a motion process of the anterior segment structure in a target environment. The tracking of the anterior segment structure based on the segmentation data, the positioning data, and the motion anterior segment image video to obtain an anterior segment structure motion trajectory comprises: extracting every adjacent two image frames in a time sequence order from all image frames in the motion anterior segment image video to obtain a first image frame and a second image frame; determining position change information of the anterior segment based on the segmentation data, the positioning data, the first image frame, and the second image frame; obtaining the anterior segment structure motion trajectory based on each position change information.

4. The method of claim 3, wherein, The determination of the position change information of the anterior segment based on the segmentation data, the positioning data, the first image frame, and the second image frame comprises: performing encoding processing on the first image frame through a second encoder to obtain second encoding features; The second image frame is encoded by a third encoder to obtain third encoded features; The elements of the second encoded features are weighted by a second optimization module to obtain second weighted features; The elements of the third encoded features are weighted by a third optimization module to obtain third weighted features; The second weighted features and the third weighted features are estimated by a second decoder to obtain a motion field; The motion field, the segmentation data, and the positioning data are tracked by a second positioning module to determine the position change information of the anterior eye segment.

5. The method of claim 4, wherein, The elements of the second encoded features are weighted by a second optimization module to obtain second weighted features, including: The correlation between the elements of the second encoded features is calculated by the second optimization module using an attention mechanism to obtain attention weights; The second weighted features are obtained by multiplying the attention weights and the elements of the second encoded features.

6. The method according to any one of claims 1 to 5, characterized in that, After obtaining the anterior eye segment parameter data, the method further includes: Predicting a disease based on the parameter data to obtain a disease prediction result; the parameter data includes static parameters and dynamic parameters, including: Comparing a preset index with the dynamic parameters to obtain a comparison result; Analyzing and processing the comparison result to obtain a first prediction result; Predicting based on the dynamic parameters and the static parameters to obtain a second prediction result; Summarizing the first prediction result and the second prediction result to obtain the disease prediction result.

7. A tracking-based anterior segment image processing system, characterized by, The system includes: A data acquisition module configured to acquire anterior eye segment image data of an anterior eye segment of a target object collected in a target collection manner; the target collection manner includes two-dimensional image collection, video collection, and three-dimensional image collection, and the anterior eye segment includes an anterior eye segment structure; A modality determination module configured to classify the anterior eye segment image data to obtain a target modality; the target modality indicates that the anterior eye segment image data includes a two-dimensional static anterior eye segment image, a motion anterior eye segment image video, and a three-dimensional static anterior eye segment image; A first processing module configured to, if the target modality indicates that the anterior eye segment image data is a two-dimensional static anterior eye segment image, segment and position the two-dimensional static anterior eye segment image to obtain segmentation data and positioning data of the anterior eye segment structure; A second processing module configured to, if the target modality indicates that the anterior eye segment image data is a motion anterior eye segment image video, track the anterior eye segment structure based on the segmentation data, the positioning data, and the motion anterior eye segment image video to obtain an anterior eye segment structure motion trajectory; A third processing module configured to, if the target modality indicates that the anterior eye segment image data is a three-dimensional static anterior eye segment image, perform three-dimensional reconstruction based on the three-dimensional static anterior eye segment image to obtain an anterior eye segment three-dimensional model; segment and calibrate the anterior eye segment three-dimensional model based on the segmentation data and the positioning data to obtain an anterior eye segment structure three-dimensional model; and perform volume calculation on the anterior eye segment structure three-dimensional model to obtain an anterior eye segment volume parameter; A parameter quantification module is configured to perform parameter quantification based on the segmentation data, the localization data, and the ocular anterior segment structure motion trajectory, and combine the ocular anterior segment volume parameters to obtain ocular anterior segment parameter data.

8. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the tracking-based ocular anterior segment image processing method in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program is executed by the processor to implement the tracking-based ocular anterior segment image processing method in any one of claims 1 to 6.

Images