Optical coherence tomography system and image processing method

Abstract

Disclosed herein an optical coherence tomography (OCT) system and an image processing method for OCT. The OCT system comprises a computing module configured to: obtain an initial en face image of an eye of a subject; determine from the initial en face image a defective area comprising an artefact, the defective area corresponding to a region of interest of the eye; obtain a replacement en face image of the region of interest of the eye; and replace the defective area in the initial en face image with the replacement en face image to obtain a stitched en face image.

Description

OPTICAL COHERENCE TOMOGRAPHY SYSTEM AND IMAGE PROCESSINGMETHODCROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims the benefit of priority to the Singapore application no. 10202302597U filed September 15, 2023, the contents of which are hereby incorporated by reference in their entirety for all purposes.TECHNICAL FIELD

[0002] This application relates to the field of ophthalmology, and more particularly to an optical coherence tomography system and an image processing method for optical coherence tomography.BACKGROUND

[0003] Various challenges are present in the detection of the onset of early- stage eye di eases. In the absence of timely and comprehensive eye checks, the difficulty to detect early- stage eye diseases is further increased when relying mainly on patient self-detection. Oftentimes, realization of an eyesight issue by the patient comes at a later stage whereby the eye disease has progressed to an advanced stage, leading to irreversible damages such as blindness. In addition, routine eye checks are often not comprehensive, and advance imaging techniques for eye diseases are often operator dependent, bulky and costlySUMMARY

[0004] According to an aspect, disclosed herein an optical coherence tomography (OCT) system and an image processing method for OCT. The OCT system comprises a computingmodule configured to: obtain an initial en face image of an eye of a subject; determine from the initial en face image a defective area comprising an artefact, the defective area corresponding to a region of interest of the eye; obtain a replacement en face image of the region of interest of the eye; and replace the defective area in the initial en face image with the replacement en face image to obtain a stitched en face image.

[0005] According to another aspect, disclosed herein an image processing method for optical coherence tomography. The method comprising: obtaining an initial en face image of an eye of a subject; determining from the initial en face image a defective area comprising an artefact, the defective area corresponding to a region of interest of the eye; obtaining a replacement en face image of the region of interest of the eye; and replacing the defective area in the initial en face image with the replacement en face image to obtain a stitched en face image.BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Various embodiments of the present disclosure are described below with reference to the following drawings:

[0007] FIG. 1 is a schematic diagram of an Optical Coherence Tomography (OCT) system according to embodiments of the present disclosure;

[0008] FIG. 2 illustrates a data flow of an image processing method of an OCT system according to various embodiments;

[0009] FIGs. 3 and 4 show perspective views of an OCT system according to various embodiments;

[0010] FIG. 5 shows a perspective view of another OCT system according to various embodiments;

[0011] FIG. 6 is a process flow chart of an operation of the OCT system according to various embodiments;

[0012] FIG. 7 shows the various intermediate states of image processing of the camera stream to detect pupil contour from an eye image;

[0013] FIG. 8A shows exemplary artefacts in an initial en face image;

[0014] FIG. 8B shows exemplary artefacts in another initial en face image;

[0015] FIG. 8C shows an exemplary artefact in a B-scan;

[0016] FIG. 9 is a schematic diagram of a U-Net model according to various embodiments;

[0017] FIG. 10A shows a training sample of an initial en face image and a corresponding label;

[0018] FIG. 10B shows a test sample of initial en face image and a corresponding output from the U-Net model;

[0019] FIG. 11 shows a plurality of image areas segmented from an initial en face image;

[0020] FIG. 12 shows another plurality of image areas segmented from another initial en face image;

[0021] FIG. 13 shows a prototype OCT system;

[0022] FIG. 14A shows an exemplary B-scan obtained by the prototype OCT system of FIG. 13;

[0023] FIG. 14B shows en face images obtained from manual alignment;

[0024] FIG. 14C shows en face images obtained from automatic alignment;

[0025] FIG. 15 is a schematic diagram illustrating an image processing method according to various embodiments of the present disclosure; and

[0026] FIG. 16 shows a block diagram of a processing system for implementing embodiments of the present disclosure.DETAILED DESCRIPTION

[0027] The following detailed description is made with reference to the accompanying drawings, showing details and embodiments of the present disclosure for the purposes of illustration. Features that are described in the context of an embodiment may correspondingly be applicable to the same or similar features in the other embodiments, even if not explicitly described in these other embodiments. Additions and / or combinations and / or alternatives as described for a feature in the context of an embodiment may correspondingly be applicable to the same or similar feature in the other embodiments.

[0028] In the context of various embodiments, the articles “a”, “an” and “the” as used with regard to a feature or element include a reference to one or more of the features or elements.

[0029] Tn the context of various embodiments, the term “about” or “approximately” as applied to a numeric value encompasses the exact value and a reasonable variance as generally understood in the relevant technical field, e.g., within 10% of the specified value.

[0030] As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items.

[0031] The terms “en face image”, “en face optical coherence tomography image”, “en face OCT image” may be used interchangeably and may generally refer to frontal sections of retinal layers extracted from cross-sectional(s) of optical coherence tomography (OCT) scans, such as B-scans. An “en face image” typically presents details and / or information about the retina of a subject's eye, offering a view parallel to the retinal surface and allowing visualization of specific retinal sublayers, such as the retinal pigment epithelium (RPE) or the inner plexiform layer (IPL). En face is the vertical projections of B-scans, while B-scans are the cross sections.

[0032] The term “position” or “pose” may refer to a location and an orientation of an object, for example the optical coherence tomography (OCT) measurement device. The term “location” may refer to a coordinate (for example, X-coordinate, Y coordinate, Z coordinate) of an objector part of an object in a space or a frame. The term “orientation” may refer to a facing or angle (for example, a X-direction vector, a Y-direction vector, a Z-direction vector) of an object or part of an object in a space or a frame. Each of the terms “pose”, “position”, “location” and “orientation” may be defined by a coordinate system, such as a cartesian coordinate.

[0033] The present disclosure is related to an Optical Coherence Tomography (OCT) system and an image processing method for Optical Coherence Tomography (OCT). Optical coherence tomography (OCT) includes non-invasive imaging tests which make use of light waves to obtain high resolution cross-sectional images of the retina of an eye.

[0034] In addressing the various problems posed by the non-detection of early-stage eye diseases, such as irreversible damages to eyesight, the present disclosure proposes according to various embodiments, an Optical Coherence Tomography (OCT) system which is easy to operate and reliable. The proposed OCT system provides a solution for early-stage detection of eye diseases which are often operator dependent, bulky and costly. The proposed OCT system may be an automated system configured to automatically align with an eye of a subject or patient, hence does not require manual alignment. The proposed OCT system may be a portable system in a compact form.

[0035] The proposed system may improve eye care management and potentially decrease the number of overall late-stage eye diseases and total vision loss in patients due to eye diseases. In addition, with the utilization of a platform and a headrest, the proposed OCT system has a similar working distance as conventional OCT and thus is not limited by the field of view.

[0036] The proposed OCT system may be a home-based or community-based solution for early detection of onset of eye diseases. In various embodiments, the proposed OCT system may be operated by a caregiver and / or an assistant independent of need for a dedicated medical professional. The use of the OCT system is not dependent on the skill of the operator and theuse of the device is not reliant on skilled technicians who are often in short supply, hence the OCT system may be operator-independent.

[0037] The proposed OCT system may also be deployed at primary care facilities such as clinics, general practitioner clinics, community centres, malls, and within housing estates to be used during routine eye checks. An output from the OCT system may display recommendations or assessments to the patient, such as advising the patient to visit a medical practitioner for further assessment. In addition, during consultation, the medical practitioner may also have access to the OCT images acquired from the proposed OCT system.

[0038] In various embodiments, the OCT system may include sensors, such as cameras, for locating a face of the subject or a face position. This may be followed by determining a pupil of the eye or a pupil location. An OCT measurement device may then be moved in alignment with the pupil of the subject to capture one or more OCT images of the eye of the subject. In various embodiments, an actuator may be provided to move the OCT measurement device, such as a linear stage or a robotic arm.

[0039] In various embodiments, the OCT images captured by the OCT measurement device may include B-scans and / or en face images obtained from the B-scans. Due to possible imaging errors during the capture of the OCT images, each en face image may be screened for artefacts or defects. Such artefacts may be present in the en face images due to motion of the subject during the image capturing process or due to shadows. The sources of the shadow may include vitreous opacities like floaters and resulting in a shadow artefact. In some instances where there is an alignment issue, the iris may also cause the shadow artefact to form. In addition, tilt and out-of-window artefacts which are also related to alignment, may also result in lower signal in the en face image. In various embodiments, the OCT system may detect a defective area corresponding to one or more artefacts. In response, a target region of the eye of the subject which corresponds to the defective area may be determined and the OCT measurement devicemay be moved in alignment with the defective area for a re-scan of the target region to be performed. One or more rounds of re-scanning of the eye may provide one or more replacement images for the defective area, and the replacement image(s) may be stitched with the initial en face image in place of the defective area to provide a stitched image. This process may be iterated if there are multiple defective areas in the initial en face image.

[0040] The OCT system may output high-resolution OCT images without artefacts or defects which are often present in OCT scans. Such high-resolution OCT images allow primary care medical professionals to diagnose and determine early-stage eye diseases such as age- related macular degeneration, diabetic retinopathy and glaucoma. In various embodiments, the OCT system may employ machine learning (or Al-based) models for detecting artefacts in the OCT images prior to removal. In comparison to conventional OCT systems, the proposed OCT system may output higher resolution images which may be attributed to the elimination of artefacts that often plague OCT images.

[0041] FIG. 1 illustrates an optical coherence tomography (OCT) system 100 for performing an OCT measurement of an eye 92 of a subject 90, according to various embodiments of the present disclosure. The OCT system 100 may include an OCT measurement device 1 10 for capturing OCT images, such as en face images, of the eye 92 of the subject 90. In various embodiments, the OCT measurement device 110 may be coupled with or mounted to an actuator 120. The actuator 120 may vary a position or pose of the OCT measurement device 110 relative to the eye 92 of the subject 90. In some examples, the actuator 120 may move the OCT measurement device 1 10 along each of a X-Y-Z direction in cartesian coordinates. In addition, the actuator 120 may also rotate the measurement device 110 in each of the X-Y-Z direction, enabling pitching, rolling and yawing movements of the measurement device 110.

[0042] The OCT system 100 may also include one or more sensors, such as a depth camera130 for measuring a distance of the eye 92 or face of the subject 90 relative to the OCTmeasurement device 110. In addition, the OCT system 100 may include a headrest 140 for providing a support for a chin of the subject 90. The headrest 140 may define a working distance between the OCT measurement device 110 and the eye 92 or face of the subject 90. In some embodiments, the shape and position of the headrest 140 may be personalized or customized according to the subject 90, such that the working distance is at an optimal distance. The OCT system 100 may also include a light source 1 16 for providing illumination on the face of the subject 90.

[0043] The OCT system 100 may further include a computing module 1800 provided in signal communication with the OCT measurement device 110, the actuator 120 and the depth camera 130. The computing module 1800 may be configured to send control signal to and receive data from the OCT measurement device 1 10, the actuator 120 and the depth camera 130. The computing module may be configured to perform an image processing method for the detection and removal of artefacts from the en face images.

[0044] FIG. 2 illustrates a data flow of the computing module 1800 performing an image processing method, according to various embodiments. The image processing or image correction may include locating artefact(s) in an initial image; rescanning regions of the eye corresponding to the artefacts; and integrating or combining the re-scanned image with the initial image.

[0045] In various embodiments, the computing module 1800 may obtain an initial en face image 210 of an eye 92 of a subject 90. Thereafter, the computing module 1800 may determine an artefact 220 from the initial en face image 210, and hence determine a defective area 230 comprising the artefact 220. In various embodiments, there may be one or more artefacts 220 in a single defective area 230. Similarly, there may be multiple artefacts in multiple defective areas 230. Each defective area 230 may be defined by a continuous area on the initial en face image 210. The detection of the artefacts may be performed by a machine learning model, suchas a deep learning model. The machine learning model may be an object detection model, and different classes of models may be used for artefact detection.

[0046] Based on the defective area 230 of the initial en face image 210, the computing module 1800 may determine a region of interest 240 of the eye 92 which corresponds to the defective area 230. The computing module 1800 may then control the actuator 120 may moveon-Zalign the galvo scanner 1 12 of the OCT measurement device 1 10 with the region of interest 240 of the eye to obtain a replacement en face image 250 corresponding to the region of interest 240 of the eye. In other words, upon alignment, the OCT measurement device 110 may perform a re-scan of the region of interest 240. The computing module 1800 may then replace the defective area 230 in the initial en face image 210 with the replacement en face image 250 to obtain a stitched en face image 260 to make a whole image of the entire scanned area of the eye.

[0047] In various embodiments, in order to replace the defective area 230, the computing module 1800 may segment or divide the initial en face image 210 into a plurality of image areas 212. Thereafter, the computing module 1800 may determine selected ones of the plurality of image areas 212 corresponding to the defective area 230. The computing module 1800 may divide the initial en face image 210 and determine the selected ones of the plurality of image areas 212 using another machine learning model, such as an image segmentation model. Alternatively, the computing module 1800 may use a non-machine learning method. The selected ones of the plurality of image areas 212 may be removed from the initial en face image 210 to define a replacement area 235 in the initial en face image 210. The replacement en face image 250 may then be stitched or merged with the replacement area 235 to obtain a stitched en face image 260.

[0048] In other embodiments, the selected ones of the plurality of image areas 212 may be directly covered or overlaid by the replacement en face image 250 without the need for the removal of the selected ones of the plurality of image areas 212.

[0049] In various embodiments, the replacement en face image 250 may have a different image resolution from the defective area 230 of the initial en face image 210.

[0050] Tn some instances, the stitched en face image 260 may include further artefacts and defective regions. Hence, an iteration process 280 may be performed by the computing module 1800 to further determine from the stitched en face image 260 a second defective area comprising a second artefact, wherein the second defective area corresponding to a second region of interest of the eye. In addition, the computing module 1800 may obtain a second replacement en face image of the second region of interest of the eye and replace the second defective area in the stitched en face image with the second replacement en face image.

[0051] FIGs. 3 and 4 illustrate different perspective views of an exemplary embodiment of an OCT system 100. The OCT system 100 may include an OCT measurement device 110 mounted on a mounting structure such as a structural arm 111. The OCT measurement device 1 10 may include a galvo scanner 1 12, a fixation target module 1 14 and a pair of stereo cameras 130 or depth cameras mounted to the structural arm 111. The stereo cameras 130 may be undermounted and positioned on opposing sides of the OCT measurement device 110 facing the face of the subject. The galvo scanner 112 may include an electrically tunable lens (ETL) and fast steering mirror (FSM). The actuator 120 may be a robotic arm coupled to the OCT measurement device 1 10. Tn addition, one or more infra-red (IR) light emitting diodes 1 16 (LEDs), for example four LEDs, may be provided on a front edge of the measurement device 110 to provide illumination on the face / eye of the subject for better contrast.

[0052] Referring to FIG. 5, in another exemplary embodiment, in addition to a galvo scanner112, the OCT system 100 may include an inline pupil camera 130a and a right pupil camera130b. The actuator 120 may be a linear stage integrated with the OCT measurement device 110 in a compact form.

[0053] In various embodiments as shown in FIGs. 3, 4 and 5, the structural mounts and components of the OCT system 100 may be in a modular setup or modular configuration. In some examples, the structural components may be made hollow in non-load bearing regions, and reinforced in load-bearing regions. The structural components may either be fabricated via additive manufacturing or 3D printing, or be directed molded using methods such as plastic injection molding, etc. The structural arm 111 may be mounted to a robotic platform / linear stage 120 controllable by the computing module 1800. The computing module 1800 may receive feedback from the depth cameras or 3D cameras, and control the actuator 120 (e.g. robotic arm or linear stage) to move the structural arm 1 1 1 and hence the OCT measurement device in alignment with the pupil of the subject 90.

[0054] In various embodiments, the OCT system 100 may be a Spectral Domain Optical Coherence Tomography (SD-OCT) system based on Spectral Domain Optical Coherence Tomography. SD-OCT method is a non-invasive, trans -pupillary technique that provides high- resolution cross-sectional images of the retina and the optic nerve head of the eye, using a principle analogous to B-scan ultrasound. A beam of infrared light strikes the retina and the delayed light reflected from the various layers of the retina and choroid is compared to a reference beam projected against a mirror. The elaboration of the three-dimensional image is made based on the principles of low-coherence interferometry. SD-OCT method provides detailed data on the morphology, reliably and reproducibly measures the thickness and volume of the optic nerve, peripapillary area and the various retinal layers at the macular level. In other embodiments, the OCT system 100 may be based on Time Domain Optical Coherence Tomography (TD-OCT) device and / or Swept Source Optical Coherence Tomography (SS-OCT).

[0055] FIG. 6 is a process flow chart showing a process flow 300 of the proposed OCT system according to various embodiments. At the beginning of an eye measurement or eye assessment session, the OCT system 100 may first perform an alignment 310 with the eye of a subject, followed by an initial scan 312 of the eye. The OCT system 100 may control the actuator to move the OCT measurement device in alignment with the pupil position, prior to obtaining the initial en face image 210. The OCT system 100 may first detect a position of the pupil of the eye or the pupil position and by controlling the actuator, to move the OCT measurement device to in alignment with a centre of the pupil. The automatic alignment process alleviates the need for a technician present to operate the OCT system 100 during scanning. Upon alignment of the OCT measurement device and the pupil of the subject within an acceptable tolerance / error, one or more initial en face images 210 of the eye may be captured by the OCT measurement device.

[0056] In various embodiments as shown in FIG. 6, performing an alignment 310 may include a sequential process of a rough alignment 310a followed by a fine alignment 310b. In various embodiments, the OCT system 100 may capture an image of a face of the subject and perform a rough alignment 310a between the OCT measurement device and the face using a face detection method. Upon completion of the rough alignment 310a, the OCT system 100 may detect a position of the pupil of the eye using a pupil detection method and perform a fine alignment 310b between the OCT measurement device and the pupil. The two-step process may improve the efficiency and speed of the alignment 310 process.

[0057] The quality of each initial en face image 210 may be assessed upon acquisition. This may be done by the detection of one or more artefacts in each of the initial en face image 210. In various embodiments, the en face image 210 may be input into a machine learning model 320 in determining whether one or more artefacts 220 or defects are present. The machine learning model 320 may also detect the location of the one or more artefacts 220.

[0058] Quality assessment 305 may be performed on the initial en face image 210. In some embodiments, quality assessment 305 may be performed based on one or more output of the machine learning model 320. If the initial en face image 210 passes quality assessment 305, i.e. the initial en face image 210 is free of artefact / defect, the initial en face image 210 may be used as an output en face image 270 suitable to be used for clinical assessment in process 370.

[0059] However, if the initial en face image 210 does not pass quality assessment 305, i.e. the initial en face image 210 comprises one or more artefacts 220 or defects, artefact localization 330 may be performed. Artefact localization 330 includes determining a defective area 230 of the initial en face image 210, in other words, determining an area which includes the artefact(s). In various embodiments, the initial en face image may be input into a machine learning model and the defective area 230 may be determined from an output of the machine learning model.

[0060] According to various embodiments, to determine the defective area 230, the initial en face image 210 may be segmented into a plurality of image areas 212. With information of the artefacts 220 / dcfcctivc area 230, selected ones of the plurality of image areas 212 corresponding to the defective area 230 may be determined. Concurrently or sequentially, upon the initial en face image 210 not passing quality assessment 305, the region of interest 240 of the eye which corresponds to the defective area 230 in the initial en face image 210 may also be determined. The galvo scanner of the OCT system 100 may steer and align the OCT measurement device with the region of interest 240 of the eye of the subject for re-scanning. Upon alignment of the OCT measurement device and the region of interest 240 of the eye of the subject, the OCT measurement device may perform a re-scan 350 of the region of interest 240 to obtain a replacement en face image 250.

[0061] In various embodiments, the machine learning model 320, the quality assessment 305 and the artefact localization 330 may be combined in a single process wherein the machinelearning model 320 may perform both the quality assessment 305 and the artefact localization 330 concurrently.

[0062] Moving on, the defective area 230 in the initial en face image 210 may be replaced by the replacement en face image 250 to obtain a stitched en face image 260. In various embodiments, the defective area 230 (or selected ones of the plurality of image areas 212) may be removed from the initial en face image 210 to define a replacement area 235 in the initial en face image 210. Thereafter, the replacement en face image 250 may be stitched onto the replacement area 235 of the initial en face image 210 to obtain the stitched en face image 260 to make a whole image of the entire scanned area of the eye.

[0063] The stitched en face image 260 may undergo another round of quality assessment 305 to further determine if any artefacts / defects are still present in the stitched en face image 260. If the stitched en face image 260 passes quality assessment 305, the stitched en face image 260 may be used as an output en face image 270 suitable to be used for clinical assessment in process 370. However, if one or more artefacts / defects arc still present in the stitched on face image 260, the stitched en face image 260 may undergo further steps of: artefact localization 330, alignment 340, re-scan 350 and stitching 360 processes iteratively, to remove all artefacts / defects in the en face image prior to being used as the output en face image 270.

[0064] The above process flow 300 may be performed automatically for each en face scan in a feedback loop manner. The process may be iteratively performed until all artefacts are removed from the en face image. In various embodiments, multiple output en face images 270 which includes one or more stitched en face images 260, may be used to form a volumetric scan of the eye of the subject which is free of artefacts / defects for clinical assessment 370.

[0065] The process flow 300 departs from conventional OCT processes in which the technician will only reahze the presence of defects or poor image quality at the end of the OCTscan and thus have to repeat the entire OCT scan. The feature of automatic correction saves time and cost during the OCT measurement process.

[0066] In various embodiments, upon completion of the OCT scans, the process flow 300 may include a clinical assessment 370, in which a further assessment process or algorithm may be used for the detection of eye anomaly in the OCT scan which may require medical attention. Such assessment process may be performed by another machine learning model. The subject may receive an outcome from the OCT system 100 translatable or resulting in an appointment to seek help from a medical professional, such as a general practitioner at the clinic or hospital.

[0067] According to various embodiments, in the process of alignment 310, a depth camera may be used to identify the face and eyes of the subject. The depth camera may detect a pupil position of the eye from depth images obtained from the depth camera. Face detection may be carried out using a single-shot detector (SSD) model using an OpenCV package to obtain a face region of the subject. In addition, the eyes may be detected within the face region using a Haar- fcaturc classifier. Rule-based methods may be used which ensure high-confidence detections and reducing false positives. Once the eyes are identified, the respective 3D coordinates relative to the depth camera and OCT measurement device / actuator may be obtained by aligning the depth stream with the RGB stream.

[0068] For pupil detection, cameras may be used with signal communication managed by the Spinnaker SDK in C++ and Lab VIEW. The detection algorithm may include thresholding, morphological operations, and contour detection. To achieve low latency, traditional image processing techniques and 4x4-pixel binning may be used, with four InfraRed Light Emitting Diodes (IR-LEDs, see FIGs. 3A, 3B) providing illumination for better contrast. Triangulation of the pupil may be performed after stereo calibration of the depth cameras using a chessboard target and OpenCV library. Depth information may be obtained relative to the left camera using the calibrated positions of the right camera and the pupil centroids from both cameras. FIG. 7shows the various intermediate states of image processing of the camera stream to detect pupil contour from an eye image.

[0069] FIGs. 8A and 8B illustrate exemplary artefacts 220 in the initial en face images 210 detected by the proposed OCT system. FIG. 8A shows an exemplary initial en face image 210 with artefacts (break 220a and shadow 220b) detected by the machine learning model 320. Similarly, FIG. 8B shows another exemplary initial en face image 210 with artefacts (break 220a and shadow 220b) detected by the machine learning model 320. It may be appreciated that the examples for artefacts 220 are not limited to break(s) and shadow(s), and may include artefacts or defects of other nature, such as tilt and out-of-window artefacts. In addition, FIG. 8C shows an exemplary artefact 222 in a B-scan obtained by the OCT system. The artefact 222 in the B-scan may manifest in one or more en face images.

[0070] According to various embodiments, detection of artefacts 220 in the initial en face image may be performed by a machine learning model 320, such as a deep learning model. FIG. 9 is a schematic diagram showing a structure of a machine learning model 320 used in the OCT system according to various embodiments. The machine learning model 320 may be a U-Net model 320 comprising multiple connected neural network layers. The U-Net model may include a contracting path 322 which comprises multiple encoder layers 324, and an expanding path 326 which comprises multiple decoder layers 328. Skip connections 325 may be provided between respective pairs of encoder layer 324 and decoder layer 328. In other embodiments, the machine learning model may be a transformer model.

[0071] FIG. 10A shows a training sample of an initial en face image and a corresponding label for training the U-Net model 320. FIG. 10B shows a test sample of initial en face image and a corresponding output from the U-Net model 320. Output from the U-Net model 320 may locate artefacts such as breaks 320a and shadows 320b from the initial en face image. In addition, areas corresponding to the optical nerve head 232 may also be shown in the outputfrom the U-Net model 320. The U-Net model 320 may be trained with manual data of the eye of multiple subjects. The training data may be OCT images, such as en face images, captured from test subjects and are labelled manually. In other implementations, the en face images may be labelled using automated and semi-automated methods. The test subjects may be of different demographics, such as: age range: age 40 and above, race: any race, gender: both genders, health status: various health status, etc. However, as the proposed OCT system 100 is a screening system, data from test subjects younger than 40 years old may also be included.

[0072] According to various embodiments as shown in FIG. 11, in the process of artefact localization 330, the initial en face image 210 may be segmented or divided into a plurality of image areas 212 or a plurality of tiles. Selected ones 213 of the plurality of image areas 212 (or selected tiles 213) corresponding to the defective area 230 may be determined. The divi ion of the plurality of image areas 212 and the detection of the artefacts in the selected ones 213 may be performed by another machine learning model, such as deep learning models, or alternatively by non-machinc learning algorithms / programming. Selected ones 213 of the plurality of image areas 212 or the selected tiles 213 may be removed from the initial en face image 210 to define a replacement area 235 in the initial en face image 210. In some embodiments, the replacement area 235 may be a continuous area in the initial en face image 210 previously occupied by the selected ones 213 of the plurality of image areas 212 or the selected tiles 213.

[0073] Still referring to FIG. 11, in various embodiments, each of the plurality of image areas 212 may be segmented or divided in an array structure. In various embodiments, the plurality of image areas 212 may be formed into a NxN array or a square array, e.g. a 4x4 array as shown in FIG. 10. In other embodiments, the plurality of image areas 212 may be formed into a AxB array, such that the number of rows in the array is not equal to the number of columns. In various embodiments, each of the plurality of image areas 212 may have a uniform size, or in other words, each of the plurality of image areas 212 may have a uniform area.

[0074] Departing from conventional thinking, initial observations have shown that increasing the resolution of the plurality of image areas 212 (or the number of tiles) results in a diminish in accuracy of matching between prediction and mask (see Table 1), or in other words, diminish in accuracy of the detection of artefacts. Therefore, in various embodiments, a resolution of the NxN array may be inversely proportional to an accuracy of detection of the artefact. This is a surprising outcome in contrary' to the conventional understanding that a higher resolution yields higher accuracy. In some embodiments, a minimum size may be enforced on each of the image areas 212 such that each of the plurality of image areas 212 may have a size larger than a minimum size or a threshold size.

[0075] Referring to Table 1, tiling protocol (AxB) refers to the granularity of the artefact detection, with a 4x4 tiling protocol indicating four tiles (or four image areas) in the horizontal direction (A) and four tiles (or four image areas) in the vertical direction (B). Opaques refer to regions of localized signal loss, which appear as darkened regions and Breaks refer to discontinuous regions due to eye movements during image acquisition.Table 1 . Accuracy of matching between the prediction and mask.

[0076] Referring to FIG. 12, in various embodiments, the plurality of image areas 212 may have different dimensions / sizes. In an exemplary embodiment, first ones 212a of the plurality of image areas disposed at a center of the initial en face image 210 may have a different size in relative to second ones 212b of the plurality of image areas disposed at a periphery of the initial en face image. For example, the image areas 212a are smaller in area than image area 212b, as shown in FIG. 12. This may be useful for situations where artefacts 220 (dashed box in FIG. 12 showing a break artefact) are primarily located near or adjacent to the periphery of the initial en face image 210. Therefore, more computing resources are concentrated at the periphery for computational efficiency.

[0077] FIG. 13 shows a prototype of the proposed OCT system. In the prototype OCT system is configured to perform an image processing method. In the image processing method, the artefacts in the images are located using a deep learning model and the regions (defective area) in which the artefacts are located are determined by the deep learning model. For image segmentation, the U-Nct convolutional neural network is used due to suitability for biomedical scans. In this case, the U-Net is used for the segmentation of artefacts in the retinal en face image. The two key artefacts detected are breaks and shadows. Breaks refer to a misalignment in consecutive B-scans due to patient movement, while shadows refer to a reduction or loss in signal due to obscuration. The segmentation model is trained based on images which are manually labelled. Further, the labelled images does not only include imaging artefacts, but also include the optic nerve head of the eye as a positive control.

[0078] In conventional OCT systems, manual alignment of the eye often results in the present of significant motion artefact(s) and vignetting artefact(s). Provision of the automatic alignment serves to alleviate such avoidable artefacts. This is due to the implementation of the auto-alignment feature resulting in a better or more optimal placement of OCT measurement device relative to the pupil pivot. FIG. 14A shows an exemplary B-scan captured / taken usingthe prototype OCT system. FIG. 14B comprises en face images taken from 6 different human subjects, using manual alignment. Vignetting and motion artefacts are obvious due to poor compensation for subject motion. FIG. 14C comprises en face images taken from both eyes of 4 different human subjects using the auto-alignment features of the prototype OCT system.

[0079] According to another aspect of the invention, an image processing method 700 is disclosed according to various embodiments. The method 700 comprises: in 710, obtaining an initial en face image of an eye of a subject; in 720, determining from the initial en face image a defective area comprising an artefact, the defective area corresponding to a region of interest of the eye; in 730, obtaining a replacement en face image of the region of interest of the eye; and in 740, replacing the defective area in the initial en face image with the replacement en face image to obtain a stitched en face image.

[0080] In various embodiments, the method may further include: segmenting the initial en face image into a plurality of image areas; and determining selected ones of the plurality of image areas corresponding to the defective area. In various embodiments, the method may further include: removing the selected ones of the plurality of image areas from the initial en face image to provide a replacement area in the initial en face image; and stitching the replacement en face image to the replacement area to obtain the stitched en face image, wherein the replacement en face image has a different image resolution from the defective area. In various embodiments, the method may further include: inputting the initial en face image into a machine learning model; and determining the defective area from an output of the machine learning model.

[0081] In various embodiments, the method may be iterative performed when there are multiple artefacts and multiple defective areas. The method may further include: determining from the stitched en face image a second defective area comprising a second artefact, the second defective area corresponding to a second region of interest of the eye; obtaining a secondreplacement en face image of the second region of interest of the eye; and replacing the second defective area in the stitched en face image with the second replacement en face image. In various embodiments, the method may further include: forming a volumetric scan from a plurality of stitched en face images.

[0082] In various embodiments, the method may further include: detecting a pupil position of the eye from depth images obtained from a depth camera. In various embodiments, the method may further include: moving an OCT measurement device in alignment with the pupil position, prior to obtaining the initial en face image. In various embodiments, the method may further include: capturing an image of a face of the subject; and performing a rough alignment of the face relative to the OCT measurement device.

[0083] In accordance with embodiments of the present disclosure, a block diagram representative of components of processing system 1800 or that may be provided to carry out the digital signal processing functions or computations in accordance with embodiments of the disclosure, or any other modules or sub-modules of the system is illustrated in FIG. 16. One skilled in the art will recognize that the exact configuration of each processing system provided within these modules may be different and the exact configuration of processing system 1800 may vary and the arrangement illustrated in FIG. 16 is provided by way of example only.

[0084] In embodiments of the disclosure, processing system 1800 may comprise controller1801 and user interface 1802. User interface 1802 is arranged to enable manual interactions between a user and the computing module as required and for this purpose includes the input / output components required for the user to enter instructions to provide updates to each of these modules. A person skilled in the art will recognize that components of user interface1802 may vary from embodiment to embodiment but will typically include one or more of display 1840, keyboard 1835 and optical device 1836.

[0085] Controller 1801 is in data communication with user interface 1802 via bus 1815 and includes memory 1820, processing unit, processing element or processor 1805 mounted on a circuit board that processes instructions and data for performing the method of this embodiment, an operating system 1806, an input / output (I / O) interface 1830 for communicating with user interface 1802 and a communications interface, in this embodiment in the form of a network card 1850. Network card 1850 may, for example, be utilized to send data from these modules via a wired or wireless network to other processing devices or to receive data via the wired or wireless network. Wireless networks that may be utilized by network card 1850 include, but are not limited to, Wireless-Fidelity (Wi-Fi), Bluetooth, Near Field Communication (NFC), cellular networks, satellite networks, telecommunication networks. Wide Area Networks (WAN) and etc.

[0086] Memory 1820 and operating system 1806 are in data communication with processor 1805 via bus 1810. The memory components include both volatile and non-volatile memory and more than one of each type of memory, including Random Access Memory (RAM) 1823, Read Only Memory (ROM) 1825 and a mass storage device 1845, the last comprising one or more solid-state drives (SSDs). One skilled in the art will recognize that the memory' components described above comprise non-transitory computer-readable media and shall be taken to comprise all computer-readable media except for a transitory, propagating signal. Typically, the instructions are stored as program code in the memory components but can also be hardwired. Memory 1820 may include a kernel and / or programming modules such as a software application that may be stored in either volatile or non-volatile memory.

[0087] Herein the term “processor” is used to refer generically to any device or component that can process such instructions and may include: a microprocessor, a processing unit, a plurality of processing elements, a microcontroller, a programmable logic device or any other type of computational device. That is, processor 1805 may be provided by any suitable logiccircuitry for receiving inputs, processing them in accordance with instructions stored in memory and generating outputs (for example to the memory components or on display 1840). In this embodiment, processor 1805 may be a single core or multi-core processor with memory addressable space. In one example, processor 1805 may be multi-core, comprising — for example — an 8 core CPU. In another example, it could be a cluster of CPU cores operating in parallel to accelerate computations.

[0088] All examples described herein, whether of methods, materials, or products, are presented for the purpose of illustration and to aid understanding and are not intended to be limiting or exhaustive. Modifications may be made by one of ordinary skill in the art without departing from the scope of the invention as claimed.

Claims

CLAIMS1. An optical coherence tomography (OCT) system, comprising: a computing module comprising a processing unit; and a non-transitory media readable by the processing unit, the media storing instructions that when executed by the processing unit, causes the processing unit to: obtain an initial en face image of an eye of a subject; determine from the initial en face image a defective area comprising an artefact, the defective area corresponding to a region of interest of the eye; obtain a replacement en face image of the region of interest of the eye; and replace the defective area in the initial en face image with the replacement en face image to obtain a stitched en face image.

2. The optical coherence tomography system as recited in claim 1, wherein the instructions further cause the processing unit to: segment the initial en face image into a plurality of image areas; and determine selected ones of the plurality of image areas corresponding to the defective area.

3. The optical coherence tomography system as recited in claim 2, wherein each of the plurality of image areas has a uniform size.

4. The optical coherence tomography system as recited in claim 3, wherein the plurality of image areas are formed into a N x N array.

5. The optical coherence tomography system as recited in claim 4, wherein a resolution of the N x N array is inversely proportional to an accuracy of detection of the artefact.

6. The optical coherence tomography system as recited in any one of claims 2 to 5. wherein the instructions further causes the processing unit to: remove the selected ones of the plurality of image areas from the initial en face image to define a replacement area in the initial en face image; and stitch the replacement en face image to the replacement area to obtain the stitched en face image, wherein the replacement en face image has a different image resolution from the defective area.

7. The optical coherence tomography system as recited in any one of the above claims, wherein the instructions further cause the processing unit to: determine from the stitched en face image a second defective area comprising a second artefact, the second defective area corresponding to a second region of interest of the eye; obtain a second replacement en face image of the second region of interest of the eye; and replace the second defective area in the stitched en face image with the second replacement en face image.

8. The optical coherence tomography system as recited in any one of the above claims, wherein the instructions further cause the processing unit to: input the initial en faceimage into a machine learning model; and determine the defective area from an output of the machine learning model.

9. The optical coherence tomography system as recited in any one of the above claims, further comprising: an OCT measurement device for capturing en face images of the eye of the subject; and an actuator coupled to the OCT measurement device to vary a position of the OCT measurement device relative to the eye of the subject.

10. The optical coherence tomography system as recited in claim 9, further comprising a depth camera, wherein the instructions further cause the processing unit to: detect a pupil position of the eye from depth images obtained from the depth camera.

11. The optical coherence tomography system as recited in claim 10, wherein the instructions further cause the processing unit to: control the actuator to move the OCT measurement device in alignment with the pupil position, prior to obtaining the initial en face image.

12. The optical coherence tomography system as recited in any one of claims 9 to 11, wherein the instructions further cause the processing unit to: capture an image of a face of the subject; and perform a rough alignment of the face relative to the OCT measurement device.

13. The optical coherence tomography system as recited in any one of the above claims, wherein the optical coherence tomography system is a portable system.

14. An image processing method for optical coherence tomography, comprising: obtaining an initial en face image of an eye of a subject; determining from the initial en face image a defective area comprising an artefact, the defective area corresponding to a region of interest of the eye; obtaining a replacement en face image of the region of interest of the eye; and replacing the defective area in the initial en face image with the replacement en face image to obtain a stitched en face image.

15. The image processing method as recited in claim 14, further comprising: segmenting the initial en face image into a plurality of image areas; and determining selected ones of the plurality of image areas corresponding to the defective area.

16. The image processing method as recited in claim 15, wherein each of the plurality of image areas has a uniform size.

17. The image processing method as recited in claim 16, wherein the plurality of image areas are formed into a N x N array.

18. The image processing method as recited in claim 17, wherein a resolution of the N x N array is inversely proportional to an accuracy of detection of the artefact.

19. The image processing method as recited in any one of claims 15 to 18, further comprising: removing the selected ones of the plurality of image areas from the initial en face image to provide a replacement area in the initial en face image; and stitching the replacement en face image to the replacement area to obtain the stitched en face image, wherein the replacement en face image has a different image resolution from the defective area.

20. The image processing method as recited in any one of claims 14 to 19, further comprising: determining from the stitched en face image a second defective area comprising a second artefact, the second defective area corresponding to a second region of interest of the eye; obtaining a second replacement en face image of the second region of interest of the eye; and replacing the second defective area in the stitched en face image with the second replacement en face image.

21. The image processing method as recited in any one of claims 14 to 20, further comprising: inputting the initial en face image into a machine learning model; and determining the defective area from an output of the machine learning model.

22. The image processing method as recited in any one of claims 14 to 21, further comprising: detecting a pupil position of the eye from depth images obtained from a depth camera.

23. The image processing method as recited in claim 22, further comprising: moving an OCT measurement device in alignment with the pupil position, prior to obtaining the initial en face image.

24. The image processing method as recited in any one of claims 22 and 23, further comprising: capturing an image of a face of the subject; and performing a rough alignment of the face relative to the OCT measurement device.

Images