OCT equipment and scanning method thereof, ophthalmic imaging device and system

Abstract

The present invention provides an OCT device and a scanning method thereof, an ophthalmic imaging device and a system, and relates to the field of ophthalmic detection technology. The OCT device is integrated with a fundus imaging device. The method comprises: obtaining a first fundus image and a second fundus image of a target object, the second fundus image is obtained by scanning the target object by the fundus imaging device, and the imaging range of the second fundus image is smaller than the imaging range of the first fundus image; determining a target scanning area of ​​the OCT device on the target object according to the region of interest in the first fundus image and the second fundus image; and performing OCT / OCTA scanning on the target scanning area. The present invention combines a first fundus image with a larger imaging range with a second fundus image with a smaller imaging range, and can quickly and accurately locate the target scanning area of ​​the OCT device, realize accurate positioning and efficient scanning of target areas such as lesion areas, thereby improving the accuracy and scanning efficiency of the OCT device scanning.

Description

Technical Field

[0001] The present invention relates to the field of ophthalmic detection technology, and in particular to an OCT device and a scanning method thereof, an ophthalmic imaging device and a system. Background Art

[0002] Optical coherence tomography (OCT) is a commonly used device for ophthalmic examination. Before performing OCT / OCTA (Optical Coherence Tomography Angiography), the OCT device usually needs to locate the scanning area through a fundus imaging device. However, the field of view of the fundus imaging device is small, and the fundus image taken is not clear enough, which makes it inconvenient to accurately locate the required scanning area, resulting in low scanning efficiency. Summary of the invention

[0003] The object of the present invention is to provide an OCT device and a scanning method thereof, an ophthalmic imaging device and a system, so as to improve the accuracy and scanning efficiency of OCT scanning.

[0004] In a first aspect, the present invention provides a scanning method of an OCT device, which is applied to an OCT device integrated with a fundus imaging device; the method comprises:

[0005] Acquire a first fundus image of the target object, where the first fundus image is obtained by photographing the target object with a fundus camera;

[0006] Acquire a second fundus image of the target object, where the second fundus image is obtained by scanning the target object with a fundus imaging device, and an imaging range of the second fundus image is smaller than an imaging range of the first fundus image;

[0007] Determine a target scanning area of ​​the OCT device on the target object according to the region of interest in the first fundus image and the second fundus image;

[0008] Perform OCT / OCTA scan on the target scanning area.

[0009] Further, obtaining a first fundus image of the target object includes:

[0010] According to the object identifier of the target object, a first fundus image of the target object is acquired from a database or a fundus camera used for photographing the target object.

[0011] Further, determining a target scanning area of ​​the OCT device on the target object according to the region of interest in the first fundus image and the second fundus image includes:

[0012] Acquire a region of interest in a first fundus image;

[0013] determining a target area corresponding to the region of interest according to the second fundus image;

[0014] According to the target area, a target scanning area of ​​the OCT device on the target object is determined; wherein the target scanning area covers the target area.

[0015] Further, determining a target area corresponding to the region of interest according to the second fundus image includes:

[0016] Determine position information matching the region of interest in the second fundus image, and determine the target region according to the region of interest and the position information; or,

[0017] The first fundus image and the second fundus image are registered to obtain registration information, and the target area is obtained according to the registration information and the area of ​​interest.

[0018] Further, the center of the target scanning area is aligned with the center of the target area.

[0019] Furthermore, an OCT / OCTA scan is performed on the target scanning area, including:

[0020] Acquire a current second fundus image of the target object in real time;

[0021] Determine the motion information of the target object according to the current second fundus image and the reference second fundus image; wherein the reference second fundus image is one of the acquired historical second fundus images;

[0022] Determine the current scanning position according to the motion information and the target scanning area;

[0023] Perform OCT / OCTA scan on the current scan position.

[0024] In a second aspect, the present invention further provides an OCT device, including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements the method of the first aspect when executing the computer program.

[0025] In a third aspect, the present invention further provides an ophthalmic imaging device, comprising:

[0026] A display module, used for displaying on a screen a second fundus image obtained by scanning the target object with a fundus imaging device;

[0027] A marking module is used to mark a target scanning area for performing OCT / OCTA scanning on a target object on a second fundus image, wherein the target scanning area is determined based on an area of ​​interest in a first fundus image of the target object, wherein the first fundus image is obtained by photographing the target object with a fundus camera, and an imaging range of the first fundus image is greater than an imaging range of the second fundus image.

[0028] Furthermore, the ophthalmic imaging device further comprises:

[0029] A determination module, used to determine a current scanning area of ​​the OCT device to which the fundus imaging device belongs according to a scanning protocol;

[0030] The moving module is used to move the current scanning area until the current scanning area covers the target scanning area.

[0031] In a fourth aspect, the present invention further provides an ophthalmic imaging system, comprising the OCT device of the second aspect, and further comprising a data transmission platform and a fundus camera, wherein the data transmission platform is connected to the OCT device and the fundus camera respectively;

[0032] The data transmission platform is used to store the first fundus image uploaded by the fundus camera and transmit the corresponding first fundus image to the OCT device.

[0033] In a fifth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method of the first aspect is executed.

[0034] In the OCT device and scanning method thereof, ophthalmic imaging device and system provided by the present invention, the OCT device is integrated with a fundus imaging device, and the method comprises: obtaining a first fundus image of a target object, the first fundus image is obtained by photographing the target object with a fundus camera; obtaining a second fundus image of the target object, the second fundus image is obtained by scanning the target object with a fundus imaging device, and the imaging range of the second fundus image is smaller than the imaging range of the first fundus image; determining a target scanning area of ​​the OCT device on the target object according to the area of ​​interest in the first fundus image and the second fundus image; and performing OCT / OCTA scanning on the target scanning area. The imaging range of the first fundus image is larger, the fundus area information is more comprehensive, and the area of ​​interest can be accurately located. Therefore, the area of ​​interest in the first fundus image is combined with the second fundus image with a smaller imaging range, so that the target scanning area of ​​the OCT device can be quickly and accurately located, and the accurate positioning and efficient scanning of target areas such as lesion areas can be achieved, thereby improving the accuracy and scanning efficiency of OCT / OCTA scanning. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 A schematic diagram of a flow chart of a scanning method of an OCT device provided in an embodiment of the present invention;

[0037] Figure 2 A fundus color photograph provided by an embodiment of the present invention;

[0038] Figure 3 An OCT scanning result provided by an embodiment of the present invention;

[0039] Figure 4 A schematic diagram of an initial scanning area of ​​an OCT device provided in an embodiment of the present invention;

[0040] Figure 5 A schematic diagram of a target scanning area of ​​an OCT device provided in an embodiment of the present invention;

[0041] Figure 6 A schematic diagram of the structure of a scanning device of an OCT device provided in an embodiment of the present invention;

[0042] Figure 7 A schematic diagram of the structure of an OCT device provided in an embodiment of the present invention;

[0043] Figure 8 A schematic diagram of the structure of an ophthalmic imaging device provided by an embodiment of the present invention;

[0044] Fig. 9 A schematic diagram of the structure of an ophthalmic imaging system provided by an embodiment of the present invention;

[0045] Fig.10 A schematic diagram of a connection architecture of an ophthalmic imaging system provided by an embodiment of the present invention;

[0046] Fig.11 A schematic diagram of a communication data format provided by an embodiment of the present invention;

[0047] Fig.12 A schematic diagram of a process for identifying a lesion area using a convolutional neural network model provided by an embodiment of the present invention;

[0048] Fig.13 A schematic diagram of the overall processing flow of an ophthalmic imaging system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] Commonly used equipment for ophthalmic examinations include OCT equipment and fundus cameras. For patients with poor fundus conditions, fundus color photographs taken by fundus cameras are easier to observe the location and morphology of lesions than fundus grayscale images collected by fundus imaging equipment integrated in the OCT equipment. In addition, the fundus range captured by fundus color photographs is larger than that of fundus grayscale images, and the marginal lesion area not shown in the fundus grayscale image can be located. OCT scanning combined with fundus color photographs can more accurately scan the fundus lesion area, thereby improving the effectiveness of the scanning results. Based on this, an OCT device and its scanning method, an ophthalmic imaging device and system provided in an embodiment of the present invention can improve the accuracy of OCT scanning, and achieve accurate positioning and efficient scanning of the lesion area by combining fundus color photographs and AI (Artificial Intelligence) technology.

[0051] To facilitate understanding of this embodiment, a scanning method of an OCT device disclosed in an embodiment of the present invention is first introduced in detail.

[0052] The embodiment of the present invention provides a scanning method of an OCT device, which can be applied to an OCT device integrated with a fundus imaging device, and the fundus imaging device is used to locate a scanning area for the OCT device. The fundus imaging device can be, but is not limited to, a confocal scanning system ophthalmoscope (CSSO), a confocal laser scanning ophthalmoscope (CSLO), a scanning laser ophthalmoscope (SLO), or a line scanning ophthalmoscope (LSO).

[0053] See also Figure 1 The schematic diagram of the flow chart of a scanning method of an OCT device is shown, and the scanning method of the OCT device mainly includes the following steps S110 to S140:

[0054] Step S110 , obtaining a first fundus image of the target object, where the first fundus image is obtained by photographing the target object with a fundus camera.

[0055] The target object may be a person or animal that needs to undergo an OCT scan, and the first fundus image may be, but is not limited to, a color fundus photograph of the target object taken by a fundus camera. Optionally, the color fundus photograph may be taken by an ultra-wide-angle fundus camera. Specifically, the ultra-wide-angle fundus camera may be equipped with a dual-focus concave elliptical mirror and a confocal scanning laser ophthalmoscope. By utilizing the point-to-point reflection laser principle of the elliptical mirror and measuring the central visual angle within the eye, it can achieve an ultra-large fundus field of view of 200° horizontally and 170° vertically under small pupil conditions. By capturing a single anteroposterior (posterior pole) fundus image, a clear image can be formed to the ora serrata. In conjunction with the nasotemporal eye position guidance, a fundus range of 220° to 240° horizontally can even be achieved. In addition, the ultra-wide-angle fundus camera uses the red-green dual-color laser imaging principle to achieve color images of the retina, thereby improving the recognition of the diseased area. Figure 2 As shown, the color fundus photos taken by the ultra-wide-angle fundus camera can clearly show areas such as arteries, veins, optic discs and macula, facilitating accurate identification of lesion areas.

[0056] The fundus camera can capture and store a first fundus image corresponding to the target object, and can also store the first fundus image in a preset database; wherein the database can be a cloud database or a data center. In one possible implementation, the fundus camera can transmit the first fundus image to a preset data transmission platform, which is stored in a database of the data transmission platform and sent to the OCT device through the data transmission platform; in another possible implementation, the fundus camera can be integrated into the OCT device or communicated with the OCT device, and the OCT device can directly obtain the first fundus image corresponding to the target object from the fundus camera that captures the target object. Based on this, the above step S110 may include: according to the object identifier of the target object, obtaining the first fundus image of the target object from the database or the fundus camera used to capture the target object.

[0057] In specific implementation, taking the first fundus image as a fundus color photo as an example, the OCT device can send a data acquisition request carrying an object identifier to the data transmission platform or the fundus camera. The data transmission platform or the fundus camera can query the fundus color photo according to the object identifier, and send the queried fundus color photo to the OCT device, and the OCT device can receive the fundus color photo of the target object. Among them, the object identifier is used to uniquely identify the target object. Optionally, the object identifier can include one or more of the ID number, medical number, and mobile phone number, and can also include one or more of the basic information such as name, age, and address. The data acquisition request can also carry the device identifier of the fundus camera, such as SN (Serial Number). When the data transmission platform fails to find the corresponding fundus color photo, it can determine the IP address of the fundus camera according to the device identifier of the fundus camera, actively obtain the corresponding fundus color photo from the corresponding fundus camera, and then transmit it to the OCT device.

[0058] Step S120, acquiring a second fundus image of the target object, where the second fundus image is obtained by scanning the target object with a fundus imaging device, and an imaging range of the second fundus image is smaller than an imaging range of the first fundus image.

[0059] The main difference between the first fundus image and the second fundus image is that they are obtained by imaging devices with different imaging ranges. The second fundus image can be obtained by laser scanning the target object by the fundus imaging device, and the second fundus image can be, but is not limited to, a CSSO fundus image, a CSLO fundus image, an SLO fundus image, or an LSO fundus image. The first fundus image is preferably a fundus color photo, which has a larger imaging range and is a color image, making it easier to identify areas of interest such as lesion areas, and the detection rate of fundus lesions is significantly greater than that of second fundus images such as LSO images, and the missed diagnosis rate is lower.

[0060] Step S130: determining a target scanning area of ​​the OCT device on the target object according to the region of interest in the first fundus image and the second fundus image.

[0061] The above-mentioned region of interest may be a lesion area or a user-specified area, etc., which requires OCT / OCTA scanning. The user-specified area may be, for example, a key area containing key information such as the macula and the optic disc. Since the OCT device is integrated with the fundus imaging device, the two may use the same image coordinate system. Therefore, the target area corresponding to the region of interest in the coordinate system of the second fundus image may be determined first, and then the target scanning area of ​​the OCT device may be determined based on the target area. It should be noted that since the imaging range of the second fundus image is relatively small, in the coordinate system of the second fundus image, the target area corresponding to the region of interest in the first fundus image may be located entirely or partially in the second fundus image, or may be located outside the second fundus image.

[0062] In some possible embodiments, the above step S130 may include the following sub-steps:

[0063] Step S131, acquiring a region of interest in a first fundus image.

[0064] The region of interest in the first fundus image may be manually annotated by a user in advance, or may be automatically identified by a preset recognition algorithm. The first fundus image acquired by the OCT device may be the first fundus image that requires region of interest recognition, or may be the first fundus image with a region of interest recognition result, that is, the region of interest recognition step of the first fundus image may be performed by the OCT device, or may be performed by a data transmission platform or a fundus camera, etc.

[0065] In a possible implementation, the ROI identification result is an ROI identifier, the first fundus image carries the ROI identifier, and the area indicated by the ROI identifier in the first fundus image can be determined as the ROI. The ROI identifier can be, but is not limited to, a detection box or contour corresponding to a lesion area / user-specified area.

[0066] In another possible implementation, after the OCT device acquires the first fundus image, it can use the trained recognition model to identify the lesion area of ​​the first fundus image to obtain the region of interest; wherein the recognition model is obtained by training a convolutional neural network model based on the first fundus image data set with the lesion area label.

[0067] Step S132: determining a target area corresponding to the region of interest according to the second fundus image.

[0068] This embodiment provides two methods for determining the target area, which are as follows:

[0069] Method 1: Determine position information matching the region of interest in the second fundus image, and determine the target region based on the region of interest and the position information.

[0070] Method 1 is applicable to the case where the target area is at least partially located in the second fundus image. Specifically, feature matching can be performed based on the feature information of the region of interest and the second fundus image to obtain matching position information; then it is determined whether there is a first sub-region in the region of interest that fails to match the second fundus image; if not, it is determined that the target area is entirely located in the second fundus image, and the area indicated by the position information can be directly determined as the target area; if so, it is determined that the target area is only partially located in the second fundus image, and the first sub-region can be mapped to the coordinate system where the second fundus image is located according to the matching relationship between the position information and the region of interest to obtain the second sub-region, and the area obtained by merging the second sub-region with the area indicated by the position information is used as the target area.

[0071] Method 2: Perform image registration on the first fundus image and the second fundus image to obtain registration information, and obtain the target area according to the registration information and the area of ​​interest.

[0072] Method 2 is a general method, which is applicable not only to the case where the target area is at least partially located in the second fundus image, but also to the case where the target area is not located in the second fundus image at all. Specifically, image registration can be performed based on the feature information of the first fundus image and the feature information of the second fundus image, and then based on the obtained registration information, the region of interest is mapped to the coordinate system of the second fundus image to obtain the target area. In specific implementation, the features such as the optic disc, the fovea, and the large blood vessels can be used to match the position relationship (i.e., image registration), and then the region of interest such as the lesion area can be mapped.

[0073] A preset image feature matching algorithm can be used to match the second fundus image with the region of interest, or the first fundus image and the second fundus image can be registered. The image feature matching algorithm can be selected according to actual needs and is not limited here. For example, the image feature matching algorithm can use one or more of the following: SIFT (Scale Invariant Feature Transform), SURF (Speeded Up Robust Features), ORB (Oriented Fast and Rotated Brief).

[0074] In order to further improve the scanning efficiency, when determining the target area based on the registration information, multiple sampling points can be selected on the contour of the region of interest, and then the matching points of each sampling point in the second fundus image are determined according to the registration information, and then curve fitting is performed on each matching point, and the area surrounded by the fitted target curve is determined as the target area. In this way, the target area can be obtained without acquiring matching points for each point in the region of interest, which greatly reduces the amount of calculation and shortens the calculation time, thereby improving the scanning efficiency.

[0075] The above-mentioned registration information is the position transformation relationship between the first fundus image and the second fundus image, and the matching point of any pixel point in the first fundus image in the second fundus image can be determined based on the registration information. For example, the registration information is a transformation matrix, and the coordinates of a pixel point in the first fundus image can be multiplied by the transformation matrix to obtain the coordinates of the corresponding matching point in the second fundus image.

[0076] Step S133, determining a target scanning area of ​​the OCT device on the target object according to the target area; wherein the target scanning area covers the target area.

[0077] In a possible implementation, the target scanning area can be determined according to a preset scanning protocol and a target area. After the user selects a scanning protocol, the scanning range (i.e., the size of the scanning area) of the OCT device is determined, and the target scanning area that can cover the target area can be determined according to the scanning range corresponding to the scanning protocol.

[0078] In another possible implementation, a matching target scanning range can be determined based on the target area, and then the target scanning area can be determined based on the target area and the target scanning range, and then the scanning protocol corresponding to the target scanning range can be selected for OCT / OCTA scanning. The OCT device stores optional scanning ranges corresponding to multiple scanning protocols. When the user has not selected a scanning protocol, the target scanning range can be selected based on the coverage size of the target area, and the target scanning range is not less than the coverage range of the target area, so as to determine the target scanning area that can cover the target area based on the target scanning range. In order to improve the scanning efficiency, the smallest optional scanning range among the optional scanning ranges that satisfies the condition of not being less than the coverage range of the target area is preferably used as the target scanning range.

[0079] Optionally, the center of the target scanning area is aligned with the center of the target area. This can ensure a comprehensive scan of the target area. When the target area is an irregular shape, the center of the target area can be the center of mass or center of gravity of the target area.

[0080] Step S140: performing OCT / OCTA scanning on the target scanning area.

[0081] The OCT device can automatically adjust the scanning position based on the target scanning area to achieve OCT / OCTA scanning of the target area. After the scanning position of the OCT device is adjusted to the target scanning area, the OCT / OCTA scanning of the target object can be started.

[0082] During the scanning process, the OCT device can be combined with CSSO for real-time tracking, and accurately scan the target area such as the lesion area to collect detailed data of the target area such as the lesion area. Based on this, the above step S140 can include: acquiring the current second fundus image of the target object in real time; determining the motion information of the target object according to the current second fundus image and the reference second fundus image; wherein the reference second fundus image is one of the acquired historical second fundus images; determining the current scanning position according to the motion information and the target scanning area; and performing OCT / OCTA scanning on the current scanning position.

[0083] During the scanning process of the OCT device, the eyes of the target object may move, so it is necessary to update the scanning position in real time to achieve accurate scanning of the target scanning area. The fundus imaging device will acquire the second fundus image in real time. The second fundus image acquired at the current moment is called the current second fundus image, and the second fundus image acquired before the current moment is called the historical second fundus image. One of the historical second fundus images can be selected as a reference second fundus image. The reference second fundus image can be a historical second fundus image adjacent to or at a preset interval of the current second fundus image, where the preset interval can be set according to actual needs and is not limited here. The motion information of the target object can be determined by performing displacement calculation on the current second fundus image and the reference second fundus image. The motion information may include displacement information such as the moving direction and the moving distance, and the scanning position is updated in real time according to the motion information, and the scanning is continued according to the changed scanning position (i.e., the current scanning position).

[0084] It should be noted that when the target area is divided into multiple separated areas and the scanning range of the OCT device cannot cover the target area, the target area can be divided into multiple sub-areas, each of which can be completely covered by the scanning range of the OCT device, and OCT / OCTA scanning can be performed on each sub-area one by one.

[0085] like Figure 3 As shown, the OCT scanning results may include a CSSO fundus image and thumbnails of B-SCAN images obtained after multiple OCT scans. The corresponding B-SCAN image can be enlarged and displayed by selecting the thumbnail of the B-SCAN image. The arrow shows the correspondence between the currently enlarged B-SCAN image and its thumbnail. At the same time, the scanning position corresponding to the OCTA tomographic image will be displayed in the CSSO fundus image, as shown by the horizontal line in the CSSO fundus image.

[0086] After selecting the OCTA scanning protocol, the initial scanning area displayed in the current CSSO fundus image is as follows: Figure 4 As shown; Figure 4 The corresponding scanning protocol may be Angio 6×6 512×512 R4, where Angio indicates the OCTA scanning mode, 6×6mm indicates that the size of the scanning area is 6 mm×6 mm; 512×512 indicates that the number of scanning lines in the horizontal and vertical directions is 512; and R4 indicates the number of scanning repetitions for each scanning line. The determined target area may exceed the range displayed by the current CSSO fundus image, so that the target scanning area adjusted by the OCT device may exceed the range displayed by the current CSSO fundus image. For example, the target scanning area determined after the CSSO fundus image is registered with the first fundus image is as follows: Figure 5 As shown, OCT / OCTA imaging can be performed on the target scanning area. Figure 4 and Figure 5 In each displayed scanning area (box), two scanning lines in different directions are exemplarily shown.

[0087] The scanning method of the OCT device provided in the embodiment of the present invention can obtain a first fundus image of the target object, the first fundus image is obtained by photographing the target object with a fundus camera; obtain a second fundus image of the target object, the second fundus image is obtained by scanning the target object with a fundus imaging device, and the imaging range of the second fundus image is smaller than the imaging range of the first fundus image; determine the target scanning area of ​​the OCT device on the target object according to the area of ​​interest in the first fundus image and the second fundus image; and perform OCT / OCTA scanning on the target scanning area. The imaging range of the first fundus image is larger, the fundus area information is more comprehensive, and the area of ​​interest can be accurately located. Therefore, the area of ​​interest in the first fundus image is combined with the second fundus image with a smaller imaging range, so that the target scanning area of ​​the OCT device can be quickly and accurately located, and the precise positioning and efficient scanning of target areas such as lesion areas can be achieved, thereby improving the accuracy and scanning efficiency of OCT / OCTA scanning.

[0088] Corresponding to the above-mentioned scanning method of the OCT device, an embodiment of the present invention further provides a scanning device of the OCT device, and the scanning device of the OCT device is applied to the OCT device integrated with the fundus imaging device. Figure 6 The structure diagram of a scanning device of an OCT device is shown, and the scanning device of the OCT device includes:

[0089] A first acquisition module 601 is used to acquire a first fundus image of a target object, where the first fundus image is obtained by photographing the target object with a fundus camera;

[0090] A second acquisition module 602 is used to acquire a second fundus image of the target object, where the second fundus image is obtained by scanning the target object with a fundus imaging device, and the imaging range of the second fundus image is smaller than the imaging range of the first fundus image;

[0091] A target scanning area determination module 603 is used to determine a target scanning area of ​​the OCT device on the target object according to the region of interest in the first fundus image and the second fundus image;

[0092] The scanning module 604 is used to perform OCT / OCTA scanning on the target scanning area.

[0093] The scanning device of the OCT device provided in the embodiment of the present invention can obtain a first fundus image of the target object, the first fundus image is obtained by photographing the target object with a fundus camera; obtain a second fundus image of the target object, the second fundus image is obtained by scanning the target object with a fundus imaging device, and the imaging range of the second fundus image is smaller than the imaging range of the first fundus image; determine the target scanning area of ​​the OCT device on the target object according to the area of ​​interest in the first fundus image and the second fundus image; and perform OCT / OCTA scanning on the target scanning area. The imaging range of the first fundus image is larger, the fundus area information is more comprehensive, and the area of ​​interest can be accurately located. Therefore, the area of ​​interest in the first fundus image is combined with the second fundus image with a smaller imaging range, so that the target scanning area of ​​the OCT device can be quickly and accurately located, and the accurate positioning and efficient scanning of target areas such as lesion areas can be achieved, thereby improving the accuracy and scanning efficiency of OCT / OCTA scanning.

[0094] Furthermore, the first acquisition module 601 is specifically configured to acquire, according to the object identifier of the target object, a first fundus image of the target object from a database or a fundus camera used for photographing the target object.

[0095] Furthermore, the above-mentioned target scanning area determination module 603 is specifically used to: obtain the area of ​​interest in the first fundus image; determine the target area corresponding to the area of ​​interest based on the second fundus image; determine the target scanning area of ​​the OCT device on the target object based on the target area; wherein the target scanning area covers the target area.

[0096] Furthermore, the above-mentioned target scanning area determination module 603 is also used to: determine the position information matching the region of interest in the second fundus image, and determine the target region based on the region of interest and the position information; or, perform image registration on the first fundus image and the second fundus image to obtain registration information, and obtain the target region based on the registration information and the region of interest.

[0097] Furthermore, the center of the target scanning area is aligned with the center of the target area.

[0098] Furthermore, the above-mentioned scanning module 604 is specifically used to: acquire the current second fundus image of the target object in real time; determine the motion information of the target object based on the current second fundus image and the reference second fundus image; wherein the reference second fundus image is one of the acquired historical second fundus images; determine the current scanning position based on the motion information and the target scanning area; and perform OCT / OCTA scanning on the current scanning position.

[0099] The scanning device of the OCT device provided in this embodiment has the same implementation principle and technical effects as those of the scanning method embodiment of the aforementioned OCT device. For the sake of brief description, for matters not mentioned in the scanning device embodiment of the OCT device, reference may be made to the corresponding contents in the scanning method embodiment of the aforementioned OCT device.

[0100] like Figure 7 As shown, an OCT device provided by an embodiment of the present invention includes: a processor 701, a memory 702 and a bus, the memory 702 stores a computer program that can be run on the processor 701, when the OCT device is running, the processor 701 and the memory 702 communicate through the bus, and the processor 701 executes the computer program to implement the scanning method of the above-mentioned OCT device.

[0101] Specifically, the memory 702 and the processor 701 can be general-purpose memories and processors, which are not specifically limited here.

[0102] The present invention also provides an ophthalmic imaging device, see Figure 8 The structure diagram of an ophthalmic imaging device shown in FIG. 1 includes:

[0103] A display module 801 is used to display on a screen a second fundus image obtained by scanning a target object with a fundus imaging device;

[0104] The marking module 802 is used to mark the target scanning area for performing OCT / OCTA scanning on the target object on the second fundus image, where the target scanning area is determined based on the area of ​​interest in the first fundus image of the target object. The first fundus image is obtained by photographing the target object with a fundus camera, and the imaging range of the first fundus image is larger than the imaging range of the second fundus image.

[0105] This achieves visualization of the target scanning area in the second fundus image, making it convenient for the user to intuitively view the position of the target scanning area.

[0106] Furthermore, the above-mentioned ophthalmic imaging device also includes:

[0107] A determination module 803 is used to determine a current scanning area of ​​the OCT device to which the fundus imaging device belongs according to a scanning protocol;

[0108] The moving module 804 is used to move the current scanning area until the current scanning area covers the target scanning area.

[0109] This achieves automatic adjustment of the scanning area, thereby enabling OCT / OCTA scanning of the target scanning area.

[0110] The present invention also provides an ophthalmic imaging system. Fig. 9 The structural diagram of an ophthalmic imaging system shown in the figure includes the above-mentioned OCT device 901, and also includes a data transmission platform 902 and a fundus camera 903. The data transmission platform 902 is connected to the OCT device 901 and the fundus camera 903 respectively; the data transmission platform 902 is used to store the first fundus image uploaded by the fundus camera 903, and transmit the corresponding first fundus image to the OCT device 901. The data transmission platform 902 can be connected to at least one OCT device 901 and at least one fundus camera 903; the number of OCT devices 901 and the number of fundus cameras 903 can be set according to actual needs, and are not limited here. Fig. 9 Just an example.

[0111] The fundus camera 903 may be, but is not limited to, an ultra-wide-angle fundus camera. Taking the first fundus image as a fundus color photo as an example, the fundus camera 903 may directly upload the captured fundus color photo to the data transmission platform 902, or may perform region of interest identification on the captured fundus color photo, and then upload the fundus color photo with the region of interest identification result to the data transmission platform 902. The data transmission platform 902 may receive and store the fundus color photo uploaded by the fundus camera 903, or may perform region of interest identification on the received fundus color photo, and then store the fundus color photo with the region of interest identification result. The fundus color photo transmitted by the data transmission platform 902 to the OCT device 901 may be a fundus color photo that needs to be identified with a region of interest, or may be a fundus color photo with a region of interest identification result. The OCT device 901 is integrated with a fundus imaging device, which may be a CSSO, CSLO, SLO or LSO, etc. The OCT device 901 includes a data processing system and a scanning system. The data processing system is used to identify regions of interest (such as lesion regions) and acquire target regions, and the scanning system is used to adjust scanning regions and perform OCT scanning based on the target regions obtained by the data processing system.

[0112] The following takes the fundus camera 903 as an ultra-wide-angle fundus camera and the OCT device 901 integrated with CSSO as an example to describe the working process of the above-mentioned ophthalmic imaging system in detail.

[0113] The OCT scanning process using the above-mentioned ophthalmic imaging system mainly includes the following steps:

[0114] 1. Data collection: Use an ultra-wide-angle fundus camera to take color fundus photos to obtain comprehensive fundus image information.

[0115] 2. Data transmission: The ultra-wide-angle fundus camera transmits fundus color photos to the data processing system on the OCT device through the data transmission platform.

[0116] 3. Identification of lesion areas: In OCT equipment, the data processing system uses deep learning AI algorithms to analyze fundus color photos and automatically identify lesion areas.

[0117] 4. Area matching and positioning: The data processing system matches the identified lesion area with the CSSO fundus image to determine the actual scanning position of the lesion area on the OCT device.

[0118] 5. OCTA scan: After determining the location of the lesion area, the OCT device performs targeted OCTA scans and uses CSSO fundus images for real-time tracking to collect more detailed information data on the lesion area.

[0119] See also Fig.10 The diagram of the connection architecture of an ophthalmic imaging system shown in the figure mainly involves multiple OCT devices, multiple ultra-wide-angle fundus cameras, a communication network, and a data transmission platform. The OCT devices and ultra-wide-angle fundus cameras transmit data with the data transmission platform through the communication network. Each OCT device and each ultra-wide-angle fundus camera has a device identifier and an IP address for communication. The device identifier can be set according to actual needs. For example, the device identifier uses SN. The data transmission platform will maintain the device information mapping that communicates with it (that is, maintain the mapping relationship between the device identifier and the IP address), such as SN1 corresponds to IP1, SN2 corresponds to IP2, etc.

[0120] The above-mentioned ultra-wide-angle fundus camera can have an ultra-large fundus field of view such as 200° horizontally and 170° vertically, and use the red and green dual-color laser imaging principle to realize the color image of the retina. The ultra-large fundus field of view combined with the color image can improve the recognition accuracy of the lesion area.

[0121] The above data transmission platform can be a data processing system independently deployed on a server. The server communicates with the acquisition equipment (ultra-wide-angle fundus camera and OCT equipment) in real time through a communication network, and automatically collects and manages the mapping relationship between the SN number and IP address of the acquisition equipment. The communication network can be, but is not limited to, the network of a hospital or other institution. Fig.11As shown, the data format of the communication may include five parts, wherein the first part Camera SN is the SN number of the ultra-wide-angle fundus camera, the second part OCT SN is the SN number of the OCT device, the third part Patient ID is the object identifier, the fourth part PatientBasics is the basic information of the object, and the fifth part Fundus Color Photo is the captured fundus color photo data.

[0122] The ultra-wide-angle fundus camera transmits the collected fundus color photos to the data transmission platform in real time in the specified data format. After receiving the data containing the fundus color photos, the data transmission platform parses the data to obtain the object identification, and stores the object identification and its fundus color photo data. After the data processing system on the OCT device obtains the fundus color photos, it uses the deep learning AI algorithm to match them and automatically identify the lesion area.

[0123] The deep learning AI algorithm will first receive a training data set (i.e., a data set of fundus color photos with lesion area labels), perform feature marking on the fundus color photos in the training data set, and use a convolutional neural network (CNNs) model architecture to continuously improve the accuracy of lesion area identification through continuous model training.

[0124] See also Fig.12 A flow chart of using a convolutional neural network model to identify lesion areas is shown, first preparing a fundus color photo dataset of different lesion categories, each fundus color photo in the fundus color photo dataset is configured with a corresponding lesion area label; dividing the fundus color photo dataset into a training set, a validation set and a test set; preprocessing the training set data, such as one or more of image scaling, normalization and data enhancement; inputting the preprocessed data into a CNNs model for training, such as tuning the model parameters of the CNNs model based on the output lesion area prediction results and the corresponding lesion area labels; using the validation set and the test set to evaluate the performance of the trained CNNs model, and observing indicators such as accuracy and confusion matrix; according to the corresponding results of the validation set and the test set, tuning the model architecture of the CNNs model to improve the recognition accuracy of the model; deploying the optimized CNNs model to the data processing system of the OCT device; the data processing system of the OCT device identifies the lesion area of ​​the received fundus color photo based on the optimized CNNs model, and matches the identified lesion area to the CSSO fundus map to perform OCTA precise scanning.

[0125] Before the OCT device starts to collect the CSSO fundus image, it requests the target object's fundus color photo from the data transmission platform. After receiving the fundus color photo, the data processing system identifies the lesion area, compares the CSSO fundus image with the identified lesion area, automatically matches the scanning area on the CSSO fundus image, and combines the CSSO fundus image for real-time tracking to accurately scan the lesion area.

[0126] See also Fig.13 The overall processing flow diagram of an ophthalmic imaging system shown in FIG. 1 includes:

[0127] The ultra-wide-angle fundus camera takes a fundus color photo and determines whether the fundus color photo is taken successfully. If not (i.e., the fundus color photo is not taken successfully), the step of taking the fundus color photo is re-executed; if yes (i.e., the fundus color photo is taken successfully), the fundus color photo data is packaged and sent to the data transmission platform.

[0128] After receiving the fundus color photo data, the data transmission platform determines whether the data format is legal (such as whether information is missing. If information is missing, the data format is illegal). If not (i.e., the data format is illegal), the process ends (optionally, before the process ends, a prompt message indicating that the data format is illegal can be returned to the ultra-wide-angle fundus camera); if yes (i.e., the data format is legal), the fundus color photo data, such as the object identifier, name, ID number, fundus color photo and other data, is saved and parsed.

[0129] After receiving the request sent by the OCT device, the data transmission platform determines whether the request is legal (such as whether it contains necessary information such as the object identifier; if any information is missing, it is illegal). If not (i.e., the request is illegal), the process ends (optionally, before the process ends, a prompt message indicating that the request is illegal may be returned to the OCT device); if yes (i.e., the request is legal), the corresponding fundus color photograph is queried according to the object identifier in the request; and it is determined whether the fundus color photograph is found; if not (i.e., the fundus color photograph is not found), the process ends (optionally, before the process ends, a prompt message indicating that the fundus color photograph is not found may be returned to the OCT device); if yes (i.e., the fundus color photograph is found), the encapsulated fundus color photograph data is returned to the OCT device.

[0130] The data processing system of the OCT device sends a request to obtain fundus color photo data to the data transmission platform, and receives the fundus color photo data returned by the data transmission platform; determines whether the data format is legal (such as determining whether information is missing. If information is missing, the data format is illegal); if not (i.e., the data format is illegal), the process ends (optionally, before the process ends, a prompt message indicating that the data format is illegal can also be displayed to the user); if yes (i.e., the data format is legal), the fundus color photo data is parsed to obtain the fundus color photo; the fundus color photo is input into the deep learning AI algorithm to automatically identify the lesion area on the fundus color photo; the CSSO fundus image is compared with the fundus color photo to automatically identify the corresponding lesion area on the CSSO fundus image; OCTA precision scanning is performed on the lesion area on the CSSO fundus image to obtain detailed data of the lesion area.

[0131] The beneficial effects of the embodiments of the present invention are as follows:

[0132] 1. Combined with the color fundus photos taken by the ultra-wide-angle fundus camera, the accuracy and efficiency of OCT equipment scanning are greatly improved.

[0133] 2. Effectively utilize the more comprehensive fundus area information provided by fundus color photography, while solving the problem that fundus color photography cannot accurately scan the lesion area.

[0134] 3. Using the data transmission platform, the OCT device can quickly obtain color fundus photographs.

[0135] 4. On the OCT device, the AI ​​algorithm that uses deep learning in the data processing system analyzes the color fundus photos and automatically identifies the diseased area.

[0136] 5. Match the identified lesion area with the CSSO fundus image, conduct targeted OCTA scanning of the lesion area, and perform real-time tracking in combination with the CSSO fundus image to collect detailed data of the lesion area.

[0137] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the scanning method of the OCT device in the above method embodiment is executed. The computer-readable storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a RAM, a magnetic disk, or an optical disk, etc., various media that can store program codes.

[0138] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0139] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limiting, and thus other examples of the exemplary embodiments may have different values.

[0140] The flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and a part of a module, a program segment or a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0141] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or modules can be electrical, mechanical or other forms.

[0142] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0143] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A scanning method of an OCT device, characterized in that: The method is applied to an OCT device integrated with a fundus imaging device, wherein the fundus imaging device is used to locate a scanning area for the OCT device; the method comprises: Acquire a first fundus image of a target object, where the first fundus image is obtained by photographing the target object with a fundus camera; Acquire a second fundus image of the target object, where the second fundus image is obtained by scanning the target object by the fundus imaging device, and an imaging range of the second fundus image is smaller than an imaging range of the first fundus image; Determining a target scanning area of ​​the OCT device on the target object according to the region of interest in the first fundus image and the second fundus image; Performing OCT / OCTA scanning on the target scanning area; The step of determining a target scanning area of ​​the OCT device on the target object according to the region of interest in the first fundus image and the second fundus image includes: Acquire a region of interest in the first fundus image; determining a target area corresponding to the region of interest according to the second fundus image; According to the target area, a target scanning area of ​​the OCT device on the target object is determined; wherein the target scanning area covers the target area.

2. The method according to claim 1, characterized in that The step of acquiring a first fundus image of the target object includes: According to the object identification of the target object, a first fundus image of the target object is acquired from a database or a fundus camera used for photographing the target object.

3. The method according to claim 1, characterized in that The determining, according to the second fundus image, a target area corresponding to the region of interest includes: determining position information matching the region of interest in the second fundus image, and determining the target region according to the region of interest and the position information; or, Performing image registration on the first fundus image and the second fundus image to obtain registration information, and obtaining the target region according to the registration information and the region of interest.

4. The method according to claim 1, characterized in that: The center of the target scanning area is aligned with the center of the target area.

5. The method according to claim 1, characterized in that The performing OCT / OCTA scanning on the target scanning area includes: Acquire a current second fundus image of the target object in real time; Determine the motion information of the target object according to the current second fundus image and the reference second fundus image; wherein the reference second fundus image is one of the acquired historical second fundus images; Determining a current scanning position according to the motion information and the target scanning area; Perform an OCT / OCTA scan on the current scanning position.

6. An OCT device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.

7. An ophthalmic imaging device, characterized in that: include: A display module, used for displaying on a screen a second fundus image obtained by scanning the target object by a fundus imaging device, wherein the fundus imaging device is used for locating a scanning area for the OCT device; a marking module, configured to mark a target scanning area for performing OCT / OCTA scanning on the target object on the second fundus image, wherein the target scanning area is determined according to a region of interest in a first fundus image of the target object, wherein the first fundus image is obtained by photographing the target object with a fundus camera, and an imaging range of the first fundus image is greater than an imaging range of the second fundus image; Wherein, the target scanning area is determined in the following manner: acquiring the region of interest in the first fundus image; determining the target area corresponding to the region of interest based on the second fundus image; determining the target scanning area of ​​the OCT device on the target object based on the target area; wherein the target scanning area covers the target area.

8. The ophthalmic imaging device according to claim 7, characterized in that: The ophthalmic imaging device further comprises: A determination module, used to determine a current scanning area of ​​the OCT device to which the fundus imaging device belongs according to a scanning protocol; The moving module is used to move the current scanning area until the current scanning area covers the target scanning area.

9. An ophthalmic imaging system, characterized in that: The OCT device according to claim 6 further comprises a data transmission platform and a fundus camera, wherein the data transmission platform is connected to the OCT device and the fundus camera respectively; The data transmission platform is used to store the first fundus image uploaded by the fundus camera and transmit the corresponding first fundus image to the OCT device.

Images