Fundus imaging method and system based on meta-imaging

Abstract

This application relates to the field of fundus imaging technology, and particularly to a fundus imaging method and system based on meta-imaging. Among them, the method includes: performing light field imaging on the fundus through light field imaging in cooperation with synchronous illumination; extracting pixels at the same angle from the light field image and fusing them to obtain a single-view image, and finally obtaining multiple multi-view images with improved spatial sampling rate; calculating the offset matrix of multiple view images according to a pre-designed calculation strategy, and using the offset matrix to perform digital offset correction on the images of multiple views; performing image stitching and fusion on the corrected images of multiple views to obtain the final imaging result of the fundus to be imaged. Thus, the embodiments of this application can complete focusing on different regions of the fundus through a single shot, and at the same time can maintain large-field-of-view and high-resolution imaging, break the constraint between spatial resolution and angular resolution, and can achieve high spatial resolution while obtaining multi-angle sampling.

Description

Technical Field

[0001] This application relates to the field of fundus imaging technology, and particularly relates to a fundus imaging method and system based on meta-imaging. Background Art

[0002] The retina is a developmental extension of the central nervous system and has a similar structure and function to the central nervous system. The retina can observe pathological changes from the brain to a certain extent and has potential diagnostic value for neurodegenerative diseases such as Alzheimer's disease and Parkinson's disease. For example, some studies have modeled AD in mice and observed the retinal lesions in mice at the pre-onset, mid-onset, and end-stage of AD, to see if cell inflammation of glial cells and apoptosis of retinal ganglion cells can be detected before the onset of AD symptoms and are correlated with the progression of AD. In order to perform high-precision subcellular-level observation in this aspect of research, it is necessary to sacrifice the mice and image the retinal and brain tissue sections.

[0003] Current methods mainly remain at the animal research stage. To perform subcellular-level fundus imaging, it is necessary to sacrifice the mice for tissue sectioning. To apply fundus imaging to the early warning and clinical diagnosis and treatment of AD, non-invasive fundus imaging is required. However, in clinical practice, fundus cameras cannot perform subcellular-scale fundus imaging. In addition, due to the influence of eye curvature, etc., the current imaging systems have large differences in aberration, and it is necessary to separately focus on different regions to collect images, resulting in multiple shots at different depths for stitching. Long-term multiple shots, on the one hand, may cause damage to the eyes, and on the other hand, the movement of the subjects is likely to cause blurring. Therefore, there is an urgent need for fast, large-scale, and high-resolution fundus imaging. For traditional light field imaging, increasing the angular resolution will result in loss of spatial resolution. Summary of the Invention

[0004] This application provides a fundus imaging method, device, system, electronic device, and computer-readable storage medium based on meta-imaging, which can obtain spatial and angular information through light field imaging, focus on different regions of the fundus, and at the same time be able to maintain large-field and high-resolution imaging, break the constraint between spatial resolution and angular resolution, and be able to achieve high spatial resolution while obtaining multi-angle sampling.

[0005] The first aspect embodiment of this application provides a fundus imaging method based on meta-imaging, including the following steps: multiple light field images at multiple relative positions are collected during imaging to obtain a multi-view image with improved spatial sampling rate; an offset matrix of the multiple view images is calculated according to a pre-designed calculation strategy, and the images of the multiple views are digitally offset-corrected using the offset matrix; the corrected images of the multiple views are stitched and fused to obtain the final imaging result of the fundus to be imaged.

[0006] In the embodiments of the present application, it further includes: in the imaging stage / image acquisition stage, performing light field imaging on the fundus by combining light field imaging with synchronous illumination.

[0007] In the embodiments of the present application, during imaging, multiple light field images at multiple relative positions are collected, and a multi-view image with an increased spatial sampling rate is obtained through pixel extraction and stitching, including: changing the propagation direction of the imaging light of the fundus to be imaged according to a preset control strategy to obtain imaging light in multiple propagation directions; collecting the imaging light in the multiple propagation directions to obtain the multiple light field images at multiple relative positions. Extracting and fusing the pixels at the same angle of each microlens to obtain an image of a single view; performing the same operation for different angles to obtain the multi-view image.

[0008] In the embodiments of the present application, stitching and fusing the images of multiple views after correction to obtain the final imaging result of the fundus to be imaged, including: obtaining the relative position relationship of the corrected images of multiple views; performing stitching and fusing on the corrected images of multiple views according to the relative position relationship to obtain the final imaging result of the fundus to be imaged.

[0009] In the embodiments of the present application, after obtaining the final imaging result of the fundus to be imaged, it further includes: obtaining the area to be focused of the fundus to be imaged; matching the best focusing angle of the final imaging result according to the area to be focused, and using the best focusing angle to focus on the target area of the final imaging result.

[0010] In the embodiments of the present application, after obtaining the final imaging result of the fundus to be imaged, it further includes: determining whether the resolution of the final imaging result reaches the subcellular level resolution; if the resolution of the final imaging result reaches the subcellular level resolution, outputting the final imaging result.

[0011] An embodiment of the second aspect of the present application provides a fundus imaging device based on meta-imaging, including: an acquisition module for acquiring multiple light field images at multiple relative positions during imaging, and obtaining a multi-view image with an increased spatial sampling rate through pixel extraction and stitching; a correction module for calculating the offset matrix of the images of multiple views according to a preset calculation strategy, and digitally offset-correcting the images of multiple views by using the offset matrix; a fusion module for stitching and fusing the corrected images of multiple views to obtain the final imaging result of the fundus to be imaged.

[0012] In an embodiment of the present application, it includes: a control module that changes the propagation direction of the imaging light rays of the fundus to be imaged according to a preset control strategy to obtain imaging light rays in multiple propagation directions; collects the imaging light rays in the multiple propagation directions to obtain light field images at the multiple relative positions, and then extracts and stitches pixels to obtain a multi-view image with an increased spatial sampling rate.

[0013] In an embodiment of the present application, the fusion module is used to: obtain the relative position relationship of the corrected multi-view images; perform stitching and fusion on the corrected images of the multiple views according to the relative position relationship to obtain the final imaging result of the fundus to be imaged.

[0014] In an embodiment of the present application, it further includes: a focusing module that, after obtaining the final imaging result of the fundus to be imaged, acquires the area to be focused on the fundus to be imaged; matches the best focusing angle of the final imaging result according to the area to be focused, and uses the best focusing angle to focus on the target area of the final imaging result.

[0015] In an embodiment of the present application, it further includes: a judgment module that, after obtaining the final imaging result of the fundus to be imaged, judges whether the resolution of the final imaging result reaches the subcellular level resolution; if the resolution of the final imaging result reaches the subcellular level resolution, outputs the final imaging result.

[0016] An embodiment of the third aspect of the present application provides a meta-imaging system for the fundus, including: an imaging component for acquiring light field images at multiple relative positions obtained by multiple acquisitions during imaging; the fundus imaging device based on meta-imaging as described in the above embodiment.

[0017] In an embodiment of the present application, the imaging component includes: a galvanometer mirror, a microlens array, and a camera sensor. Among them, the meta-imaging device is used to, during imaging, control the galvanometer mirror to change the propagation direction of the imaging light rays of the fundus to be imaged according to a preset strategy to obtain imaging light rays in multiple propagation directions, and collect the imaging light rays in the multiple propagation directions through the microlens array and the camera sensor to obtain the light field images at the multiple relative positions.

[0018] An embodiment of the fourth aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the program to implement the fundus imaging method based on meta-imaging as described in the above embodiment.

[0019] An embodiment of the fifth aspect of the present application provides a computer-readable storage medium, on which a computer program is stored, and the program is executed by a processor to be used to implement the fundus imaging method based on meta-imaging as described in the above embodiment.

[0020] Accordingly, the present application has at least the following beneficial effects:

[0021] It can achieve post - shooting focusing. The light - field imaging retains angular information and can complete focusing on different regions of the fundus through a single shot, effectively solving the problems in related technologies where due to the curvature of the fundus, imaging requires separate focusing and image acquisition for different regions, resulting in a long shooting time and inability to focus on some regions. At the same time, it can maintain large - field - of - view and high - resolution imaging. For traditional light - field imaging, increasing angular resolution will result in loss of spatial resolution. By controlling scanning to change the light propagation direction to achieve moving sampling in the dense space of the microlens array, the constraint between spatial resolution and angular resolution is broken, enabling multi - angle sampling while achieving high spatial resolution, and then high - resolution imaging is achieved through the fusion of multiple frames.

[0022] Additional aspects and advantages of the present application will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above - mentioned and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description of the embodiments in conjunction with the drawings, where:

[0024] Figure 1 FIG. is a system block diagram of a meta - imaging system for fundus according to an embodiment of the present application;

[0025] Figure 2 FIG. is a schematic flowchart of a fundus imaging method based on meta - imaging according to an embodiment of the present application;

[0026] Figure 3 FIG. is a meta - imaging acquisition example system for fundus according to an embodiment of the present application;

[0027] Figure 4 FIG. is a schematic diagram of a fundus imaging device based on meta - imaging according to an embodiment of the present application;

[0028] Figure 5 FIG. is a block schematic diagram of a meta - imaging system according to an embodiment of the present application;

[0029] Figure 6 FIG. is a schematic structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] Embodiments of the present application will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present application, and should not be construed as a limitation of the present application.

[0031] Next, a fundus imaging method, apparatus, system, electronic device, and computer-readable storage medium based on meta-imaging according to embodiments of the present application will be described with reference to the accompanying drawings. The hardware involved in the following embodiments, such as Figure 1 shown, mainly includes a lens, a microlens array, a galvanometer mirror, a camera, a flash, and a board, etc. Among them, the lens can be an objective lens, and the camera sensor and the flash cooperate through the board.

[0032] Specifically, Figure 2 is a schematic flowchart of a fundus imaging method based on meta-imaging provided by an embodiment of the present application.

[0033] As Figure 2 shown, the fundus imaging method based on meta-imaging includes the following steps:

[0034] In step S101, multiple light field images at multiple relative positions are acquired during imaging, and then a multi-view image with an improved spatial sampling rate is obtained.

[0035] It can be understood that the embodiments of the present application can adopt meta-imaging technology. Through multi-angle dense spatial sampling, on the one hand, multi-view information is retained to achieve three-dimensional imaging, and on the other hand, the mutual constraint between spatial resolution and angular resolution can be broken. Losing a certain time resolution while ensuring high angular resolution and spatial resolution, thereby realizing large-field-of-view and high-resolution three-dimensional fundus imaging.

[0036] In the embodiments of the present application, acquiring multiple light field images at multiple relative positions during imaging and then obtaining a multi-view image with an improved spatial sampling rate includes: changing the relative position between the fundus to be imaged and the meta-imaging system according to a preset control strategy to obtain a light field image with a high spatial sampling rate; extracting and fusing pixels at the same angle of each microlens to obtain an image of a single view; performing the same operation for different angles to obtain the multi-view image.

[0037] Among them, the preset control strategy can be selected or set according to the actual situation, and no specific limitation is made thereto.

[0038] It can be understood that in the embodiments of the present application, by controlling scanning to change the light propagation direction, mobile sampling in the dense space of the microlens array is achieved, breaking the constraint between spatial resolution and angular resolution. It is possible to obtain multi-angle sampling while achieving high spatial resolution, extract pixels at the same angle from the light field image and fuse them to obtain a single-view image, and finally obtain multiple multi-view images with improved spatial sampling rate.

[0039] Taking Figure 3 the scenario shown as an example, in the embodiments of the present application, the cooperation of a flash, a camera sensor, and a galvanometer can be realized through a board card to achieve scanning imaging. Specifically: the fundus image is transmitted to the galvanometer through an objective lens, reflected and then enters the microlens array, and finally enters the camera sensor and is recorded. By changing the light propagation direction through the galvanometer, mobile sampling in the dense space of the microlens array is achieved, breaking the constraint between spatial resolution and angular resolution, and it is possible to obtain multi-angle sampling while achieving high spatial resolution.

[0040] It should be noted that since there is no light in the fundus, in the embodiments of the present application, light field imaging of the fundus can be performed by cooperating with synchronous illumination through light field imaging during the imaging stage / image acquisition stage.

[0041] In step S102, an offset matrix of multiple view images is calculated according to a pre-designed calculation strategy, and the multiple view images are digitally offset-corrected using the offset matrix.

[0042] It can be understood that in the embodiments of the present application, an offset matrix can be obtained by offset estimation of multiple views through the correlation method or the optical flow method based on the images of multiple views. The aberration matrix is obtained by two-dimensional integration of the offset matrix, and the accurate offset matrix is obtained by removing the Zernike defocus term from the phase difference matrix and then performing two-dimensional differentiation, realizing digital offset correction of multiple view images.

[0043] In step S103, the corrected multiple view images are stitched and fused to obtain the final imaging result of the fundus to be imaged.

[0044] It can be understood that in the embodiments of the present application, high-resolution imaging can be achieved through the fusion between multiple frames. Specifically, through the adaptive optical method, the acquired image queue can be fused to achieve high-resolution imaging and keep the imaging field of view large.

[0045] In the embodiments of the present application, stitching and fusing the corrected multiple view images to obtain the final imaging result of the fundus to be imaged includes: obtaining the relative position relationship of the corrected multiple view images; stitching and fusing the corrected multiple view images according to the relative position relationship to obtain the final imaging result of the fundus to be imaged.

[0046] It can be understood that the embodiments of the present application can directly splice and fuse the offset image according to the relative position relationship of the offset to obtain the final imaging result of the fundus to be imaged.

[0047] In the embodiments of the present application, after obtaining the final imaging result of the fundus to be imaged, it further includes: acquiring the area to be focused of the fundus to be imaged; matching the best focusing angle of the final imaging result according to the area to be focused, and focusing on the target area of the final imaging result by using the best focusing angle.

[0048] It can be understood that the embodiments of the present application adopt the meta-imaging technology, which can image first and then focus, solve the problem that different fundus positions cannot be photographed simultaneously, store the multi-view image queue through multi-view spatial sampling, retain the angle information, and thus can be reconstructed into a three-dimensional image to achieve shooting first and then focusing. Thus, on the one hand, the problem of separately collecting different fundus areas is avoided, and on the other hand, the motion artifacts caused by long-time shooting with traditional multi-area separate focusing are avoided.

[0049] In the embodiments of the present application, after obtaining the final imaging result of the fundus to be imaged, it further includes: determining whether the resolution of the final imaging result reaches the subcellular level resolution; if the resolution of the final imaging result reaches the subcellular level resolution, outputting the final imaging result.

[0050] It can be understood that the resolution of the final imaging result of the embodiments of the present application can reach the subcellular level to achieve high-resolution imaging.

[0051] According to the fundus imaging method based on meta-imaging proposed by the embodiments of the present application, it can achieve shooting first and then focusing. The light field imaging retains the angle information and can complete the focusing of different fundus areas through a single shot, effectively solving the problems in the related art that due to the curvature of the fundus, imaging requires separate focusing and image acquisition for different areas, resulting in a long shooting time and some areas being unable to be focused; and at the same time, it can maintain large-field and high-resolution imaging. For traditional light field imaging, the problem that increasing the angular resolution will result in loss of spatial resolution is solved by scanning control to change the light propagation direction to achieve the moving sampling of the dense space of the microlens array, breaking the constraint between the spatial resolution and the angular resolution, being able to obtain multi-angle sampling while achieving high spatial resolution, and then achieving high-resolution imaging through the fusion between multiple frames.

[0052] Next, a fundus imaging device based on meta-imaging proposed according to the embodiments of the present application will be described with reference to the accompanying drawings.

[0053] Figure 4 It is a block diagram of the fundus imaging device based on meta-imaging according to the embodiments of the present application.

[0054] As Figure 4As shown in the figure, the fundus imaging device 100 based on meta-imaging includes: an acquisition module 110, a correction module 120, and a fusion module 130.

[0055] Among them, the acquisition module 110 is used to acquire light field images at multiple relative positions during imaging, and obtain a multi-view image with an improved spatial sampling rate through pixel extraction and stitching; the correction module 120 is used to calculate the offset matrix of the images at multiple views according to a pre-designed calculation strategy, and perform digital offset correction on the images at multiple views using the offset matrix; the fusion module 130 is used to perform stitching and fusion of the corrected images at multiple views to obtain the final imaging result of the fundus to be imaged.

[0056] In the embodiment of the present application, the fundus imaging device 100 based on meta-imaging further includes: a control module, which is used to change the propagation direction of the imaging light of the fundus to be imaged according to a preset control strategy during imaging, obtain imaging light at multiple propagation directions, collect imaging light at multiple propagation directions, acquire light field images at multiple relative positions during imaging, and then obtain a multi-view image with an improved spatial sampling rate through pixel extraction and stitching.

[0057] In the embodiment of the present application, the fusion module 130 is used to: obtain the relative position relationship of the corrected images at multiple views; perform stitching and fusion on the corrected images at multiple views according to the relative position relationship to obtain the final imaging result of the fundus to be imaged.

[0058] In the embodiment of the present application, the fundus imaging device 100 based on meta-imaging further includes: a focusing module, which is used to, after obtaining the final imaging result of the fundus to be imaged, acquire the area to be focused of the fundus to be imaged; match the best focusing angle of the final imaging result according to the area to be focused, and focus on the target area of the final imaging result using the best focusing angle.

[0059] In the embodiment of the present application, the fundus imaging device 100 based on meta-imaging further includes: a judgment module, which is used to, after obtaining the final imaging result of the fundus to be imaged, judge whether the resolution of the final imaging result reaches the subcellular level resolution; if the resolution of the final imaging result reaches the subcellular level resolution, output the final imaging result.

[0060] It should be noted that the foregoing explanation of the embodiment of the fundus imaging method based on meta-imaging also applies to the fundus imaging device based on meta-imaging in this embodiment, and will not be elaborated here.

[0061] The fundus imaging device based on meta-imaging proposed according to the embodiments of the present application can achieve post-shooting focusing. The light field imaging retains angular information and can complete focusing on different regions of the fundus through a single shot, effectively solving the problems in the related art that due to the curvature of the fundus, imaging requires separate focusing and image acquisition for different regions, resulting in a long shooting time and inability to focus on some regions; and at the same time, it can maintain large-field-of-view and high-resolution imaging. For traditional light field imaging, the problem that increasing angular resolution will result in loss of spatial resolution is solved by scanning control to change the propagation direction of light rays to achieve moving sampling in the dense space of the microlens array, breaking the constraint between spatial resolution and angular resolution, enabling multi-angle sampling while achieving high spatial resolution, and then achieving high-resolution imaging through the fusion between multiple frames.

[0062] Next, a meta-imaging system proposed according to the embodiments of the present application will be described with reference to the accompanying drawings.

[0063] Figure 5 It is a block diagram of a meta-imaging system for the fundus according to the embodiments of the present application.

[0064] As Figure 5 shown, the meta-imaging system 10 for the fundus includes: a meta-imaging device 100 and an imaging component 200.

[0065] Among them, the imaging component 200 is used to obtain light field images at multiple relative positions collected multiple times during imaging, and obtain a multi-view image with an improved spatial sampling rate through pixel extraction and stitching; the fundus imaging device 100 based on meta-imaging is used to obtain light field images at multiple relative positions collected multiple times during imaging; calculate the offset matrix of the images at multiple perspectives according to a pre-designed calculation strategy, and use the offset matrix to perform digital offset correction on the images at multiple perspectives; perform image stitching and fusion on the corrected images at multiple perspectives to obtain the final imaging result of the fundus to be imaged.

[0066] It should be noted that the explanation of the fundus imaging device 100 based on meta-imaging can refer to the above embodiments. To avoid redundancy, it will not be elaborated here.

[0067] In the embodiments of the present application, as Figure 1 shown, the imaging component 200 includes: a galvanometer mirror, a microlens array, and a camera sensor. Among them, the meta-imaging device is used to control the galvanometer mirror to change the propagation direction of the imaging light rays of the fundus to be imaged according to a preset strategy during imaging, obtain imaging light rays in multiple propagation directions, collect the imaging light rays in multiple propagation directions through the microlens array and the camera sensor, obtain light field images at multiple relative positions, and obtain a multi-view image with an improved spatial sampling rate through pixel extraction and stitching.

[0068] It should be noted that the foregoing explanations of the fundus imaging method based on meta-imaging and the embodiments of the fundus imaging device based on meta-imaging also apply to the fundus meta-imaging system of this embodiment, and will not be elaborated here.

[0069] The fundus meta-imaging system provided by the embodiments of the present application can achieve post-shot focusing. The light field imaging retains angular information and can complete focusing on different regions of the fundus through a single shot, effectively solving the problems in the related art that due to the curvature of the fundus, imaging requires separate focusing and image acquisition for different regions, resulting in a long shooting time and some regions being unable to be focused; and at the same time, it can maintain large-field and high-resolution imaging. For traditional light field imaging, increasing the angular resolution will result in a loss of spatial resolution. By scanning control to change the light propagation direction to achieve moving sampling in the dense space of the microlens array, the constraint between spatial resolution and angular resolution is broken, and high spatial resolution can be achieved while obtaining multi-angle sampling, and then high-resolution imaging is achieved through the fusion between multiple frames.

[0070] Figure 6 The following is a schematic structural diagram of an electronic device provided by an embodiment of the present application. The electronic device may include:

[0071] A memory 601, a processor 602, and a computer program stored on the memory 601 and executable on the processor 602.

[0072] When the processor 602 executes the program, it implements the fundus imaging method based on meta-imaging provided in the above embodiments.

[0073] Further, the electronic device further includes:

[0074] A communication interface 603 for communication between the memory 601 and the processor 602.

[0075] The memory 601 is used to store a computer program executable on the processor 602.

[0076] The memory 601 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0077] If the memory 601, the processor 602, and the communication interface 603 are implemented independently, the communication interface 603, the memory 601, and the processor 602 can be interconnected through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 only a thick line is used to represent it in Figure 6 , but it does not mean that there is only one bus or one type of bus.

[0078] Optionally, in a specific implementation, if the memory 601, the processor 602, and the communication interface 603 are integrated on a single chip, the memory 601, the processor 602, and the communication interface 603 can communicate with each other through an internal interface.

[0079] The processor 602 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.

[0080] The embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the fundus imaging method based on meta-imaging as described in the above embodiments.

[0081] In the description of this specification, the descriptions with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0082] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0083] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment, or portion of code including one or more executable instructions for implementing a customized logical function or process. The scope of the preferred embodiments of this application includes additional implementations, where functions may be performed in a substantially simultaneous manner or in a reverse order according to the functions involved, rather than in the order shown or discussed, which should be understood by those skilled in the art to which the embodiments of this application pertain.

[0084] It should be understood that various parts of this application can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one of the following techniques known in the art or a combination thereof can be used: discrete logic circuits with logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits with suitable combinational logic gate circuits, programmable gate arrays, field-programmable gate arrays, etc.

[0085] Those of ordinary skill in the art of this technology can understand that all or part of the steps carried out in implementing the above method embodiments can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

Claims

1. A fundus imaging method based on meta-imaging, characterized in that, it includes the following steps: During imaging, multiple light field images at multiple relative positions are acquired, and then a multi-view image with an increased spatial sampling rate is obtained; According to a pre-designed calculation strategy, an offset matrix of the images of the multiple views is calculated, and the images of the multiple views are digitally offset corrected by using the offset matrix; The corrected images of the multiple views are stitched and fused to obtain the final imaging result of the fundus to be imaged; During imaging, multiple light field images at multiple relative positions are acquired, and then a multi-view image with an increased spatial sampling rate is obtained, including: Changing the relative position between the fundus to be imaged and the meta-imaging system according to a preset control strategy to obtain a light field image with a high spatial sampling rate; Extracting and fusing the pixels at the same angle of each microlens to obtain an image of a single view; performing the same operation for different angles to obtain the multi-view image.

2. The method according to claim 1, characterized in that, it further includes: During the imaging stage / image acquisition stage, light field imaging of the fundus is performed by cooperating with synchronous illumination in light field imaging.

3. The method according to claim 1, characterized in that, The stitching and fusing of the images of the corrected multiple views to obtain the final imaging result of the fundus to be imaged includes: Obtaining the relative position relationship of the corrected multi-view images; Performing stitching and fusing on the corrected multi-view images according to the relative position relationship to obtain the final imaging result of the fundus to be imaged.

4. The method according to claim 1, characterized in that, After obtaining the final imaging result of the fundus to be imaged, it further includes: Obtaining the to-be-focused area of the fundus to be imaged; Matching the best focusing angle of the final imaging result according to the to-be-focused area, and focusing on the target area of the final imaging result by using the best focusing angle.

5. A fundus imaging device based on meta-imaging, characterized in that, it includes: An acquisition module for the meta-imaging system to obtain a multi-view image with an increased spatial sampling rate; A correction module for calculating an offset matrix of the images of the multiple views according to a pre-designed calculation strategy, and digitally offset correcting the images of the multiple views by using the offset matrix; A fusion module for performing stitching and fusing on the corrected images of the multiple views to obtain the final imaging result of the fundus to be imaged; A control module for changing the propagation direction of the imaging light of the fundus to be imaged according to a preset control strategy during imaging to obtain imaging light of multiple propagation directions; collecting the imaging light of the multiple propagation directions to obtain the images of the multiple views.

6. The device according to claim 5, characterized in that, The fusion module is used for: Obtaining the relative position relationship of the corrected multi-view images; Performing stitching and fusing on the corrected multi-view images according to the relative position relationship to obtain the final imaging result of the fundus to be imaged.

7. The device according to claim 5, characterized in that, it further includes: A focusing module, configured to obtain a to-be-focused area of the fundus to be imaged after obtaining the final imaging result of the fundus to be imaged; Match the best focusing angle of the final imaging result according to the to-be-focused area, and focus on the target area of the final imaging result by using the best focusing angle.

8. The apparatus according to any one of claims 5-7, wherein, further comprising: A judgment module, configured to judge whether the resolution of the final imaging result of the fundus to be imaged reaches the subcellular-level resolution after obtaining the final imaging result of the fundus to be imaged; If the resolution of the final imaging result reaches the subcellular-level resolution, output the final imaging result.

9. A meta-imaging system for the fundus, wherein, comprising: An imaging component, configured to collect the fundus to be imaged multiple times during imaging to obtain multiple light field images at relative positions; The fundus imaging apparatus based on meta-imaging according to any one of claims 5-8.

10. The system according to claim 9, wherein, The imaging component includes: a galvanometer mirror, a microlens array, and a camera sensor. Among them, the fundus imaging apparatus is configured to control the galvanometer mirror to change the propagation direction of the imaging light of the fundus to be imaged according to a preset strategy during imaging, obtain imaging lights in multiple propagation directions, collect the imaging lights in multiple propagation directions through the microlens array and the camera sensor, obtain multiple light field images at relative positions, and further obtain a multi-view image with an improved spatial sampling rate.

11. An electronic device, wherein, comprising: A memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the program to implement the meta-imaging-based fundus imaging method according to any one of claims 1-4.

12. A computer-readable storage medium, on which a computer program is stored, wherein, the program is executed by a processor to be used to implement the meta-imaging-based fundus imaging method according to any one of claims 1-4.

Images