A deep learning-based OCT optical distortion correction method, medium and device

Abstract

The present application relates to the technical field of optical distortion correction, in particular to an OCT optical distortion correction method based on deep learning, medium and equipment. The method is: input the distorted calibration board image into the target convolutional neural network to obtain the corrected image, generate the corrected pixel difference according to the pixel coordinate difference of the two calibration points, and then convert the scanning position correction difference through coordinate conversion, and then convert it into X, Y direction driving compensation voltage, so as to correct the optical vibration mirror driving voltage in the OCT imaging unit. The technical scheme introduces a multi-layer convolutional neural network, including an adaptive convolutional layer, a pooling layer and a fully connected layer, optimizes the parameters by using supervised learning, automatically learns the distortion characteristics, establishes a nonlinear mapping relationship to generate the corrected pixel difference, and improves the correction accuracy and efficiency. The method realizes automatic and accurate correction, solves the problems of insufficient accuracy of the prior art, avoids manual intervention, simplifies the process and reduces the calculation complexity.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of optical distortion correction, in particular to an OCT optical distortion correction method based on deep learning, a medium and equipment. BACKGROUND

[0002] As a high-precision optical imaging method, optical coherence tomography (OCT) technology is widely used in medical imaging, industrial detection, material science and other fields, and plays an important role in precise geometric measurement and internal defect detection. OCT technology can realize high-resolution depth imaging and non-destructively obtain internal structure information of an object by utilizing the coherence of light. However, OCT is often affected by optical distortion in practical application, resulting in distortion of the imaging result, which seriously affects the accuracy and reliability of the measurement. Therefore, it is particularly important to use effective distortion correction methods for distortion problems in OCT imaging.

[0003] At present, for the problem of optical distortion in OCT imaging, traditional distortion correction methods usually calibrate based on optical distortion parameters. Although these methods can correct distortion to some extent, their accuracy and stability are often limited. For example, traditional methods may not achieve ideal correction results when dealing with complex distortion types, and the calculation complexity is high. In addition, these methods usually rely on manual calibration and parameter adjustment, and are difficult to adapt to changes in different imaging environments and devices, so their adaptability is poor and their application scenarios are restricted. SUMMARY

[0004] To solve one of the above technical problems, the technical solution adopted by the present application is as follows:

[0005] According to one aspect of the present application, an OCT optical distortion correction method based on deep learning is provided, the method comprising the following steps:

[0006] inputting a distorted calibration board image obtained by using an OCT imaging unit to be corrected into a target convolutional neural network to generate a corrected calibration board image;

[0007] generating a correction pixel difference corresponding to each calibration point according to the difference between the pixel coordinates of each calibration point in the distorted calibration board image and the corrected calibration board image; the correction pixel difference includes a horizontal pixel difference and a vertical pixel difference;

[0008] transforming the correction pixel difference corresponding to each calibration point into a scanning position correction difference corresponding to each calibration point according to the transformation relationship between the pixel coordinate system and the OCT coordinate system; the scanning position correction difference includes a horizontal scanning position correction difference and a vertical scanning position correction difference;

[0009] According to the conversion relationship between the OCT coordinate system and the optical galvanometer driving voltage, the scanning position correction difference corresponding to each calibration point is converted into an X-direction driving compensation voltage and a Y-direction driving compensation voltage corresponding to each calibration point.

[0010] The X-direction driving compensation voltage and the Y-direction driving compensation voltage corresponding to each calibration point are used to correct the driving voltage of the optical galvanometer in the OCT imaging unit to be corrected.

[0011] According to a second aspect of the present application, a non-transitory computer readable storage medium is provided, the non-transitory computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the above-mentioned OCT optical distortion correction method based on deep learning.

[0012] According to a third aspect of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement the above-mentioned OCT optical distortion correction method based on deep learning.

[0013] The present application has at least one of the following beneficial effects:

[0014] The technical solution of the present application introduces a deep learning network, especially a convolutional neural network (CNN), to process the distorted marker point coordinates. The network structure adopts a multi-layer convolutional neural network, which includes a plurality of adaptive convolutional layers, pooling layers and fully connected layers. In the training process, the known marker board image with distortion and its corresponding standard non-distorted marker board image are used to optimize the network parameters using supervised learning. To ensure that the network can automatically learn the distortion features and establish a nonlinear mapping relationship between the distorted marker point coordinates and the non-distorted marker point coordinates. This nonlinear mapping relationship can stably generate the correction pixel difference corresponding to each calibration point, and can greatly improve the accuracy and efficiency of distortion correction. At the same time, the application of deep learning network makes the distortion correction process automatic and accurate, thereby effectively solving the problems of insufficient accuracy, complex processing and poor stability in the existing optical distortion correction technology, and has significant beneficial effects. In addition, compared with the traditional method relying on manual calibration and preset parameters, the present application avoids manual intervention, simplifies the entire processing process, and significantly reduces the computational complexity.

[0015] Secondly, the application of the technical solution of the present application in the OCT imaging system has wide adaptability. Through the deep learning-based image distortion correction method, the present application can process various types of optical distortion, especially in complex imaging environments, showing strong stability and robustness.

[0016] Finally, the automatic correction system designed by the present application greatly improves the automation level and work efficiency of the optical distortion correction process, and adapts to the batch and standardized operation requirements of the OCT equipment. The system can automatically complete the correction of the image without human intervention, improving the operation efficiency of the entire system, and is particularly suitable for large-scale production and high-precision application scenarios such as medical imaging and industrial detection. In general, the present application not only optimizes the precision and efficiency of optical distortion correction, but also promotes the further application and development of OCT technology in multiple fields. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 A flowchart of an OCT optical distortion correction method based on deep learning provided by an embodiment of the present application is shown in the figure.

[0019] Figure 2 A schematic diagram of a U-Net network provided by an embodiment of the present application is shown in the figure.

[0020] Figure 3 A schematic diagram of a deformable adaptive convolution kernel in a U-Net network provided by an embodiment of the present application is shown in the figure.

[0021] Figure 4 A schematic diagram of the working principle of optical galvanometer control scanning imaging provided by an embodiment of the present application is shown in the figure.

[0022] Figure 5 A comparison diagram of the effects of traditional distortion correction and the distortion correction method based on deep learning proposed by the present application is shown in the figure. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0024] Optical elements in OCT devices, such as lenses, optical path systems, can cause distortion of the geometry of the acquired images, mainly manifested as radial distortion and tangential distortion. Radial distortion usually causes the central region of the image to be clear, while the edge part is stretched or shrunk; tangential distortion is often caused by the tilt of the imaging system or optical path error. Optical distortion can seriously affect the image quality and the accuracy of subsequent measurements, therefore, in order to ensure the accuracy and reliability of the data, effective distortion correction must be performed on the image.

[0025] As a possible embodiment of the present application, as shown in Figure 1 , an OCT optical distortion correction method based on deep learning is provided, the method comprising the following steps:

[0026] S100: input the distorted reticle image acquired by using the OCT imaging unit to be corrected into the target convolutional neural network, and generate the corrected reticle image.

[0027] The target convolutional neural network can be CNN (convolutional neural network), and the network structure adopts a multi-layer convolutional neural network, which includes a plurality of adaptive convolutional layers, pooling layers and fully connected layers. When training, known reticle images with distortion and their corresponding standard reticle images without distortion are used to form training samples and labels, and the network parameters are optimized using supervised learning. Thus, through the training and learning of the CNN, it is ensured that the network can automatically learn the distortion features and establish a nonlinear mapping relationship between the distorted mark point coordinates and the non-distorted mark point coordinates. That is, how to correct the distorted mark points at different positions in the image to the standard position.

[0028] Specifically, as shown in Figure 2 and Figure 3 , the target convolutional neural network in this embodiment includes a U-Net network. The standard convolution kernel in the U-Net network is replaced by a deformable adaptive convolution kernel.

[0029] In the convolution operation of the traditional U-Net network, the size and weight of the convolution kernel are fixed (such as a standard convolution kernel of 3x3), and this kind of network architecture is suitable for the segmentation task of most conventional non-distorted images. However, when the image is distorted (for example, the X and Y directions are stretched or compressed), the traditional convolution may not be able to effectively handle these changes and cannot more effectively extract the corresponding distortion features. In this use scenario, in addition to the optical elements in the OCT device causing distortion of the image, there is also a reciprocating motion mode of acceleration, uniform speed and deceleration of the scanning components on the scanning path during the scanning process of the OCT device. As a result, there is also distortion (stretching or compression) caused by changes in the motion state during the acceleration and deceleration motion stages, which usually occurs along the scanning path. That is, it can occur in the horizontal direction (i.e., the X direction) or the vertical direction (i.e., the Y direction), so in order to adapt to the characteristics of this kind of distortion, the traditional standard convolution kernel needs to be replaced by a deformable adaptive convolution kernel.

[0030] The adaptive convolution can handle the distortion of the image through convolution kernels of different scales. In particular, in the X and Y directions of the image, the distortion can manifest as stretching or compression. Adaptive convolution can respond adaptively to different regions by adjusting the scale of the convolution kernel, in order to better capture local features. For example, when processing the distortion of different regions in the image, the convolution kernel can automatically scale according to the changes in the region, ensuring that feature extraction during convolution is not affected by differences in X or Y direction distortion. Thus, the improved U-Net network in this embodiment can better extract different distortion features in the distorted image during the training process, more accurately, in order to improve the final correction accuracy of the model on the distorted image.

[0031] In addition, the training images used in the training process are usually distorted calibration board images and their corresponding standard non-distorted calibration board images. In these training images, the center coordinates of the image corresponding to each marker point in the calibration board image need to be obtained in order for the model to learn.

[0032] Specifically, the pixel coordinates of each marker point in the calibration board image are obtained according to the following steps:

[0033] S101: Convert the calibration board image to a grayscale image. To remove the interference of color information and highlight the geometric shape of the calibration board.

[0034] S102: Smooth the grayscale image using Gaussian filtering. To reduce the interference of image noise and ensure that the marker point outline is clear and visible.

[0035] S103: using threshold processing, converting the gray scale image into a binary image to clearly separate the marker points from the background area. Specifically, the gray scale threshold can be determined according to actual needs, since the marker points and the background area have very obvious gray scale difference, so through the threshold division method, the image corresponding to the marker points can be segmented out very conveniently and quickly.

[0036] S104: using the least square method to fit the edge data of the marker points in the gray scale image to optimize the coordinate position of the edge points.

[0037] Since the OCT image (i.e. the calibration plate image with distortion) has certain distortion, when extracting the marker points, considering the influence of the distortion on the edge positioning, the least square method is used to fit the edge data to optimize the coordinate position of the edge points. The least square fitting is used to fit an optimal continuous curve or straight line from a set of discrete, noisy edge points, and the accurate position of each point is recalculated accordingly, so as to obtain more accurate, smoother and more consistent with the physical real edge of the marker point coordinate position.

[0038] S105: applying the contour detection algorithm to extract the edge of the marker points and calculating the center coordinates of the marker points.

[0039] On the basis of the foregoing S101-S104 processing, the contour detection algorithm (such as Canny, Sobel, Laplacian) is applied to extract the edge of the marker points and calculate the center coordinates thereof.

[0040] In addition, since the distortion of the OCT image in the present use field presents spatial distribution non-uniformity, especially in the edge area of the image, the distortion is often more serious, while the distortion of the central area is relatively light. In order to effectively process these differences in the distortion correction process, the traditional uniform loss function cannot provide sufficient fine adjustment. Correspondingly, the present application proposes a weighted loss function design, so that the model can pay different degrees of attention to different areas of the image during the training process, so as to realize the accurate optimization of the distortion of different areas.

[0041] Specifically, the loss function in the training process of the target convolutional neural network is a weighted loss function, which satisfies the following conditions:

[0042] .

[0043] Wherein, L is the total loss, Loss(x, y) is the distortion error of the image at pixel (x, y), and W(x, y) is the weighting coefficient of the image at pixel (x, y).

[0044] W(x, y) is the key to enable the model to pay different degrees of attention to different regions of the image during the training process. The specific calculation method of W(x, y) can be any one of the following three methods.

[0045] First

[0046] .

[0047] where r is the radial distance from the pixel point (x, y) to the center of the image, and a and β are the first and second adjustment coefficients, respectively, used to control the weighting intensity of the loss function at different positions. By adjusting these two coefficients, the weighting ratio between the center and the edge region can be flexibly adjusted to ensure that the network can effectively optimize the distortion correction of all regions. a and β can be determined with the training of the model.

[0048] Polar coordinate system conversion: To effectively identify different regions in the image, especially the edge and center regions of the image, the present application converts the image coordinates from the Cartesian coordinate system to the polar coordinate system. In the polar coordinate system, each pixel of the image can be described not only by the radial distance (the distance from the image center to the pixel point), but also by the angle, which helps to clearly reflect the distribution characteristics of distortion at different positions, especially in the case of radial variation of distortion.

[0049] Weighted loss matching: By introducing the polar coordinate system, the loss weight is adjusted according to the radial distance of each pixel. Specifically, the farther the region is from the center of the image, the more obvious the distortion, so these regions are given higher loss weights during the training process. This weighting strategy can ensure that the network pays more attention to the edge part of the image during optimization, thereby achieving higher accuracy in the correction of edge region distortion.

[0050] This polar coordinate system-based weighted loss matching scheme not only helps the network pay more attention to the edge region with more serious distortion when correcting the distortion of the OCT image, but also effectively improves the correction accuracy of the entire image, especially in the case of complex distortion distribution.

[0051] Second

[0052] Instead of:

[0053] .

[0054] where (x0, y0) is the center coordinate of the image, (x, y) is the coordinate of any pixel point in the image, and σ is the standard deviation of the distance between pixel points, used to control the speed of weight decay.

[0055] Third

[0056] is replaced by:

[0057] .

[0058] where (x0, y0) is the center coordinate of the image, (x, y) is the coordinate of any pixel point in the image, σ x is the standard deviation of the distance between pixel points in the horizontal direction, and σ y is the standard deviation of the distance between pixel points in the vertical direction.

[0059] The weight calculated by the weight calculation method in the second and the third will also increase with the distance from the center. The weight of the second conforms to the Gaussian distribution, which is more in line with the spatial distribution characteristics of general distortion around the image center. The weight of the third conforms to the elliptical distribution, and σ x and σ y respectively control the weight attenuation in the horizontal direction and the vertical direction, which can more flexibly adjust the size of the weight at different positions and can adapt to distortion distribution of different shapes.

[0060] Before S100: inputting the distorted calibration plate image obtained by using the OCT imaging unit to be corrected into the target convolutional neural network, the method further comprises:

[0061] S110: if the distortion degree of the distorted calibration plate image obtained by using the OCT imaging unit to be corrected is greater than a preset distortion threshold, performing preliminary correction processing on the distorted calibration plate image using a geometric correction method.

[0062] In this embodiment, the distortion degree can be obtained by the following method: drawing a horizontal straight line at a certain mark point in the distorted calibration plate image, and taking the number of mark points falling on the straight line as the distortion degree, or calculating the distance of the mark points in the row to the straight line as the distortion degree.

[0063] Generally, in order to obtain better correction results, for the OCT image with more serious geometric distortion, the classical geometric correction method is used for preliminary correction processing before the OCT image is input into the deep learning model. For trapezoidal distortion, a homographic transformation is used to adjust the trapezoidal region of the image to a rectangle; for nonlinear distortion, a nonlinear transformation is used to further correct the geometric structure of the image. After these preprocessing steps, most of the geometric distortion in the image has been eliminated, providing a more standardized input for the subsequent training of the deep learning model.

[0064] S200: generating a correction pixel difference corresponding to each calibration point according to the difference between the pixel coordinates of each calibration point in the distorted calibration plate image and the corrected calibration plate image. The correction pixel difference includes horizontal pixel difference and vertical pixel difference.

[0065] S300: According to the conversion relationship between the pixel coordinate system and the OCT coordinate system, the corrected pixel difference corresponding to each calibration point is converted into the scanning position correction difference corresponding to each calibration point. The scanning position correction difference includes the horizontal scanning position correction difference and the vertical scanning position correction difference.

[0066] S400: According to the conversion relationship between the OCT coordinate system and the optical vibration mirror driving voltage, the scanning position correction difference corresponding to each calibration point is converted into the X-direction driving compensation voltage and the Y-direction driving compensation voltage corresponding to each calibration point.

[0067] The conversion relationship between the pixel coordinate system (image coordinate system) and the OCT coordinate system is as follows:

[0068]

[0069] Wherein: X OCT and Y OCT are the target coordinates in the OCT coordinate system, X grid and Y grid are the original pixel coordinates, R01 and R11 are the rotation angle coefficients of the image coordinate system converted into the OCT coordinate system, Tx and Ty are the translation coefficients of the image coordinate system converted into the OCT coordinate system.

[0070] The conversion relationship between the OCT coordinate and the optical vibration mirror voltage is as follows:

[0071]

[0072] Wherein: Vx0 and Vy0 are the output voltages of the optical vibration mirror after conversion in the X and Y directions, and s is a scaling factor, used to adjust the proportion of the coordinate system. It determines the proportional relationship between each unit in the OCT coordinate system and the corresponding unit in the optical vibration mirror voltage coordinate system.

[0073] S500: The driving voltage of the optical vibration mirror in the OCT imaging unit to be corrected is corrected by using the X-direction driving compensation voltage and the Y-direction driving compensation voltage corresponding to each calibration point.

[0074] The image distortion correction method of the present application directly obtains the distortion correction mapping relationship model through the inference ability of the deep learning network, instead of relying on traditional optical distortion equations and preset parameters. According to the Figure 4As shown in the working principle of the scanning imaging, the method corrects by converting the pixel coordinate system in the image to the OCT coordinate system, and establishing the relationship matrix between the X and Y coordinates of the OCT coordinate system and the output voltages of the optical galvanometer in the X and Y directions, so as to realize the correction. Specifically, first, the distortion features in the image are extracted through an image processing algorithm, and are mapped into the OCT coordinate system. Then, the trained deep learning model is used to correct the output voltages of the optical galvanometer in the X and Y directions based on the mapping relationship between the input distortion coordinates and the output standard coordinates. In this way, manual intervention and complex parameter calibration can be avoided, and the efficiency and accuracy of distortion correction can be improved.

[0075] Since the target convolutional neural network can correct the image according to the prior experience obtained through training, the pixel distance between the horizontal and vertical directions of the same calibration point before and after correction, i.e. the distortion direction and size of the position corresponding to the calibration point. Therefore, based on the corrected pixel difference, the distortion compensation amount corresponding to the correction of the position can be obtained. Then, through the conversion between the coordinate systems, the distortion compensation amount corresponding to the scanning position of the corresponding marker point in the OCT coordinate system and the compensation amount corresponding to the driving voltage of the optical galvanometer are obtained.

[0076] The deep learning model is used to predict the "voltage pre-distortion (obtained by correcting the pixel difference)", and then the driving voltage of the galvanometer is adjusted in advance to exactly offset the pre-distortion, so as to finally realize the correction of the OCT scanning. It ingeniously combines coordinate transformation with deep learning modeling, realizes the end-to-end closed-loop correction from "observed distortion" to "control parameter adjustment", and does not directly do post-processing on the image to "repair" the distortion, but adjusts the driving signal of the galvanometer by learning the relationship between "ideal image coordinates" and "required galvanometer voltage", so as to generate an image without distortion in the next scanning. Specifically, the comparison result between the image corrected by the correction method proposed in the embodiment and the image corrected by the traditional Zhang Zhengyou calibration method is as shown in the figure. Figure 5 As shown in the figure, the effect of the image corrected by the correction method proposed in the embodiment is better.

[0077] As another embodiment of the present application, after S500: the driving voltage of the optical galvanometer in the OCT imaging unit to be corrected is corrected using the X-direction driving compensation voltage and the Y-direction driving compensation voltage corresponding to each calibration point, the method further comprises:

[0078] S600: comparing the corrected calibration plate image obtained by the corrected OCT imaging unit with the non-distortion calibration plate image to generate a total loss error.

[0079] S700: using the error back propagation mechanism to continue to optimize the target convolutional neural network model.

[0080] The total loss error Loss satisfies the following condition:

[0081] .

[0082] Wherein, λ1 and λ2 are weighting coefficients.

[0083] .

[0084] Wherein, Loss 直线度 is a straightness distortion error, m n is the slope of the fitting straight line of the n-th row of the marked points in the corrected calibration board image, C n is the intercept of the fitting straight line of the n-th row of the marked points in the corrected calibration board image, x i gtn and y i gtn are the position coordinates of the i-th marked point in the n-th row of the corrected calibration board image, and N is the number of marked points.

[0085] .

[0086] Wherein, Loss 平行度 is a parallelism distortion error, m i and m j are the slopes of the fitting straight lines corresponding to the i-th row and the j-th row of the corrected calibration board image, respectively.

[0087] Generally, if the correction effect is very good, each row or each column of the marked points in the corrected calibration board image should be strictly located on the same horizontal or vertical line, and the straight lines between different rows or columns should be parallel. However, generally, the correction cannot achieve such an effect, and thus, Loss 直线度 and Loss 平行度 are needed to be calculated for evaluation.

[0088] Specifically, Loss 直线度 and Loss 平行度 are both calculated based on the corrected calibration board image, a straight line is fitted in each row of the corrected calibration board image, and then the distance of each marked point to the straight line is calculated to determine the straightness of each row after correction, and the difference between the slopes of any two rows is calculated to evaluate the parallelism.

[0089] .

[0090] Wherein, Loss 像素坐标 is a pixel position error, x i and y i are the position coordinates of the i-th marked point in the non-distorted calibration board image, x i gtand y i gt is the position coordinate of the i-th marker point in the calibrated image after correction.

[0091] In order to further improve the correction accuracy in this embodiment, an optimization strategy based on error evaluation is proposed, which is used to optimize the target convolutional neural network again to further correct the OTC imaging unit after correction. Specifically, by comparing the corrected image with the actual geometric shape of the calibration board (the standard non-distorted calibration board image), the geometric distortion and the image pixel position error are calculated. The geometric distortion is evaluated by the Euclidean distance difference between the corrected marker point coordinates and the actual marker point coordinates, while the pixel position error is evaluated by calculating the position deviation of the marker point in the image. According to these errors, the error backpropagation mechanism is used to optimize the deep learning model again, and the goal is to minimize the distortion error and the pixel position deviation. The introduction of error evaluation and optimization mechanism further improves the correction accuracy, minimizes the error between the corrected image and the actual geometric shape of the calibration board, and ensures the high accuracy of the measurement results, thereby improving the correction accuracy, stability and reliability.

[0092] In addition, although the various steps of the methods in the present disclosure are described in a particular order in the accompanying drawings, this does not require or imply that the steps must be performed in this particular order, or that all of the steps shown must be performed to achieve the desired results. Additionally or alternatively, certain steps can be omitted, multiple steps can be combined into one step, and / or one step can be divided into multiple steps, etc.

[0093] From the above description of the embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.) or a network, and includes a number of instructions to make a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) execute the methods according to the embodiments of the present disclosure.

[0094] In the exemplary embodiments of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0095] Those skilled in the art can understand that each aspect of the present application can be implemented as a system, a method or a program product. Therefore, each aspect of the present application can be embodied in a form of entirely hardware, entirely software (including firmware, microcode, etc.), or a combination of hardware and software, which can be collectively referred to as "circuitry", "module" or "system".

[0096] The electronic device according to this embodiment of the present application. The electronic device is merely an example and should not bring any limitation to the function and use range of the embodiments of the present application.

[0097] The electronic device is in the form of a general computing device. The components of the electronic device can include, but are not limited to, the at least one processor described above, the at least one memory described above, and a bus connecting different system components (including the memory and the processor).

[0098] The memory stores program codes which can be executed by the processor, so that the processor executes the steps according to various exemplary embodiments of the present application described in the "Exemplary Method" section of the present specification.

[0099] The memory can include a readable medium in the form of a volatile memory, such as a random access memory (RAM) and / or a cache memory, and can further include a read-only memory (ROM).

[0100] The memory can further include programs / utilities with a set of (at least one) program modules, such as an operating system, one or more application programs, other program modules, and program data, each of which or some combination of which can include the implementation of a network environment.

[0101] The bus can be one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or a local bus using any of a variety of bus structures.

[0102] The electronic device can also communicate with one or more external devices such as a keyboard or a pointing device, through an I / O interface. The electronic device can communicate with one or more devices that enable a user to interact with it through a communication interface. The communication interface can enable communication through a communication network. Examples of communication networks include a local area network (LAN) and a wide area network (WAN), an inter-network (e.g., the Internet), etc. Communication through the communication network can be enabled by a communication interface using any one of a number of transfer protocols. Examples of communication interfaces include a modem, a network adapter, a Bluetooth® device, a wireless device, a telephone link, and / or the like. Further, communication can be enabled using infrared data association (IrDA) or other forms of wireless communication. The communication interface can also enable communication between the electronic device and one or more databases, such as a database that stores information about the user, the user's preferences, and / or the like. The electronic device can also include one or more internal storage devices, such as a hard disk drive, a floppy disk drive, a CD-ROM device, a DVD-ROM device, a Blu-ray® device, a flash memory device, a multimedia card, and / or the like, for storing information on various computer-readable storage media.

[0103] In the example embodiments of the present disclosure, a computer readable storage medium having stored thereon a program product capable of implementing the above-described methods of the present specification is also provided. In some possible implementations, various aspects of the present disclosure can also be implemented as a program product in the form of a computer readable medium embodying a sequence of instructions executable by a terminal device, which causes the terminal device to execute the steps described in the above-mentioned "Example Methods" section of the present specification according to various example embodiments of the present disclosure.

[0104] The program product can take any combination of one or more computer readable media. The computer readable media can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0105] The computer readable signal medium can include a computer readable signal medium that stores the program code in a modulated data signal. Such a modulated data signal can carry program code for use by one or more devices connected to a communication network, such as the Internet. The modulated data signal can be carried by a carrier wave. The computer readable storage medium can also be one of, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above.

[0106] The program code embodied on the computer readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0107] The program code, when executed, can perform a method of the present application. The program code can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider. The present application can also be implemented as a computer-readable storage medium having stored thereon the program code.

[0108] Furthermore, the above-described diagrams are only schematic and are non-limiting. It is expressly contended that the processes set forth in the diagrams can be performed in an order other than that depicted. Additionally, it is expressly contemplated that the processes set forth in the diagrams can be performed synchronously or asynchronously, e.g., in a plurality of modules.

[0109] It should be noted that, although several modules or units for device for action execution are mentioned in the foregoing detailed description, such a division into modules or units is not mandatory. Indeed, according to an embodiment of the present disclosure, the features and functionalities of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functionalities of one module or unit described above can be further divided into several modules or units embodied by.

[0110] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any changes or replacements easily conceived by those skilled in the art within the technical scope disclosed by the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A deep learning-based OCT optical distortion correction method, characterized in that, The method comprises the following steps: inputting the distorted calibration board image obtained by using the OCT imaging unit to be corrected into a target convolutional neural network to generate a corrected calibration board image; generating a correction pixel difference corresponding to each calibration point according to the difference between the pixel coordinates of each calibration point in the distorted calibration board image and the corrected calibration board image; the correction pixel difference comprises a horizontal pixel difference and a vertical pixel difference; translating the correction pixel difference corresponding to each calibration point into a scanning position correction difference corresponding to each calibration point according to the conversion relationship between the pixel coordinate system and the OCT coordinate system; the scanning position correction difference comprises a horizontal scanning position correction difference and a vertical scanning position correction difference; translating the scanning position correction difference corresponding to each calibration point into an X-direction drive compensation voltage and a Y-direction drive compensation voltage corresponding to each calibration point according to the conversion relationship between the OCT coordinate system and the optical galvanometer drive voltage; correcting the drive voltage of the optical galvanometer in the OCT imaging unit to be corrected by using the X-direction drive compensation voltage and the Y-direction drive compensation voltage corresponding to each calibration point.

2. The method of claim 1, wherein, The target convolutional neural network comprises a U-Net network; the standard convolution kernel in the U-Net network is replaced by a deformable adaptive convolution kernel.

3. The method of claim 1, wherein, The loss function in the training process of the target convolutional neural network is a weighted loss function, and the weighted loss function satisfies the following conditions: ； wherein L is the total loss, Loss(x, y) is the distortion error of the image at pixel (x, y), and W(x, y) is the weighting coefficient of the image at pixel (x, y); ； wherein r is the radial distance of pixel point (x, y) to the center of the image, and α and β are respectively the first adjustment coefficient and the second adjustment coefficient for controlling the weighting intensity of the loss function at different positions.

4. The method of claim 3, wherein, is replaced by: ； wherein (x0, y0) is the center coordinate of the image, (x, y) is the coordinate of any pixel point in the image, and σ is the standard deviation of the distance between the pixel points, for controlling the speed of weight decay.

5. The method of claim 3, wherein, is replaced by: ； where (x0, y0) is the center coordinate of the image, (x, y) is the coordinate of any pixel point in the image, σ x is the standard deviation of the distance between pixel points in the horizontal direction, and σ y is the standard deviation of the distance between pixel points in the vertical direction.

6. The method of claim 2, wherein, After the drive voltage of the optical galvanometer in the OCT imaging unit to be corrected is corrected by using the X-direction drive compensation voltage and the Y-direction drive compensation voltage corresponding to each calibration point, the method further comprises: comparing the corrected calibration board image obtained by the corrected OCT imaging unit with the non-distorted calibration board image to generate a total loss error; continuing to optimize the target convolutional neural network using the error back propagation mechanism; the total loss error Loss satisfies the following conditions: ； wherein λ1 and λ2 are weighting coefficients; ； wherein, Loss 直线度 is the straightness distortion error, m n is the slope of the fitted straight line of the n-th row of the marker points in the rectified calibration board image, C n is the intercept of the fitted straight line of the n-th row of the marker points in the rectified calibration board image, x i gtn and y i gtn are the position coordinates of the i-th marker point in the n-th row of the rectified calibration board image, and N is the number of marker points. ； wherein Loss 平行度 is the parallelism distortion error, m i and m j are the slopes of the fitted straight lines corresponding to the i-th and j-th rows in the image of the calibrated plate after correction, respectively. ； where Loss 像素坐标 is the pixel position error, x i and y i are the position coordinates of the i-th marker point in the undistorted calibration board image, x i gt and y i gt are the position coordinates of the i-th marker point in the rectified calibration board image.

7. The method of claim 1, wherein, Before inputting the distorted calibration board image obtained by using the OCT imaging unit to be corrected into the target convolutional neural network, the method further comprises: if the distortion degree of the distorted calibration board image obtained by using the OCT imaging unit to be corrected is greater than a preset distortion threshold, a geometric correction method is used to perform preliminary correction processing on the distorted calibration board image.

8. The method of claim 1, wherein, The pixel coordinates corresponding to each calibration point in the calibration board image are obtained according to the following steps: convert the calibration board image into a grayscale image; smooth the grayscale image by using Gaussian filtering; convert the grayscale image into a binary image by using threshold processing to clearly separate the marker points from the background area; The edge data of the mark point in the gray image is fitted by using a least square method to optimize the coordinate position of the edge point; An edge of the mark point is extracted by using a contour detection algorithm, and a center coordinate of the mark point is calculated. 9.A non-transitory computer-readable storage medium storing a computer program, the computer program comprising instructions configured to cause a processor to perform the method according to any one of claims 1 to 8. The computer program, when executed by a processor, implements the deep learning-based OCT optical distortion correction method of any one of claims 1-8.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor, when executing the computer program, implements the deep learning-based OCT optical distortion correction method of any one of claims 1-8.

Images