A distortion image correction method based on optical imaging

Abstract

This invention provides a method for correcting distorted images based on optical imaging. The method includes training a distortion correction model using historical data from a source imaging device to obtain optical features and similarity weights for each region of the source and target devices. After training the image correction model based on the source device's historical data, the model is combined with the target device's historical data and similarity weights to generate corrected images and spatial transformation features for each region of the target device. Subsequently, an image quality evaluation index for each region is calculated based on the correction results, transformation features, and optical features, and the degree of distortion correction for the target device is summarized. This method effectively utilizes the rich data from the source device to improve the distortion correction accuracy of the target device and achieve quantitative verification of the correction effect.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically, to a distortion image correction method based on optical imaging. Background Art

[0002] In the field of optical imaging, image distortion is a common problem. Distorted images can seriously affect the quality of images and subsequent application effects. For example, in medical imaging, satellite remote sensing, industrial detection and other fields, the presence of distortion may lead to diagnostic errors, inaccurate target recognition and other problems. Traditional distortion correction methods mainly rely on the geometric model and calibration process of the optical system to correct the image by calculating the distortion parameters. Although these methods can improve image quality to a certain extent, they often require a complex calibration process, and the correction effect is not ideal for complex distortion patterns, such as nonlinear distortion. In addition, traditional distortion correction methods are usually optimized for a single imaging device. When applied to different imaging devices, recalibration and parameter adjustment are required, which increases the complexity and cost of correction.

[0003] In the process of implementing the embodiments of the present invention, there are at least the following problems or defects in the prior art: the distortion correction method of the prior art relies on a complex calibration process, and has limited correction effect on complex distortion patterns. At the same time, it is difficult to adapt to the differences between different imaging devices, resulting in poor versatility and adaptability of the correction. Summary of the Invention

[0004] The present invention provides a method for correcting distorted images based on optical imaging, comprising: training a distortion correction model based on a training set including historical distorted image data and optical parameter data of a source imaging device and a target imaging device, and obtaining optical features and optical feature similarity weights of each region of the source imaging device and the target imaging device; wherein the amount of historical data of the source imaging device is greater than that of the target imaging device to be corrected; training an image correction model based on the historical distorted image data of the source imaging device; obtaining corrected image results and spatial transformation features of each region of the target imaging device using the image correction model based on the optical feature similarity weights and the historical distorted image data of the target imaging device; obtaining an image quality evaluation index of the corresponding region based on the corrected image results, spatial transformation features and the optical features of each region, summarizing the image quality evaluation index of each region, and obtaining the degree of distortion correction completion of the target imaging device.

[0005] Furthermore, the image quality evaluation index of the corresponding area is calculated based on the following formula: ;in, : Weight factor for correcting the image result, : Weight factor of spatial transformation feature, : weight factor of optical characteristics, : Corrected image result of the region, : Spatial transformation characteristics of the region, : Optical characteristics of the region.

[0006] Furthermore, the distortion correction model includes a feature network and a transformation network; the distortion correction model is trained, including: based on the training set, using the feature network to extract the optical features of the source imaging device and the target imaging device respectively; performing region division based on the optical features, and using the transformation network to perform spatial transformation learning of each region to obtain comprehensive optical embedding features of each region of the source imaging device and the target imaging device; based on the comprehensive optical embedding features of the source imaging device and the target imaging device, respectively, using a weighted network and a weighted network loss function to learn region weights to obtain similarity weights of each region of the source imaging device and the target imaging device; after the training is completed, the comprehensive optical embedding features and similarity weights obtained based on the training set using the distortion correction model are the optical features and optical feature similarity weights of each region of the source imaging device and the target imaging device.

[0007] Furthermore, extracting the optical features includes: constructing a position association matrix based on the pixel positions of the source imaging device and the target imaging device respectively; constructing an optical point association matrix based on the optical point distribution of the source imaging device and the target imaging device respectively; constructing an edge similarity matrix based on the edge features of the source imaging device and the target imaging device respectively; constructing a distortion type association matrix based on the distortion pattern data of the source imaging device and the target imaging device respectively; and obtaining the optical features based on the above-mentioned multiple matrices.

[0008] Furthermore, the transformation network sequentially utilizes multi-level adaptive learning based on structural similarity to perform spatial transformation learning of each region, including: utilizing a multi-scale convolutional network to process the position association matrix, optical point association matrix, edge similarity matrix, and distortion type association matrix to obtain a feature embedding vector corresponding to each matrix; utilizing the self-attention mechanism of the feature fusion layer to update and fuse the feature embedding vectors of the source imaging device and the target imaging device respectively to obtain an updated feature embedding, and outputting a comprehensive optical embedding feature through affine transformation; utilizing domain-level adaptive learning, pixel-level adaptive learning, and neighborhood-level adaptive learning based on structural similarity to sequentially perform spatial transformation learning of each region.

[0009] Furthermore, the domain-level adaptive learning includes: extracting embedding features of the central area, edge area and distortion area in the updated feature embedding, and calculating the structural similarity of the three types of embedded features; calculating the loss function of the corresponding region type based on the structural similarity of the three types of embedded features; and training the transformation network based on the domain-level overall loss function obtained from the loss function of each type of region type.

[0010] Furthermore, the domain-level overall loss function is expressed as: ;in, : domain-level overall loss function, : Domain-level loss of source imaging device, : domain-level loss of target imaging device, : structural loss in the central area, : Edge area gradient loss, : Distorted region feature loss, : weight coefficient, : edge adjacency matrix, : edge similarity matrix, : are the distortion feature embeddings of the source device and the target device respectively, : gradient operator, : Structural similarity index.

[0011] Furthermore, the pixel-level adaptive learning includes: calculating the structural similarity between the integrated optical embedding features of the source imaging device and the target imaging device at corresponding pixel positions; calculating the gradient amplitude difference between the integrated optical embedding features of the source imaging device and the target imaging device at corresponding pixel positions; and constructing a pixel-level loss function based on the structural similarity and the gradient amplitude difference: ;in, is the pixel-level feature vector of the source device, : pixel-level feature vector of the target device, : gradient operator, : Weight coefficient; Training the transformation network based on the pixel-level loss function.

[0012] Furthermore, the neighborhood-level adaptive learning includes: obtaining a neighborhood feature vector by connecting embedded features of a neighboring pixel pair from a source imaging device and a target imaging device; and training the transformation network based on the neighborhood feature vector using a structural similarity loss function.

[0013] Furthermore, the weighted network loss function is: ;in, : mean square error, is the image distortion parameter predicted by the model, : The actual measured image distortion parameter, is the weight of the regularization term, is the optical characteristic similarity weight of the source imaging device, is the optical feature similarity weight of the target imaging device.

[0014] The above-mentioned embodiments of the present invention have at least the following beneficial effects: 1. By utilizing the rich historical data of the source imaging device to train the distortion correction model, and combining it with the regional optical feature similarity weights of the target device, the problem of low correction accuracy of the target device due to insufficient data is solved, and the distortion correction knowledge transfer across devices is realized, thereby improving the image correction effect of the target device.

[0015] 2. A multi-level adaptive learning mechanism is adopted to perform refined spatial transformation optimization based on the optical characteristics of different regions, overcoming the lack of adaptability of traditional methods in complex distortion scenarios, and being able to more accurately handle the correction problems of central areas, edge areas, and distortion areas.

[0016] 3. By constructing an image quality evaluation index and conducting a quantitative evaluation based on the correction results, spatial transformation characteristics, and optical characteristics, the problem that traditional methods are difficult to objectively measure the correction effect is solved. This provides a reliable basis for the optimization and adjustment of distortion correction, ensuring that the quality of the corrected image meets the actual application requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the following detailed description with reference to the accompanying drawings, in which several embodiments of the present invention are shown by way of example and not limitation.

[0018] Figure 1 A flowchart of a method for correcting distorted images based on optical imaging is provided in accordance with an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided solely to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. Rather, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.

[0020] Those skilled in the art will appreciate that the embodiments of the present invention may be implemented as a system, apparatus, device, method, or computer program product. Therefore, the present invention may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software.

[0021] It should be noted that any number of elements in the drawings is for illustration only and not for limitation, and any naming is only for distinction and does not have any limiting meaning.

[0022] Reference below Figure 1 , Figure 1 The flowchart of the distortion image correction method based on optical imaging provided by one embodiment of the present invention is as follows. Figure 1 As shown, a distortion image correction method based on optical imaging includes: S1, based on a training set including historical distortion image data and optical parameter data of a source imaging device and a target imaging device, training a distortion correction model, and obtaining optical features and optical feature similarity weights of each region of the source imaging device and the target imaging device; wherein the amount of historical data of the source imaging device is greater than that of the target imaging device to be corrected.

[0023] S2. An image correction model is obtained by training based on historical distorted image data of the source imaging device.

[0024] S3. Based on the optical feature similarity weights and the historical distorted image data of the target imaging device, the image correction model is used to obtain the correction image results and spatial transformation features of each area of ​​the target imaging device.

[0025] S4. Based on the corrected image results, spatial transformation features, and optical features of each region, an image quality evaluation index of the corresponding region is obtained, and the image quality evaluation indexes of each region are summarized to obtain a degree of distortion correction completion of the target imaging device.

[0026] It should be noted that the present invention proposes a method for distorted image correction based on optical imaging. This method trains a distortion correction model using historical distorted image data from a source imaging device. Based on optical feature similarity weights and the historical distorted image data from a target imaging device, this model is used to obtain corrected image results and spatial transformation features for each region of the target imaging device. The source imaging device refers to an imaging device with a large amount of historical distorted image data, while the target imaging device refers to the imaging device that requires distortion correction. Optical features refer to optical-related characteristics exhibited by an imaging device during the imaging process, such as distortion patterns and edge features. These characteristics can reflect the imaging quality of the imaging device. The optical feature similarity weight is a parameter that measures the degree of similarity in optical features between the source and target imaging devices. This weight can better apply the correction experience of the source imaging device to the target imaging device. Finally, by summarizing the image quality evaluation index of each region, the distortion correction completion degree of the target imaging device is obtained, thereby effectively correcting the distorted image of the target imaging device.

[0027] Specifically, the distortion correction model, a core component of the present invention, comprises a feature network and a transformation network. The feature network extracts optical features from the distorted image data of the source and target imaging devices. These features include a position correlation matrix constructed from pixel positions, an optical point correlation matrix constructed from optical point distribution, an edge similarity matrix constructed from edge features, and a distortion type correlation matrix constructed from distortion pattern data. The transformation network is responsible for partitioning regions based on the extracted optical features and learning the spatial transformation characteristics of each region to achieve distorted image correction. During training, a weighted network and a weighted network loss function are used to learn region weights, deriving similarity weights for each region of the source and target imaging devices. These weights reflect the importance of different regions in terms of optical characteristics and play a key role in subsequent image correction. The image quality evaluation index is an important indicator of corrected image quality. It comprehensively considers multiple factors, including the corrected image results, spatial transformation characteristics, and optical characteristics. It is calculated using a specific formula. The weighting factors in this formula can be adjusted according to actual needs to adapt to different correction objectives and application scenarios.

[0028] Preferably, the distortion correction model construction process can be further refined. First, the construction of the feature network needs to consider various optical feature extraction methods. For example, the position correlation matrix can be obtained by calculating the positional relationship between each pixel in the image and its surrounding pixels, the optical point correlation matrix can be constructed by analyzing the optical point distribution in the image, the edge similarity matrix is ​​calculated based on the image's edge features, and the distortion type correlation matrix is ​​constructed based on distortion pattern data. The construction methods of these matrices can be optimized and adjusted based on the specific imaging device and application scenario. In the transformation network, multi-level adaptive learning is a key step in achieving spatial transformation feature learning. It includes domain-level adaptive learning based on structural similarity, pixel-level adaptive learning, and neighborhood-level adaptive learning. Domain-level adaptive learning focuses on the overall structural features of the image, pixel-level adaptive learning emphasizes detailed features, and neighborhood-level adaptive learning considers local relationships between pixels. Through these three levels of adaptive learning, the spatial transformation features of the image can be more comprehensively learned, thereby improving the accuracy and effectiveness of distortion correction. In the calculation of the image quality assessment index, the weighting factors can be adjusted according to actual needs. For example, if you are more concerned about the visual effect of the corrected image, you can appropriately increase the weight factor of the corrected image result; if you are more concerned about the structural characteristics of the image, you can increase the weight factor of the spatial transformation characteristics. By properly setting these weight factors, the image quality evaluation index can more accurately reflect the quality of the corrected image.

[0029] In some embodiments, the image quality evaluation index of the corresponding area is calculated based on the following formula: ;in, : Weight factor for correcting the image result, : Weight factor of spatial transformation feature, : weight factor of optical characteristics, : Corrected image result of the region, : Spatial transformation characteristics of the region, : Optical characteristics of the region.

[0030] It should be noted that the method for calculating the image quality evaluation index in the present invention is carried out by comprehensively considering three factors: the corrected image results, spatial transformation characteristics, and optical characteristics, and assigning different weighting factors to each of them. This method allows the weights of each factor to be flexibly adjusted according to different correction objectives and application scenarios, thereby obtaining a comprehensive and targeted image quality evaluation index. The weight factors here reflect the importance of each feature in the image quality evaluation, while the corrected image results, spatial transformation characteristics, and optical characteristics are indicators for evaluating the corrected image from different perspectives. Together, they form the basis for image quality evaluation.

[0031] Specifically, weighting factors are key parameters used to balance the contributions of different features in image quality assessment. For example, if the visual effect after correction is the primary concern, the weighting factor for the corrected image should be set relatively high; if the fidelity of spatial structure is more important, the weighting factor for spatial transformation features should be increased accordingly; and if optical characteristics significantly impact image quality, the weighting factor for optical features should also be appropriately increased. Corrected image results can be quantified by comparing the differences between the images before and after correction, for example using metrics such as image clarity and contrast. Spatial transformation features can be evaluated by analyzing the changes in image parameters during spatial transformation. For example, in affine transformations, this can be measured by calculating the changes in the parameters of the transformation matrix. Optical features can be determined by analyzing the optical properties of the imaging device, such as the degree of distortion and edge sharpness. The specific evaluation methods and weighting factors for these features can be adjusted based on the characteristics of the imaging device and the correction objectives to ensure that the image quality evaluation index accurately reflects the correction effect.

[0032] Preferably, the calculation process of the image quality evaluation index can be further refined. In practical applications, the setting of the weight factor can be determined through experiments and experience. For example, the optimal weight combination can be selected by comparing the correction effects under different weight settings. For the corrected image results, a variety of image quality evaluation indicators can be used for comprehensive evaluation, such as structural similarity index and visual information fidelity, etc., to more comprehensively reflect the quality of the corrected image. The calculation of spatial transformation features can be combined with a specific transformation model. For example, in affine transformation, the spatial transformation features can be evaluated by calculating the parameter changes of the transformation matrix. The calculation of optical features can be based on the optical model of the imaging device, for example, by measuring parameters such as distortion coefficient and edge sharpness to quantify the optical features. Through these specific calculation methods and parameter settings, the image quality evaluation index can be calculated more accurately, thereby providing a reliable basis for the evaluation of the distortion correction effect.

[0033] In some embodiments, the distortion correction model includes a feature network and a transformation network; training to obtain the distortion correction model includes: based on the training set, using the feature network to extract the optical features of the source imaging device and the target imaging device respectively; performing region division based on the optical features, and using the transformation network to perform spatial transformation learning of each region to obtain comprehensive optical embedding features of each region of the source imaging device and the target imaging device; based on the comprehensive optical embedding features of the source imaging device and the target imaging device, respectively, using a weighted network and a weighted network loss function to learn region weights to obtain similarity weights of each region of the source imaging device and the target imaging device; after the training is completed, the comprehensive optical embedding features and similarity weights obtained based on the training set using the distortion correction model are the optical features and optical feature similarity weights of each region of the source imaging device and the target imaging device.

[0034] It should be noted that the distortion correction model of the present invention is composed of a feature network and a transformation network. The function of the feature network is to extract optical features from the distorted image data of the source imaging device and the target imaging device. These features can reflect the imaging characteristics of the imaging device. Optical features include position correlation matrix, optical point correlation matrix, edge similarity matrix and distortion type correlation matrix, etc. These matrices can describe the optical characteristics of the image from different angles. The transformation network is responsible for dividing the region according to the extracted optical features and learning the spatial transformation features of each region to achieve correction of the distorted image. The weighted network and the weighted network loss function are used to learn the regional weights and obtain the similarity weights of each region of the source imaging device and the target imaging device. These weights can reflect the degree of similarity of different regions in optical characteristics, thereby providing a basis for distortion correction.

[0035] Specifically, the feature network and transformation network are the core components of the distortion correction model. The feature network extracts optical features by constructing multiple matrices, including a position correlation matrix, an optical point correlation matrix, an edge similarity matrix, and a distortion type correlation matrix. The position correlation matrix is ​​constructed based on pixel positions and reflects the spatial distribution of pixels in the image; the optical point correlation matrix is ​​constructed based on the optical point distribution and describes the distribution of optical points in the image; the edge similarity matrix is ​​constructed based on edge features and measures the similarity of image edges; and the distortion type correlation matrix is ​​constructed based on distortion pattern data and describes the type of distortion in the image. The transformation network uses these optical features to perform region segmentation and learn the spatial transformation characteristics of each region. A weighted network and a weighted network loss function are used to learn region weights. By comparing the optical features of the source and target imaging devices, similarity weights for each region are derived. These weights reflect the degree of similarity in optical features between different regions and provide an important reference for distortion correction.

[0036] Preferably, the distortion correction model construction process can be further refined. The construction of the feature network requires consideration of various optical feature extraction methods. For example, the position correlation matrix can be obtained by calculating the positional relationship between each pixel in the image and its surrounding pixels; the optical point correlation matrix can be constructed by analyzing the distribution of optical points in the image; the edge similarity matrix is ​​calculated based on the edge features of the image; and the distortion type correlation matrix is ​​constructed based on distortion pattern data. The construction methods of these matrices can be optimized and adjusted based on the specific imaging device and application scenario. In the transformation network, multi-level adaptive learning is a key step in achieving spatial transformation feature learning. It includes domain-level adaptive learning based on structural similarity, pixel-level adaptive learning, and neighborhood-level adaptive learning. Domain-level adaptive learning focuses on the overall structural features of the image, pixel-level adaptive learning emphasizes detailed features, and neighborhood-level adaptive learning considers local relationships between pixels. Through these three levels of adaptive learning, the spatial transformation features of the image can be more comprehensively learned, thereby improving the accuracy and effectiveness of distortion correction. The construction of the weighted network and the weighted network loss function requires consideration of how to effectively learn regional weights to reflect the degree of similarity in optical features between different regions. By reasonably setting these networks and loss functions, the distortion correction model can be better adapted to different imaging devices and distortion conditions, thereby improving the correction effect.

[0037] In some embodiments, extracting the optical features includes: constructing a position association matrix based on the pixel positions of the source imaging device and the target imaging device, respectively; constructing an optical point association matrix based on the optical point distribution of the source imaging device and the target imaging device, respectively; constructing an edge similarity matrix based on the edge features of the source imaging device and the target imaging device, respectively; constructing a distortion type association matrix based on the distortion pattern data of the source imaging device and the target imaging device, respectively; and obtaining the optical features based on the above-mentioned multiple matrices.

[0038] It should be noted that the process of extracting optical features in the present invention is achieved by constructing multiple matrices, including a position correlation matrix, an optical point correlation matrix, an edge similarity matrix, and a distortion type correlation matrix. These matrices describe the optical characteristics of the imaging device from different perspectives and can provide important reference information for subsequent distortion correction. The position correlation matrix reflects the spatial distribution relationship of pixels in the image, the optical point correlation matrix describes the distribution of optical points, the edge similarity matrix measures the similarity of image edges, and the distortion type correlation matrix describes the distortion type of the image. Through the comprehensive analysis of these matrices, comprehensive optical features can be obtained, thereby more accurately performing distortion correction.

[0039] Specifically, the position association matrix is ​​constructed based on the pixel positions of the source imaging device and the target imaging device. It reflects the spatial position relationship between each pixel in the image and its surrounding pixels. The optical point association matrix is ​​constructed based on the distribution of optical points. Optical points generally refer to points in the image with specific optical characteristics, such as high-brightness points or points of specific shapes. The distribution of these points can reflect the optical characteristics of the imaging device. The edge similarity matrix is ​​constructed based on edge features. Edge features refer to areas in the image with significant changes in brightness or color. These areas usually contain rich image structural information. The distortion type association matrix is ​​constructed based on distortion pattern data. Distortion pattern data refers to data that describes image distortion characteristics, such as radial distortion, tangential distortion, etc. The construction methods of these matrices can be adjusted according to the specific imaging device and application scenario to ensure that optical features reflecting the optical characteristics of the imaging device can be accurately extracted. When constructing these matrices, it is necessary to consider how to select appropriate parameters and algorithms. For example, when calculating pixel position associations, different distance measurement methods can be selected; when analyzing the distribution of optical points, different thresholds can be set to identify optical points; when extracting edge features, different edge detection algorithms can be used; when describing distortion patterns, different mathematical models can be used to fit the distortion features.

[0040] Preferably, the process of extracting optical features can be further refined. For example, when constructing a position correlation matrix, the spatial relationship between pixels can be quantified by calculating the Euclidean distance or other distance metric between each pixel in the image and its surrounding pixels. For an optical point correlation matrix, optical points in the image can be identified by setting a brightness threshold or using a specific shape matching algorithm, and the correlation between these points can be calculated. When constructing an edge similarity matrix, the Canny edge detection algorithm or other advanced edge detection methods can be used to extract image edges, and edge similarity can be calculated by comparing the direction and intensity of edges in different regions. For a distortion type correlation matrix, appropriate mathematical models can be selected to describe the distortion characteristics based on the distortion type, such as radial distortion or tangential distortion, and the correlation between distortion types can be calculated by fitting these models. In practical applications, the construction and analysis of these matrices can be implemented by writing corresponding algorithms whose input parameters include image data, distortion pattern data, etc., and whose output is the constructed matrix. These matrices can be input as feature vectors into subsequent distortion correction models, providing the models with rich optical feature information, thereby improving the accuracy and effectiveness of distortion correction.

[0041] In some embodiments, the transformation network sequentially utilizes multi-level adaptive learning based on structural similarity to perform spatial transformation learning of each region, including: utilizing a multi-scale convolutional network to process the position association matrix, optical point association matrix, edge similarity matrix, and distortion type association matrix to obtain a feature embedding vector corresponding to each matrix; utilizing the self-attention mechanism of the feature fusion layer to update and fuse the feature embedding vectors of the source imaging device and the target imaging device respectively to obtain an updated feature embedding, and outputting a comprehensive optical embedding feature through affine transformation; utilizing domain-level adaptive learning, pixel-level adaptive learning, and neighborhood-level adaptive learning based on structural similarity to sequentially perform spatial transformation learning of each region.

[0042] It should be noted that the implementation process of the transformation network in the present invention is completed through multi-level adaptive learning based on structural similarity. This process includes using a multi-scale convolutional network to process multiple correlation matrices, such as position correlation matrix, optical point correlation matrix, edge similarity matrix and distortion type correlation matrix, to extract the feature embedding vector corresponding to each matrix. Subsequently, the feature embedding vector is updated and fused through the self-attention mechanism of the feature fusion layer, and finally the comprehensive optical embedding feature is output. This process also involves adaptive learning at the domain level, pixel level and neighborhood level, and optimizes the different levels of features of the image to achieve more accurate spatial transformation learning. This multi-level adaptive learning method can effectively handle complex distortions in images and improve the accuracy and robustness of correction.

[0043] Specifically, the multi-scale convolutional network (MCN) used in the multi-level adaptive learning process is a deep learning architecture used to process image data and extract features. It uses convolution kernels of different scales to capture local and global features in the image. The position correlation matrix, optical point correlation matrix, edge similarity matrix, and distortion type correlation matrix each describe the optical properties of the image from different perspectives. The MCN processes these matrices, generating feature embedding vectors corresponding to each matrix. These vectors contain information about the image's structure and distortion. The self-attention mechanism in the feature fusion layer is used to update and fuse these feature embedding vectors. By calculating the similarity between feature vectors, it enhances important features and suppresses unimportant ones, ultimately generating a comprehensive optical embedding feature. Domain-level adaptive learning focuses on the overall structural features of the image, pixel-level adaptive learning focuses on detailed features, and neighborhood-level adaptive learning focuses on local relationships between pixels. These different levels of learning work together to achieve comprehensive optimization of image spatial transformation.

[0044] Preferably, the construction and training process of the transformation network can be further refined. In a multi-scale convolutional network, convolution kernels of different sizes can be set to extract multi-scale features of the image. For example, 3×3, 5×5, and 7×7 convolution kernels can be used to capture fine-grained, medium-grained, and coarse-grained features of the image, respectively. In the feature fusion layer, the self-attention mechanism updates and fuses the feature embedding vectors by calculating the dot product similarity between feature vectors. For domain-level adaptive learning, features of the central, edge, and distorted regions of the image can be extracted, and the structural similarity between these regions can be calculated. Pixel-level adaptive learning can optimize the transformation parameters by calculating the difference between the feature vectors at each pixel position between the source and target images. Neighborhood-level adaptive learning can further refine the spatial transformation by considering the local neighborhood information of the pixel. During training, a structural similarity loss function can be used to optimize network parameters to ensure that the network learns the optimal spatial transformation features. Through these specific construction and training steps, the transformation network can more effectively handle image distortion and improve the correction effect.

[0045] In some embodiments, the domain-level adaptive learning includes: extracting embedding features of the central area, edge area and distortion area in the updated feature embedding, and calculating the structural similarity of the three types of embedded features; calculating the loss function of the corresponding region type based on the structural similarity of the three types of embedded features; and training the transformation network based on the domain-level overall loss function obtained from the loss function of each region type.

[0046] It should be noted that domain-level adaptive learning in the present invention is an important part of the transformation network. Its main purpose is to optimize the network parameters by extracting the embedded features of the central area, edge area, and distorted area of ​​the image and calculating the structural similarity of these areas. The domain-level overall loss function is a key indicator for measuring the effect of domain-level adaptive learning. It comprehensively considers the domain-level loss of the source imaging device and the target imaging device, as well as the structural loss, gradient loss, and feature loss of different regions. By optimizing this loss function, the transformation network can be effectively trained to better adapt to the optical characteristics of different imaging devices and improve the accuracy of distortion correction.

[0047] Specifically, terms involved in domain-level adaptive learning include central region, edge region, distorted region, structural similarity, and domain-level loss function. The central region refers to the relatively stable area of ​​the image that is less affected by distortion; the edge region refers to the boundary of the image, which typically contains rich structural information; and the distorted region refers to the part of the image with more pronounced distortion. Structural similarity is a metric that measures the degree of similarity between image regions and is typically calculated based on three dimensions: brightness, contrast, and structure. The domain-level loss function comprehensively considers the feature losses of different regions. It includes a structural loss for the central region, a gradient loss for the edge region, and a feature loss for the distorted region. These loss functions are calculated by comparing the feature embeddings of the source and target imaging devices, with weight coefficients used to balance the contributions of different losses to the overall loss. The domain-level overall loss function combines these losses to evaluate the performance of the transformation network and guide the network training process.

[0048] Preferably, the implementation process of domain-level adaptive learning can be further refined. First, the central area, edge area, and distorted area can be determined by an image segmentation algorithm. For example, an edge detection algorithm can be used to identify the edge area, a distortion detection algorithm can be used to locate the distorted area, and the remaining part is regarded as the central area. When calculating the structural similarity, the structural similarity index SSIM or other similar indicators can be used to measure the similarity between different areas. For the domain-level loss function, the central area structural loss, the edge area gradient loss, and the distorted area feature loss can be calculated separately. For example, the central area structural loss can be calculated by comparing the structural similarity of the central area; the edge area gradient loss can be obtained by calculating the gradient difference of the edge area; and the distorted area feature loss can be measured by comparing the feature vectors of the distorted area. During the training process, the contribution of the losses of different areas can be balanced by adjusting the weight coefficient, thereby optimizing the overall domain-level loss function. Through these specific steps and methods, domain-level adaptive learning can effectively improve the adaptability of the transformation network to different imaging devices and the accuracy of distortion correction.

[0049] In some embodiments, the domain-level overall loss function is expressed as: ;in, : domain-level overall loss function, : Domain-level loss of source imaging device, : domain-level loss of target imaging device, : structural loss in the central area, : Edge area gradient loss, : Distorted region feature loss, : weight coefficient, : edge adjacency matrix, : edge similarity matrix, : are the distortion feature embeddings of the source device and the target device respectively, : gradient operator, : Structural similarity index.

[0050] It should be noted that the domain-level overall loss function in this invention is an indicator used to evaluate the effectiveness of domain-level adaptive learning. It comprehensively considers the domain-level losses of the source and target imaging devices, as well as the structural loss, gradient loss, and feature loss in different regions. The domain-level overall loss function measures the performance of the transformation network by calculating these losses and guides the network training process to optimize network parameters and improve the accuracy of distortion correction. The design of this loss function enables the network to better adapt to the optical characteristics of different imaging devices, thereby improving the effectiveness of distortion correction.

[0051] Specifically, the terms involved in the domain-level overall loss function include the domain-level loss of the source imaging device, the domain-level loss of the target imaging device, the center region structure loss, the edge region gradient loss, and the distorted region feature loss. The domain-level loss of the source imaging device refers to the domain-level loss value calculated on the source imaging device, which reflects the performance of the source imaging device in domain-level adaptive learning. The domain-level loss of the target imaging device refers to the domain-level loss value calculated on the target imaging device, which reflects the performance of the target imaging device in domain-level adaptive learning. The center region structure loss is a loss function that measures the structural similarity of the center region. It is calculated by comparing the structural features of the center region. The edge region gradient loss is a loss function that measures the gradient difference of the edge region. It is calculated by calculating the gradient difference of the edge region. The distorted region feature loss is a loss function that measures the feature difference of the distorted region. It is measured by comparing the feature vectors of the distorted region. Weight coefficients are used to balance the contribution of different losses to the total loss. These coefficients can be adjusted according to actual application requirements.

[0052] Preferably, the construction and calculation process of the domain-level overall loss function can be further refined. First, the central region structural loss can be obtained by calculating the structural similarity index SSIM of the central region. SSIM is an indicator for measuring the structural similarity of images, which takes into account the three dimensions of brightness, contrast and structure. The edge region gradient loss can be obtained by calculating the gradient difference of the edge region. For example, a gradient operator can be used to calculate the gradient value of the edge region, and the loss value is obtained by comparing the gradient difference between the source imaging device and the target imaging device. The distorted region feature loss can be obtained by calculating the feature vector difference of the distorted region. For example, the Euclidean distance or other distance metrics can be used to calculate the difference between the feature vectors. The weight coefficient can be adjusted according to the actual application requirements. For example, if more attention is paid to the structural similarity of the central region, the weight coefficient of the central region structural loss can be increased. During the training process, by optimizing the domain-level overall loss function, the parameters of the transformation network can be adjusted so that it can better adapt to the optical characteristics of different imaging devices, thereby improving the accuracy of distortion correction.

[0053] In some embodiments, the pixel-level adaptive learning includes: calculating the structural similarity between the integrated optical embedding features of the source imaging device and the target imaging device at corresponding pixel positions; calculating the gradient magnitude difference between the integrated optical embedding features of the source imaging device and the target imaging device at corresponding pixel positions; and constructing a pixel-level loss function based on the structural similarity and the gradient magnitude difference: ;in, is the pixel-level feature vector of the source device, : pixel-level feature vector of the target device, : gradient operator, : Weight coefficient; Training the transformation network based on the pixel-level loss function.

[0054] It should be noted that pixel-level adaptive learning in this invention is a key step in the transformation network. It aims to construct a pixel-level loss function by calculating the structural similarity and gradient amplitude difference between the integrated optical embedding features of the source and target imaging devices at corresponding pixel locations. This process compares the feature vectors of the source and target devices at each pixel location and evaluates the similarities and differences between them, thereby optimizing network parameters and improving the accuracy of distortion correction. Pixel-level adaptive learning can process detailed information in the image, ensuring that the corrected image is as close to the quality of the source image as possible at the pixel level.

[0055] Specifically, the terms involved in pixel-level adaptive learning include the pixel-level feature vector of the source device, the pixel-level feature vector of the target device, structural similarity, gradient amplitude difference, and pixel-level loss function. The pixel-level feature vector of the source device refers to the feature vector of each pixel position extracted from the image of the source imaging device. These feature vectors contain the optical characteristics of the pixel. The pixel-level feature vector of the target device refers to the feature vector of each pixel position extracted from the image of the target imaging device. Structural similarity refers to the degree of structural similarity between two feature vectors, which is usually measured by the structural similarity index SSIM or other similar indicators. Gradient amplitude difference refers to the difference in gradient amplitude between two feature vectors, which is usually obtained by calculating the result of the gradient operator. The pixel-level loss function is constructed based on structural similarity and gradient amplitude difference. It is used to measure the difference between the source and target devices at the pixel level and guide the network training process.

[0056] Preferably, the implementation process of pixel-level adaptive learning can be further refined. First, the pixel-level feature vectors of the source device and the target device can be extracted through a multi-scale convolutional network. When calculating the structural similarity, the structural similarity index SSIM can be used to measure the structural similarity of the two feature vectors. SSIM takes into account the three dimensions of brightness, contrast and structure, and can comprehensively evaluate the similarity of the two feature vectors. When calculating the gradient amplitude difference, the Sobel operator or other gradient operators can be used to calculate the gradient amplitude of each pixel position, and then the difference is obtained by comparing the gradient amplitudes of the source device and the target device. The pixel-level loss function can combine the structural similarity and the gradient amplitude difference in a weighted manner. For example, the weight coefficient can be set to balance the contribution of these two factors in the loss function. During the training process, by optimizing the pixel-level loss function, the parameters of the transformation network can be adjusted so that it can better adapt to the optical characteristics of different imaging devices, thereby improving the accuracy of distortion correction.

[0057] In some embodiments, the neighborhood-level adaptive learning includes: obtaining a neighborhood feature vector by connecting embedded features of a neighboring pixel pair from a source imaging device and a target imaging device; and training the transformation network based on the neighborhood feature vector using a structural similarity loss function.

[0058] It should be noted that neighborhood-level adaptive learning in this invention is a key component of the transformation network, aiming to optimize spatial transformation learning by considering local neighborhood information between pixels. This process concatenates the embedded features of adjacent pixel pairs from the source and target imaging devices to obtain neighborhood feature vectors, and then trains the transformation network using a structural similarity loss function. Neighborhood-level adaptive learning can account for local relationships between pixels, ensuring that the corrected image maintains structural consistency within a local neighborhood, thereby improving the accuracy and robustness of distortion correction.

[0059] Specifically, the terms involved in neighborhood-level adaptive learning include neighborhood feature vectors and structural similarity loss functions. A neighborhood feature vector refers to a feature vector obtained by connecting the embedded features of a pair of adjacent pixels of the source imaging device and the target imaging device. These feature vectors contain the optical properties of the pixel and its neighborhood. The structural similarity loss function is constructed based on the structural similarity index SSIM, which is used to measure the similarity between neighborhood feature vectors. SSIM is an indicator for measuring the similarity of image structures, which takes into account three dimensions: brightness, contrast, and structure. In neighborhood-level adaptive learning, by calculating the SSIM value between neighborhood feature vectors, the similarity between the source device and the target device in the local neighborhood can be evaluated, and the parameters of the transformation network can be optimized accordingly.

[0060] Preferably, the implementation process of neighborhood-level adaptive learning can be further refined. First, the neighborhood information of each pixel can be extracted by means of a sliding window. For example, a 3×3 or 5×5 window can be used to extract the features of each pixel and its surrounding pixels. Then, the neighborhood feature vectors of the source device and the target device are connected to form a joint neighborhood feature vector. Next, the similarity between the joint neighborhood feature vectors is calculated using the structural similarity loss function. For example, the SSIM formula can be used to calculate the similarity of two neighborhood feature vectors in three dimensions: brightness, contrast, and structure, and these similarity values ​​are combined to obtain the final loss value. During the training process, by optimizing the structural similarity loss function, the parameters of the transformation network can be adjusted so that it can better adapt to the optical characteristics of different imaging devices, thereby improving the accuracy of distortion correction. Through this neighborhood-level optimization, it is possible to ensure that the corrected image maintains structural consistency within the local neighborhood, thereby improving the naturalness and accuracy of the correction effect.

[0061] In some embodiments, the weighted network loss function is: ;in, : mean square error, is the image distortion parameter predicted by the model, : The actual measured image distortion parameter, is the weight of the regularization term, is the optical characteristic similarity weight of the source imaging device, is the optical feature similarity weight of the target imaging device.

[0062] It should be noted that the weighted network loss function in this invention is a key tool for optimizing the weighted network. It comprehensively considers the difference between the image distortion parameters predicted by the model and the actual measured image distortion parameters, as well as the weighted similarity of the optical characteristics of the source and target imaging devices. By minimizing this loss function, the weighted network can be effectively trained, enabling it to learn the similarities and differences between different imaging devices, thereby providing more accurate weight distribution for distortion correction. This loss function design enables the network to better adapt to the optical characteristics of different imaging devices, improving the accuracy and robustness of distortion correction.

[0063] Specifically, the terms involved in the weighted network loss function include mean square error (MSE), model-predicted image distortion parameters, actually measured image distortion parameters, the weight of the regularization term, the optical feature similarity weight of the source imaging device, and the optical feature similarity weight of the target imaging device. Mean square error (MSE) is a commonly used error metric used to measure the difference between the model-predicted value and the actual value. The image distortion parameters predicted by the model refer to the distortion parameters predicted by the weighted network, which describe the characteristics of the image distortion. The actually measured image distortion parameters refer to the actual distortion parameters measured through experiments or other methods. The weight of the regularization term is used to prevent network overfitting. By adding a regularization term to the loss function, the complexity of the network parameters can be limited. The optical feature similarity weight of the source imaging device and the optical feature similarity weight of the target imaging device are parameters that measure the degree of similarity in optical features between the source and target devices. These weights are used to adjust the weight distribution between different devices.

[0064] Preferably, the construction and optimization process of the weighted network loss function can be further refined. First, the model-predicted image distortion parameters, including distortion coefficients and edge sharpness, can be obtained through forward propagation of the weighted network. The actual measured image distortion parameters can be obtained through experimental measurement or using a known distortion model. The mean squared error (MSE) can be calculated by calculating the squared difference between the model-predicted and actual values, reflecting the accuracy of the model's predictions. The weight of the regularization term can be adjusted based on the validation error during training to prevent network overfitting. The optical feature similarity weights of the source and target imaging devices can be calculated by comparing their optical feature matrices. For example, these weights can be obtained by calculating the cosine similarity between the feature matrices. During training, by minimizing the weighted network loss function, the network parameters can be adjusted to better learn the similarities and differences between different imaging devices, thereby improving the accuracy of distortion correction. Through this optimization process, the weighted network can provide a more accurate weight distribution for distortion correction, improving the accuracy and robustness of the correction effect.

[0065] The above-mentioned embodiments of the present invention have the following beneficial effects: 1. By utilizing the rich historical data of the source imaging device to train the distortion correction model, and combining it with the regional optical feature similarity weights of the target device, the problem of low correction accuracy of the target device due to insufficient data is solved, the distortion correction knowledge transfer across devices is realized, and the image correction effect of the target device is improved.

[0066] 2. A multi-level adaptive learning mechanism is adopted to perform refined spatial transformation optimization based on the optical characteristics of different regions, overcoming the lack of adaptability of traditional methods in complex distortion scenarios, and being able to more accurately handle the correction problems of central areas, edge areas, and distortion areas.

[0067] 3. By constructing an image quality evaluation index and conducting a quantitative evaluation based on the correction results, spatial transformation characteristics, and optical characteristics, the problem that traditional methods are difficult to objectively measure the correction effect is solved. This provides a reliable basis for the optimization and adjustment of distortion correction, ensuring that the quality of the corrected image meets the actual application requirements.

[0068] Furthermore, the storage medium of the embodiment of the present application stores program instructions that can implement all the above methods, wherein the program instructions can be stored in the above storage medium in the form of a software product, including a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or a terminal device such as a computer, server, mobile phone, or tablet.

[0069] The above descriptions merely illustrate some preferred embodiments of the present invention and the underlying technical principles. Those skilled in the art should understand that the scope of the invention encompassed by the embodiments of the present invention is not limited to technical solutions formed by specific combinations of the aforementioned technical features. It also encompasses other technical solutions formed by any combination of the aforementioned technical features or their equivalents, without departing from the aforementioned inventive concept. For example, a technical solution formed by replacing the aforementioned features with (but not limited to) technical features with similar functions disclosed in the embodiments of the present invention.

Claims

1. A method for correcting distorted images based on optical imaging, characterized in that: The steps include: Based on a training set including historical distorted image data and optical parameter data of a source imaging device and a target imaging device, a distortion correction model is trained to obtain optical features and optical feature similarity weights of regions of the source imaging device and the target imaging device; wherein the amount of historical data of the source imaging device is greater than that of the target imaging device to be corrected; An image correction model is obtained by training based on historical distorted image data of the source imaging device; Based on the optical feature similarity weights and historical distorted image data of the target imaging device, using the image correction model, obtaining correction image results and spatial transformation features of each area of ​​the target imaging device; Based on the corrected image results, spatial transformation features and optical features of each area, an image quality evaluation index of the corresponding area is obtained, and the image quality evaluation index of each area is summarized to obtain the distortion correction completion degree of the target imaging device.

2. The method for correcting distorted images based on optical imaging according to claim 1, wherein: The image quality evaluation index of the corresponding area is calculated based on the following formula: ;in, is the weight factor for correcting the image result, is the weight factor of the spatial transformation feature, is the weight factor of the optical feature, I is the corrected image result of the region, T is the spatial transformation feature of the region, and S is the optical feature of the region.

3. The method for correcting distorted images based on optical imaging according to claim 2, wherein: The distortion correction model includes a feature network and a transformation network; The distortion correction model is obtained by training, including: Based on the training set, optical features of the source imaging device and the target imaging device are extracted using a feature network; Performing region division based on the optical features, and using a transformation network to perform spatial transformation learning of each region to obtain comprehensive optical embedding features of each region of the source imaging device and the target imaging device; Based on the comprehensive optical embedding features of the source imaging device and the target imaging device, respectively, a weighted network and a weighted network loss function are used to learn region weights to obtain similarity weights of each region of the source imaging device and the target imaging device; After the training is completed, the comprehensive optical embedding features and similarity weights obtained based on the training set using the distortion correction model are the optical features and optical feature similarity weights of each region of the source imaging device and the target imaging device.

4. The method for correcting distorted images based on optical imaging according to claim 3, wherein: Extracting the optical features includes: constructing a position correlation matrix based on the pixel positions of the source imaging device and the target imaging device respectively; constructing optical point correlation matrices based on the optical point distributions of the source imaging device and the target imaging device respectively; constructing edge similarity matrices based on edge features of the source imaging device and the target imaging device respectively; Constructing a distortion type correlation matrix based on the distortion pattern data of the source imaging device and the target imaging device respectively; Based on the above-mentioned various matrices, the optical characteristics are obtained.

5. The method for correcting distorted images based on optical imaging according to claim 4, characterized in that: The transformation network sequentially utilizes multi-level adaptive learning based on structural similarity to perform spatial transformation learning of each region, including: Processing the position association matrix, the optical point association matrix, the edge similarity matrix, and the distortion type association matrix using a multi-scale convolutional network to obtain a feature embedding vector corresponding to each matrix; The feature embedding vectors of the source imaging device and the target imaging device are updated and fused respectively by using the self-attention mechanism of the feature fusion layer to obtain updated feature embeddings, and the comprehensive optical embedding features are outputted through affine transformation; The spatial transformation learning of each region is performed in sequence using domain-level adaptive learning, pixel-level adaptive learning and neighborhood-level adaptive learning based on structural similarity.

6. The method for correcting distorted images based on optical imaging according to claim 5, characterized in that: The domain-level adaptive learning includes: extracting embedding features of a central region, an edge region, and a distorted region in the updated feature embedding, and calculating structural similarities of the three types of embedded features; Calculating the loss function of the corresponding region type based on the structural similarity of the three types of embedded features; The transformation network is trained based on a domain-level overall loss function derived from the loss functions of various region types.

7. The method for correcting distorted images based on optical imaging according to claim 6, wherein: The domain-level overall loss function is expressed as: ;in, is the domain-level overall loss function, is the domain-level loss of the source imaging device, is the domain-level loss of the target imaging device, is the structural loss in the central area, is the edge area gradient loss, is the feature loss of the distorted region, is the weight coefficient, is the edge adjacency matrix, is the edge similarity matrix, are the distortion feature embeddings for the source device and the target device respectively, is the gradient operator, and SSIM is the structural similarity index.

8. The method for correcting distorted images based on optical imaging according to claim 5, wherein: The pixel-level adaptive learning includes: calculating the structural similarity between the integrated optical embedding features of the source imaging device and the target imaging device at corresponding pixel locations; calculating the gradient magnitude difference between the integrated optical embedding features of the source imaging device and the target imaging device at corresponding pixel locations; Based on the structural similarity and gradient magnitude difference, a pixel-level loss function is constructed: ;in, is the pixel-level feature vector of the source device, is the pixel-level feature vector of the target device, is the gradient operator, is the weight coefficient, SSIM is the structural similarity index; The transformation network is trained based on the pixel-level loss function.

9. The method for correcting distorted images based on optical imaging according to claim 5, wherein: The neighborhood-level adaptive learning includes: Obtaining a neighborhood feature vector by concatenating embedded features of a neighboring pixel pair from the source imaging device and the target imaging device; The transformation network is trained using a structural similarity loss function based on the neighborhood feature vectors.

10. The method for correcting distorted images based on optical imaging according to any one of claims 3 to 9, characterized in that: The weighted network loss function is: ; Where MSE is the mean square error, is the image distortion parameter predicted by the model, Y is the actual measured image distortion parameter, is the weight of the regularization term, is the optical characteristic similarity weight of the source imaging device, is the optical feature similarity weight of the target imaging device.

Images