Distortion image correction method based on optical imaging

Abstract

The invention provides a distortion image correction method based on optical imaging, and the method comprises the steps: training a distortion correction model through the historical data of a source imaging device, and obtaining the optical features and similarity weights of all regions of a source device and a target device; training an image correction model based on the historical data of the source device, and generating correction images and spatial transformation features of each region of the target device through the model in combination with the historical data and the similarity weight of the target device; and then calculating image quality evaluation indexes of each region according to the correction result, the transformation features and the optical features, and summarizing to obtain the distortion correction completion degree of the target equipment. According to the method, the distortion correction precision of the target equipment can be improved by effectively utilizing abundant data of the source equipment, and quantitative verification of the correction effect is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and more specifically, to a distortion image correction method based on optical imaging. Background Art

[0002] In the field of optical imaging, image distortion is a common problem. Distorted images will seriously affect the quality of images and subsequent application effects. For example, in the fields of medical imaging, satellite remote sensing, industrial inspection, etc., the existence of distortion may lead to problems such as misdiagnosis and inaccurate target recognition. Traditional distortion correction methods mainly rely on the geometric model and calibration process of the optical system, and correct images by calculating distortion parameters. Although these methods can improve the image quality to a certain extent, they often require a complex calibration process, and for complex distortion patterns, such as non-linear distortion, the correction effect is not ideal. In addition, traditional distortion correction methods are usually optimized for a single imaging device. When applied to different imaging devices, recalibration and parameter adjustment are required, which increases the complexity and cost of correction.

[0003] In the process of implementing the embodiments of the present invention, at least the following problems or defects exist in the prior art: The distortion correction method of the prior art relies on a complex calibration process, has limited correction effect for complex distortion patterns, and is difficult to adapt to the differences between different imaging devices, resulting in poor generality and adaptability of correction. Summary of the Invention

[0004] The present invention provides a distortion image correction method based on optical imaging, including: training a distortion correction model based on a training set including historical distortion image data and optical parameter data of a source imaging device and a target imaging device, and obtaining the optical characteristics and optical feature similarity weights of each region of the source imaging device and the target imaging device; wherein, the historical data volume of the source imaging device is more than that of the target imaging device to be corrected; training an image correction model based on the historical distortion image data of the source imaging device; based on the optical feature similarity weights and the historical distortion image data of the target imaging device, using the image correction model, obtaining the corrected image results and spatial transformation features of each region of the target imaging device; based on the corrected image results, spatial transformation features and optical characteristics of each region, obtaining the image quality evaluation index of the corresponding region, and summarizing the image quality evaluation indexes of each region to obtain the distortion correction completion degree of the target imaging device.

[0005] Further, calculate the image quality evaluation index of the corresponding region based on the following formula: ; wherein, : the weight factor of the corrected image result, : the weight factor of the spatial transformation feature, : Weight factor of optical features, : Calibrated image result of the region, : Spatial transformation features of the region, : Optical features of the region.

[0006] Further, the distortion correction model includes a feature network and a transformation network; training the distortion correction model includes: based on the training set, using the feature network to extract the optical features of the source imaging device and the target imaging device respectively; based on the optical features, performing region division, and using the transformation network to perform spatial transformation learning for each region to obtain the comprehensive optical embedding features of each region of the source imaging device and the target imaging device; respectively based on the comprehensive optical embedding features of the source imaging device and the target imaging device, using a weighted network and a weighted network loss function to learn the region weights to obtain the similarity weights of each region of the source imaging device and the target imaging device; after training is completed, the comprehensive optical embedding features and similarity weights obtained by using the distortion correction model based on the training set are the optical features and optical feature similarity weights of each region of the source imaging device and the target imaging device.

[0007] Further, extracting the optical features includes: respectively constructing a position correlation matrix based on the pixel positions of the source imaging device and the target imaging device; respectively constructing an optical point correlation matrix based on the optical point distributions of the source imaging device and the target imaging device; respectively constructing an edge similarity matrix based on the edge features of the source imaging device and the target imaging device; respectively constructing a distortion type correlation matrix based on the distortion mode data of the source imaging device and the target imaging device; based on the above multiple matrices, obtaining the optical features.

[0008] Further, the transformation network sequentially uses multi-level adaptive learning based on structural similarity to perform spatial transformation learning for each region, including: using a multi-scale convolutional network to process the position correlation matrix, the optical point correlation matrix, the edge similarity matrix, and the distortion type correlation matrix to obtain the feature embedding vectors corresponding to each matrix; using the self-attention mechanism of the feature fusion layer to update and fuse the feature embedding vectors of the source imaging device and the target imaging device respectively to obtain updated feature embeddings, and outputting comprehensive optical embedding features through affine transformation; using domain-level adaptive learning, pixel-level adaptive learning, and neighborhood-level adaptive learning based on structural similarity to sequentially perform spatial transformation learning for each region.

[0009] Furthermore, the domain-level adaptive learning includes: extracting the embedded features of the central region, edge region, and distortion region from the updated feature embedding, and calculating the structural similarity of the three types of embedded features; calculating the loss function for the corresponding region type based on the structural similarity of the three types of embedded features; and training the transformation network based on the domain-level overall loss function obtained from the loss functions of various region types.

[0010] Furthermore, the domain-level overall loss function is expressed as: ; where : domain-level overall loss function, : domain-level loss of the source imaging device, : domain-level loss of the target imaging device, : central region structure loss, : edge region gradient loss, : distortion region feature loss, : weight coefficient, : edge adjacency matrix, : edge similarity matrix, : distortion feature embeddings of the source device and the target device respectively, : gradient operator, : structural similarity index.

[0011] Furthermore, the pixel-level adaptive learning includes: calculating the structural similarity between the comprehensive optical embedded features of the source imaging device and the target imaging device at corresponding pixel positions; calculating the gradient magnitude difference between the comprehensive optical embedded features of the source imaging device and the target imaging device at corresponding pixel positions; and constructing a pixel-level loss function based on the structural similarity and the gradient magnitude difference: ; where is the pixel-level feature vector of the source device, : pixel-level feature vector of the target device, : gradient operator, : weight coefficient; training the transformation network based on the pixel-level loss function.

[0012] Furthermore, the neighborhood-level adaptive learning includes: obtaining a neighborhood feature vector by connecting the embedded features of an adjacent pixel pair from the source imaging device and the target imaging device; and training the transformation network using the structural similarity loss function based on the neighborhood feature vector.

[0013] Furthermore, the weighted network loss function is: ; where : mean square error, is the image distortion parameter predicted by the model, : actually measured image distortion parameter, is the weight of the regularization term, is the optical feature similarity weight of the source imaging device, is the optical feature similarity weight of the target imaging device.

[0014] According to the above embodiments of the present invention, it has at least the following beneficial effects: 1. By using the rich historical data of the source imaging device to train the distortion correction model and combining the regional optical feature similarity weight of the target device, the problem of low correction accuracy caused by insufficient data of the target device is solved, the cross-device distortion correction knowledge transfer is realized, and the image correction effect of the target device is improved.

[0015] 2. Adopting a multi-level adaptive learning mechanism to optimize the fine-grained spatial transformation for the optical features of different regions, overcomes the defect of insufficient adaptability of traditional methods in complex distortion scenarios, and can more accurately handle the correction problems of the central region, the edge region and the distortion region.

[0016] 3. By constructing an image quality evaluation index and comprehensively evaluating the correction result, the spatial transformation feature and the optical feature quantitatively, the problem that it is difficult for traditional methods to objectively measure the correction effect is solved, provides a reliable basis for the optimization and adjustment of distortion correction, and ensures that the quality of the corrected image meets the actual application requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner.

[0018] Figure 1 is a schematic flow chart of a distortion image correction method based on optical imaging provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are given only to enable those skilled in the art to better understand and then implement the present invention, and do not limit the scope of the present invention in any way. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to be able to fully convey the scope of the present invention to those skilled in the art.

[0020] Those skilled in the art know that the embodiments of the present invention can be implemented as a system, a device, an equipment, a method or a computer program product. Therefore, the present invention can be specifically implemented in the following forms, namely: completely hardware, completely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0021] It should be noted that the number of any elements in the accompanying drawings is for illustration rather than limitation, and any naming is only for distinction and does not have any limiting meaning.

[0022] The following refers to Figure 1 , Figure 1 , which is a schematic flowchart of a distortion image correction method based on optical imaging provided by an embodiment of the present invention. As Figure 1 shown, a distortion image correction method based on optical imaging includes: S1. Based on a training set including historical distortion image data and optical parameter data of a source imaging device and a target imaging device, a distortion correction model is trained, and the optical characteristics and optical characteristic similarity weights of each region of the source imaging device and the target imaging device are obtained; wherein, the amount of historical data of the source imaging device is more than that of the target imaging device to be corrected.

[0023] S2. An image correction model is trained based on the historical distortion image data of the source imaging device.

[0024] S3. Based on the optical characteristic similarity weights and the historical distortion image data of the target imaging device, using the image correction model, the corrected image results and spatial transformation characteristics of each region of the target imaging device are obtained.

[0025] S4. Based on the corrected image results, spatial transformation characteristics and the optical characteristics of each region, an image quality evaluation index for the corresponding region is obtained, and the image quality evaluation indexes of each region are summarized to obtain the distortion correction completion degree of the target imaging device.

[0026] It should be noted that the present invention proposes a distortion image correction method based on optical imaging. This method trains a distortion correction model by using the historical distortion image data of a source imaging device, and based on the optical characteristic similarity weights and the historical distortion image data of the target imaging device, uses this model to obtain the corrected image results and spatial transformation characteristics of each region of the target imaging device. Among them, the source imaging device refers to an imaging device with a large amount of historical distortion image data, and the target imaging device refers to an imaging device that needs to perform distortion correction. Optical characteristics refer to the optical-related characteristics exhibited by an imaging device during the imaging process, such as distortion patterns, edge features, etc., and these characteristics can reflect the imaging quality of the imaging device. The optical characteristic similarity weight is a parameter that measures the similarity degree of the optical characteristics between the source imaging device and the target imaging device. Through this weight, the correction experience of the source imaging device can be better applied to the target imaging device. Finally, by summarizing the image quality evaluation indexes of each region, the distortion correction completion degree of the target imaging device is obtained, thereby effectively correcting the distortion image of the target imaging device.

[0027] Specifically, the distortion correction model is the core part of the present invention, which includes a feature network and a transformation network. The role of the feature network is to extract optical features from the distorted image data of the source imaging device and the target imaging device. These features include a position association matrix constructed from pixel positions, an optical point association matrix constructed from the distribution of optical points, an edge similarity matrix constructed from edge features, and a distortion type association matrix constructed from distortion pattern data. The transformation network is responsible for dividing regions based on the extracted optical features and learning the spatial transformation features of each region to achieve the correction of distorted images. During the training process, a weighted network and a weighted network loss function are used to learn the region weights, obtaining the similarity weights of each region of the source imaging device and the target imaging device. These weights reflect the importance of different regions in terms of optical features and play a key role in subsequent image correction. The image quality evaluation index is an important indicator for measuring the quality of the corrected image. It comprehensively considers multiple factors such as the corrected image result, spatial transformation features, and optical features, and is calculated through a specific formula. The weight factors in this formula can be adjusted according to actual needs to adapt to different correction objectives and application scenarios.

[0028] Preferably, the construction process of the distortion correction model can be further refined. First, when constructing the feature network, various optical feature extraction methods need to be considered. For example, the position association matrix can be obtained by calculating the position relationship between each pixel point in the image and its surrounding pixel points. The optical point association matrix can be constructed by analyzing the distribution of optical points in the image. The edge similarity matrix is calculated based on the edge features of the image, and the distortion type association matrix is constructed according to the distortion pattern data. The construction methods of these matrices can be optimized and adjusted according to specific imaging devices and application scenarios. In the transformation network, multi-level adaptive learning is the key step to achieve the learning of spatial transformation features. It includes domain-level adaptive learning based on structural similarity, pixel-level adaptive learning, and neighborhood-level adaptive learning. Domain-level adaptive learning focuses on the overall structural features of the image, pixel-level adaptive learning focuses on the detailed features of the image, and neighborhood-level adaptive learning considers the local relationship between pixels. Through these three levels of adaptive learning, the spatial transformation features of the image can be learned more comprehensively, thereby improving the accuracy and effect of distortion correction. In the calculation of the image quality evaluation index, the setting of the weight factors can be adjusted according to actual needs. For example, if more attention is paid to the visual effect of the corrected image, the weight factor of the corrected image result can be appropriately increased; if more attention is paid to the structural features of the image, the weight factor of the spatial transformation features can be increased. By reasonably setting these weight factors, the image quality evaluation index can more accurately reflect the quality of the corrected image.

[0029] In some embodiments, the image quality evaluation index of the corresponding region is calculated based on the following formula: ; where : The weight factor for the corrected image result, : The weight factor for the spatial transformation feature, : The weight factor for the optical feature, : The corrected image result of the region, : The spatial transformation feature of the region, : The optical feature of the region.

[0030] It should be noted that in the present invention, the method for calculating the image quality evaluation index is carried out by comprehensively considering three factors: the corrected image result, the spatial transformation feature, and the optical feature, and assigning different weight factors to them respectively. This method allows for flexible adjustment of the weights of each factor according to different correction objectives and application scenarios, so as to obtain a comprehensive and targeted image quality evaluation index. The weight factor here reflects the degree of importance of each feature in the image quality evaluation, while the corrected image result, the spatial transformation feature, and the optical feature are indicators for evaluating the corrected image from different perspectives, and they jointly constitute the basis for image quality evaluation.

[0031] Specifically, the weight factor is a key parameter used to balance the contribution degrees of different features in the image quality evaluation. For example, if the corrected visual effect is the main concern, then the weight factor for the corrected image result should be set relatively high; if the fidelity of the spatial structure is more important, then the weight factor for the spatial transformation feature should be increased accordingly; if the optical characteristics have a significant impact on the image quality, then the weight factor for the optical feature should also be appropriately increased. The corrected image result can be quantified by comparing the differences between the images before and after correction, such as using indicators like image sharpness and contrast. The spatial transformation feature can be evaluated by analyzing the parameter changes during the spatial transformation of the image. For example, in an affine transformation, the parameter changes of the transformation matrix can be calculated to measure it. The optical feature can be determined by analyzing the optical characteristics of the imaging device, such as the degree of distortion and edge sharpness. The specific evaluation methods of these features and the setting of the weight factor can be adjusted according to the characteristics of the imaging device and the correction objective to ensure that the image quality evaluation index can accurately reflect the correction effect.

[0032] Preferably, the calculation process of the image quality evaluation index can be further refined. In practical applications, the setting of the weight factor can be determined through experiments and experience. For example, the optimal weight combination can be selected by comparing the correction effects under different weight settings. For the corrected image results, multiple image quality evaluation metrics can be used for comprehensive evaluation, such as the structural similarity index and visual information fidelity, etc., to more comprehensively reflect the quality of the corrected image. The calculation of the spatial transformation features can be combined with specific transformation models. For example, in affine transformation, the spatial transformation features can be evaluated by calculating the parameter changes of the transformation matrix. The calculation of the optical features can be based on the optical model of the imaging device. For example, the optical features can be quantified by measuring parameters such as the distortion coefficient and edge sharpness. Through these specific calculation methods and parameter settings, the image quality evaluation index can be calculated more accurately, thereby providing a reliable basis for the evaluation of the distortion correction effect.

[0033] In some embodiments, the distortion correction model includes a feature network and a transformation network; training the distortion correction model includes: based on the training set, using the feature network to extract the optical features of the source imaging device and the target imaging device respectively; based on the optical features, performing region division, and using the transformation network to perform spatial transformation learning on each region to obtain the comprehensive optical embedding features of each region of the source imaging device and the target imaging device; respectively based on the comprehensive optical embedding features of the source imaging device and the target imaging device, using the weighted network and the weighted network loss function to learn the region weights to obtain the similarity weights of each region of the source imaging device and the target imaging device; after the training is completed, the comprehensive optical embedding features and similarity weights obtained by using the distortion correction model based on the training set are the optical features and optical feature similarity weights of each region of the source imaging device and the target imaging device.

[0034] It should be noted that the distortion correction model of the present invention is composed of a feature network and a transformation network. The role of the feature network is to extract optical features from the distorted image data of the source imaging device and the target imaging device, and these features can reflect the imaging characteristics of the imaging device. The optical features include a position association matrix, an optical point association matrix, an edge similarity matrix, a distortion type association matrix, etc., and these matrices can describe the optical characteristics of the image from different angles. The transformation network is responsible for performing region division according to the extracted optical features and learning the spatial transformation features of each region to achieve the correction of the distorted image. The weighted network and the weighted network loss function are used to learn the region weights to obtain the similarity weights of each region of the source imaging device and the target imaging device, and these weights can reflect the similarity degree of the optical features in different regions, thereby providing a basis for distortion correction.

[0035] Specifically, the feature network and the transformation network are the core components of the distortion correction model. The feature network extracts optical features by constructing multiple matrices, including a position correlation matrix, an optical point correlation matrix, an edge similarity matrix, and a distortion type correlation matrix. The position correlation matrix is constructed based on pixel positions and is used to reflect the spatial distribution relationship of pixels in the image; the optical point correlation matrix is constructed based on the distribution of optical points and is used to describe the distribution of optical points in the image; the edge similarity matrix is constructed based on edge features and is used to measure the similarity of image edges; the distortion type correlation matrix is constructed based on distortion pattern data and is used to describe the distortion type of the image. The transformation network then uses these optical features for region division and learns the spatial transformation features of each region. The weighted network and the weighted network loss function are used to learn region weights. By comparing the optical features of the source imaging device and the target imaging device, the similarity weights of each region are obtained, and these weights can reflect the similarity degree of different regions in terms of optical features, providing an important reference basis for distortion correction.

[0036] Preferably, the construction process of the distortion correction model can be further refined. The construction of the feature network needs to consider various optical feature extraction methods. For example, the position correlation matrix can be obtained by calculating the position relationship between each pixel point in the image and its surrounding pixel points; the optical point correlation matrix can be constructed by analyzing the distribution of optical points in the image; the edge similarity matrix is calculated based on the edge features of the image; the distortion type correlation matrix is constructed according to the distortion pattern data. The construction methods of these matrices can be optimized and adjusted according to specific imaging devices and application scenarios. In the transformation network, multi-level adaptive learning is the key step to achieve spatial transformation feature learning. It includes domain-level adaptive learning based on structural similarity, pixel-level adaptive learning, and neighborhood-level adaptive learning. Domain-level adaptive learning focuses on the overall structural features of the image, pixel-level adaptive learning focuses on the detailed features of the image, and neighborhood-level adaptive learning considers the local relationship between pixels. Through these three levels of adaptive learning, the spatial transformation features of the image can be learned more comprehensively, thereby improving the accuracy and effect of distortion correction. The construction of the weighted network and the weighted network loss function needs to consider how to effectively learn region weights to reflect the similarity degree of different regions in terms of optical features. By reasonably setting these networks and loss functions, the distortion correction model can better adapt to different imaging devices and distortion situations and improve the correction effect.

[0037] In some embodiments, extracting the optical features includes: constructing a position association matrix based on the pixel positions of the source imaging device and the target imaging device respectively; constructing an optical point association matrix based on the optical point distributions of the source imaging device and the target imaging device respectively; constructing an edge similarity matrix based on the edge features of the source imaging device and the target imaging device respectively; constructing a distortion type association matrix based on the distortion mode data of the source imaging device and the target imaging device respectively; and obtaining the optical features based on the above-mentioned multiple matrices.

[0038] It should be noted that in the present invention, the process of extracting optical features is achieved by constructing multiple matrices, including a position association matrix, an optical point association matrix, an edge similarity matrix, and a distortion type association matrix. These matrices describe the optical characteristics of the imaging device from different perspectives and can provide important reference information for subsequent distortion correction. The position association matrix reflects the spatial distribution relationship of pixels in the image. The optical point association matrix describes the distribution of optical points. The edge similarity matrix measures the similarity of image edges. The distortion type association matrix describes the distortion type of the image. Through the comprehensive analysis of these matrices, comprehensive optical features can be obtained, so as to perform distortion correction more accurately.

[0039] Specifically, the position association matrix is constructed based on the pixel positions of the source imaging device and the target imaging device, and it reflects the spatial position relationship between each pixel point in the image and its surrounding pixel points. The optical point association matrix is constructed based on the optical point distribution. Optical points usually refer to points in the image with specific optical characteristics, such as high-brightness points or points with specific shapes. The distribution of these points can reflect the optical characteristics of the imaging device. The edge similarity matrix is constructed based on the edge features. Edge features refer to regions in the image where the brightness or color changes significantly, and these regions usually contain rich image structure information. The distortion type association matrix is constructed based on the distortion mode data. Distortion mode data refers to data describing the distortion characteristics of the image, such as radial distortion, tangential distortion, etc. The construction methods of these matrices can be adjusted according to specific imaging devices and application scenarios to ensure that the optical features reflecting the optical characteristics of the imaging device can be accurately extracted. When constructing these matrices, it is necessary to consider how to select appropriate parameters and algorithms. For example, when calculating the pixel position association, different distance measurement methods can be selected; when analyzing the optical point distribution, different thresholds can be set to identify optical points; when extracting edge features, different edge detection algorithms can be used; when describing the distortion mode, different mathematical models can be used to fit the distortion characteristics.

[0040] Preferably, the process of extracting optical features can be further refined. For example, when constructing the position correlation matrix, the spatial relationship between pixels can be quantified by calculating the Euclidean distance or other distance metrics between each pixel point in the image and its surrounding pixel points. For the optical point correlation matrix, optical points in the image can be identified by setting a brightness threshold or a specific shape matching algorithm, and the correlation degree between these points can be calculated. When constructing the edge similarity matrix, the Canny edge detection algorithm or other advanced edge detection methods can be used to extract the image edges, and the edge similarity can be calculated by comparing the directions and intensities of the edges in different regions. For the distortion type correlation matrix, according to the type of distortion, such as radial distortion, tangential distortion, etc., a suitable mathematical model can be selected to describe the distortion characteristics, and the correlation degree between distortion types can be calculated by fitting these models. In practical applications, the construction and analysis processes of these matrices can be implemented by writing corresponding algorithms. The input parameters of the algorithms include image data, distortion mode data, etc., and the output is the constructed matrix. These matrices can be used as feature vectors and input into the subsequent distortion correction model, providing rich optical feature information for the model, thereby improving the accuracy and effectiveness of distortion correction.

[0041] In some embodiments, the transformation network sequentially uses multi-level adaptive learning based on structural similarity to perform spatial transformation learning for each region, including: processing the position correlation matrix, optical point correlation matrix, edge similarity matrix, and distortion type correlation matrix by using a multi-scale convolutional network to obtain the feature embedding vectors corresponding to each matrix; using the self-attention mechanism of the feature fusion layer to update and fuse the feature embedding vectors of the source imaging device and the target imaging device respectively to obtain updated feature embeddings, and outputting comprehensive optical embedding features through affine transformation; using domain-level adaptive learning, pixel-level adaptive learning, and neighborhood-level adaptive learning based on structural similarity to perform spatial transformation learning for each region in sequence.

[0042] It should be noted that the implementation process of the transformation network in the present invention is completed through multi-level adaptive learning based on structural similarity. This process includes using a multi-scale convolutional network to process various correlation matrices, such as the position correlation matrix, optical point correlation matrix, edge similarity matrix, and distortion type correlation matrix, to extract the feature embedding vectors corresponding to each matrix. Subsequently, the feature embedding vectors are updated and fused through the self-attention mechanism of the feature fusion layer, and finally comprehensive optical embedding features are output. This process also involves domain-level, pixel-level, and neighborhood-level adaptive learning, which optimize different hierarchical features of the image respectively to achieve more accurate spatial transformation learning. This multi-level adaptive learning method can effectively handle complex distortions in the image and improve the accuracy and robustness of correction.

[0043] Specifically, the multi-scale convolutional network in the multi-level adaptive learning process is a deep learning architecture used to process image data and extract features. It captures local and global features in the image through convolutional kernels of different scales. The position association matrix, optical point association matrix, edge similarity matrix, and distortion type association matrix respectively describe the optical characteristics of the image from different perspectives. The multi-scale convolutional network processes these matrices to generate feature embedding vectors corresponding to each matrix, and these vectors contain the structural and distortion information of the image. The self-attention mechanism of the feature fusion layer is used to update and fuse these feature embedding vectors, enhancing important features and suppressing unimportant features by calculating the similarity between feature vectors, thereby obtaining comprehensive optical embedding features. Domain-level adaptive learning focuses on the overall structural features of the image, pixel-level adaptive learning focuses on the detailed features of the image, and neighborhood-level adaptive learning focuses on the local relationships between pixels. These different levels of learning processes work together to achieve a comprehensive optimization of the image spatial transformation.

[0044] Preferably, the construction and training process of the transformation network can be further refined. In the multi-scale convolutional network, convolutional kernels of different sizes can be set to extract multi-scale features of the image. For example, 3×3, 5×5, and 7×7 convolutional kernels can be used to capture the fine-grained, medium-grained, and coarse-grained features of the image respectively. In the feature fusion layer, the self-attention mechanism can update and fuse the feature embedding vectors by calculating the dot product similarity between feature vectors. For domain-level adaptive learning, the features of the central region, edge region, and distortion region of the image can be extracted, and the structural similarity between these regions can be calculated. Pixel-level adaptive learning can optimize the transformation parameters by calculating the differences between the feature vectors at each pixel position of the source image and the target image. Neighborhood-level adaptive learning can further refine the spatial transformation by considering the local neighborhood information of the pixels. During the training process, the structural similarity loss function can be used to optimize the network parameters to ensure that the network can learn the optimal spatial transformation features. Through these specific construction and training steps, the transformation network can more effectively process image distortion and improve the correction effect.

[0045] In some embodiments, the domain-level adaptive learning includes: extracting the embedding features of the central region, edge region, and distortion region in the updated feature embedding, and calculating the structural similarity of the three types of embedding features; calculating the loss function corresponding to the region type based on the structural similarity of the three types of embedding features; training the transformation network based on the domain-level overall loss function obtained from the loss functions of various region types.

[0046] It should be noted that domain-level adaptive learning in the present invention is an important part of the transformation network. The main purpose is to optimize the network parameters by extracting the embedding features of the central region, edge region, and distortion region of the image and calculating the structural similarity of these regions. The domain-level overall loss function is a key indicator for measuring the effect of domain-level adaptive learning. It comprehensively considers the domain-level losses of the source imaging device and the target imaging device, as well as the structural loss, gradient loss, and feature loss of different regions. By optimizing this loss function, the transformation network can be effectively trained to better adapt to the optical characteristics of different imaging devices and improve the accuracy of distortion correction.

[0047] Specifically, the terms involved in domain-level adaptive learning include the central region, edge region, distortion region, structural similarity, domain-level loss function, etc. The central region refers to the relatively stable region in the image that is less affected by distortion; the edge region refers to the boundary part of the image, which usually contains rich structural information; the distortion region refers to the part of the image with obvious distortion. Structural similarity is an index for measuring the similarity degree between image regions, usually calculated based on three dimensions: brightness, contrast, and structure. The domain-level loss function is a function that comprehensively considers the feature losses of different regions. It includes the central region structure loss, edge region gradient loss, and distortion region feature loss. These loss functions are calculated by comparing the feature embeddings of the source imaging device and the target imaging device, and the weight coefficients are used to balance the contributions of different losses to the total loss. The domain-level overall loss function evaluates the performance of the transformation network by integrating these losses and guides the training process of the network.

[0048] Preferably, the implementation process of domain-level adaptive learning can be further refined. First, the central region, edge region, and distortion region can be determined through an image segmentation algorithm. For example, an edge detection algorithm can be used to identify the edge region, a distortion detection algorithm can be used to locate the distortion region, and the remaining part is regarded as the central region. When calculating the structural similarity, the structural similarity index SSIM or other similar indicators can be used to measure the similarity between different regions. For the domain-level loss function, the central region structure loss, edge region gradient loss, and distortion region feature loss can be calculated separately. For example, the central region structure loss can be calculated by comparing the structural similarity of the central region; the edge region gradient loss can be obtained by calculating the gradient difference of the edge region; the distortion region feature loss can be measured by comparing the feature vectors of the distortion region. During the training process, the contribution of different region losses can be balanced by adjusting the weight coefficients, thereby optimizing the domain-level overall loss function. Through these specific steps and methods, domain-level adaptive learning can effectively improve the adaptability of the transformation network to different imaging devices and the accuracy of distortion correction.

[0049] In some embodiments, the domain-level overall loss function is expressed as: ; where : domain-level overall loss function : domain-level loss of the source imaging device : domain-level loss of the target imaging device : central region structure loss : edge region gradient loss : distortion region feature loss : weight coefficient : edge adjacency matrix : edge similarity matrix : are the distortion feature embeddings of the source device and the target device respectively : gradient operator : structural similarity index

[0050] It should be noted that the domain-level overall loss function in the present invention is an index for evaluating the effect of domain-level adaptive learning, which comprehensively considers the domain-level losses of the source imaging device and the target imaging device, as well as the structure losses, gradient losses and feature losses in different regions. The domain-level overall loss function measures the performance of the transformation network by calculating these losses, and guides the training process of the network to optimize the network parameters and improve the accuracy of distortion correction. The design of this loss function enables the network to better adapt to the optical characteristics of different imaging devices, thereby improving the effect of distortion correction.

[0051] Specifically, the terms involved in the domain-level overall loss function include the domain-level loss of the source imaging device, the domain-level loss of the target imaging device, the central region structure loss, the edge region gradient loss and the distortion region feature loss, etc. The domain-level loss of the source imaging device refers to the domain-level loss value calculated on the source imaging device, which reflects the performance of the source imaging device in domain-level adaptive learning. The domain-level loss of the target imaging device refers to the domain-level loss value calculated on the target imaging device, which reflects the performance of the target imaging device in domain-level adaptive learning. The central region structure loss is a loss function for measuring the structural similarity of the central region, which is calculated by comparing the structural features of the central region. The edge region gradient loss is a loss function for measuring the gradient difference of the edge region, which is obtained by calculating the gradient difference of the edge region. The distortion region feature loss is a loss function for measuring the feature difference of the distortion region, which is measured by comparing the feature vectors of the distortion region. The weight coefficient is used to balance the contributions of different losses to the total loss, and these coefficients can be adjusted according to actual application requirements.

[0052] Preferably, the construction and calculation process of the domain-level overall loss function can be further refined. First, the central region structure loss can be obtained by calculating the structural similarity index (SSIM) of the central region. SSIM is an index for measuring the structural similarity of images, which takes into account three dimensions: brightness, contrast, and structure. The edge region gradient loss can be obtained by calculating the gradient difference of the edge region. For example, a gradient operator can be used to calculate the gradient values of the edge region, and the loss value can be obtained by comparing the gradient differences between the source imaging device and the target imaging device. The distortion region feature loss can be obtained by calculating the difference between the feature vectors of the distortion region. For example, the Euclidean distance or other distance metrics can be used to calculate the difference between the feature vectors. The weight coefficient can be adjusted according to the actual application requirements. For example, if more attention is paid to the structural similarity of the central region, the weight coefficient of the central region structure loss can be increased. During the training process, by optimizing the domain-level overall loss function, the parameters of the transformation network can be adjusted to better adapt to the optical characteristics of different imaging devices, thereby improving the accuracy of distortion correction.

[0053] In some embodiments, the pixel-level adaptive learning includes: calculating the structural similarity between the comprehensive optical embedding features of the source imaging device and the target imaging device at corresponding pixel positions; calculating the gradient magnitude difference between the comprehensive optical embedding features of the source imaging device and the target imaging device at corresponding pixel positions; constructing a pixel-level loss function based on the structural similarity and the gradient magnitude difference: ; where is the pixel-level feature vector of the source device, : the pixel-level feature vector of the target device, : the gradient operator, : the weight coefficient; training the transformation network based on the pixel-level loss function.

[0054] It should be noted that the pixel-level adaptive learning in the present invention is a key link in the transformation network, aiming to construct a pixel-level loss function by calculating the structural similarity and the gradient magnitude difference between the comprehensive optical embedding features of the source imaging device and the target imaging device at corresponding pixel positions. This process evaluates the similarity and difference between the source device and the target device by comparing the feature vectors at each pixel position, thereby optimizing the network parameters and improving the accuracy of distortion correction. Pixel-level adaptive learning can process the detailed information in the image to ensure that the corrected image is as close as possible to the quality of the source image at the pixel level.

[0055] Specifically, the terms involved in pixel-level adaptive learning include the pixel-level feature vector of the source device, the pixel-level feature vector of the target device, structural similarity, gradient magnitude difference, and pixel-level loss function. The pixel-level feature vector of the source device refers to the feature vector at each pixel position extracted from the image of the source imaging device, and these feature vectors contain the optical characteristics of the pixels. The pixel-level feature vector of the target device refers to the feature vector at each pixel position extracted from the image of the target imaging device. Structural similarity refers to the degree of similarity in structure between two feature vectors, usually measured by the structural similarity index SSIM or other similar metrics. Gradient magnitude difference refers to the difference in gradient magnitude between two feature vectors, usually obtained by calculating the result of the gradient operator. The pixel-level loss function is constructed based on structural similarity and gradient magnitude difference, used to measure the difference between the source device and the target device at the pixel level, and guide the training process of the network.

[0056] Preferably, the implementation process of pixel-level adaptive learning can be further refined. First, the pixel-level feature vectors of the source device and the target device can be extracted through a multi-scale convolutional network. When calculating structural similarity, the structural similarity index SSIM can be used to measure the structural similarity between two feature vectors. SSIM considers three dimensions: brightness, contrast, and structure, and can comprehensively evaluate the similarity between two feature vectors. When calculating the gradient magnitude difference, the Sobel operator or other gradient operators can be used to calculate the gradient magnitude at each pixel position, and then the difference can be obtained by comparing the gradient magnitudes of the source device and the target device. The pixel-level loss function can combine structural similarity and gradient magnitude difference in a weighted manner. For example, weight coefficients can be set to balance the contributions of these two factors in the loss function. During the training process, by optimizing the pixel-level loss function, the parameters of the transformation network can be adjusted to better adapt to the optical characteristics of different imaging devices, thereby improving the accuracy of distortion correction.

[0057] In some embodiments, the neighborhood-level adaptive learning includes: obtaining a neighborhood feature vector by connecting the embedded features of an adjacent pixel pair from the source imaging device and the target imaging device; and training the transformation network based on the neighborhood feature vector using a structural similarity loss function.

[0058] It should be noted that the neighborhood-level adaptive learning in the present invention is an important part of the transformation network, aiming to optimize spatial transformation learning by considering the local neighborhood information between pixels. This process obtains a neighborhood feature vector by connecting the embedded features of adjacent pixel pairs from the source imaging device and the target imaging device, and uses a structural similarity loss function to train the transformation network. Neighborhood-level adaptive learning can handle the local relationships between pixels, ensuring that the corrected image maintains structural consistency within the local neighborhood, thereby improving the accuracy and robustness of distortion correction.

[0059] Specifically, the terms involved in neighborhood-level adaptive learning include neighborhood feature vectors and structural similarity loss functions. A neighborhood feature vector refers to a feature vector obtained by embedding features of an adjacent pixel pair connecting a source imaging device and a target imaging device, and these feature vectors contain the optical characteristics of the pixel and its neighborhood. The structural similarity loss function is constructed based on the structural similarity index SSIM and is used to measure the similarity between neighborhood feature vectors. SSIM is an index for measuring the structural similarity of images, which takes into account three dimensions: brightness, contrast, and structure. In neighborhood-level adaptive learning, by calculating the SSIM value between neighborhood feature vectors, the similarity between the source device and the target device in the local neighborhood can be evaluated, and the parameters of the transformation network can be optimized accordingly.

[0060] Preferably, the implementation process of neighborhood-level adaptive learning can be further refined. First, the neighborhood information of each pixel can be extracted by means of a sliding window. For example, a 3×3 or 5×5 window can be used to extract the features of each pixel and its surrounding pixels. Then, the neighborhood feature vectors of the source device and the target device are concatenated to form a joint neighborhood feature vector. Next, the structural similarity loss function is used to calculate the similarity between the joint neighborhood feature vectors. For example, the SSIM formula can be used to calculate the similarity between two neighborhood feature vectors in the three dimensions of brightness, contrast, and structure, and these similarity values are combined to obtain the final loss value. During the training process, by optimizing the structural similarity loss function, the parameters of the transformation network can be adjusted to better adapt to the optical characteristics of different imaging devices, thereby improving the accuracy of distortion correction. Through this neighborhood-level optimization, it can be ensured that the corrected image maintains structural consistency in the local neighborhood, improving the naturalness and accuracy of the correction effect.

[0061] In some embodiments, the weighted network loss function is: ; where : mean square error, is the image distortion parameter predicted by the model, : the actually measured image distortion parameter, is the weight of the regularization term, is the optical feature similarity weight of the source imaging device, is the optical feature similarity weight of the target imaging device.

[0062] It should be noted that the weighted network loss function in the present invention is a key tool for optimizing the weighted network. It comprehensively considers the difference between the image distortion parameters predicted by the model and the actually measured image distortion parameters, as well as the optical feature similarity weights of the source imaging device and the target imaging device. By minimizing this loss function, the weighted network can be effectively trained to learn the similarities and differences between different imaging devices, so as to provide more accurate weight allocation for distortion correction. The design of this loss function enables the network to better adapt to the optical characteristics of different imaging devices, improving the accuracy and robustness of distortion correction.

[0063] Specifically, the terms involved in the weighted network loss function include mean square error MSE, the image distortion parameters predicted by the model, the actually measured image distortion parameters, the weight of the regularization term, the optical feature similarity weight of the source imaging device, and the optical feature similarity weight of the target imaging device. Mean square error MSE is a commonly used error measurement method for measuring the difference between the model prediction value and the actual value. The image distortion parameters predicted by the model refer to the distortion parameters predicted by the weighted network, and these parameters describe the characteristics of image distortion. The actually measured image distortion parameters refer to the true distortion parameters obtained through experiments or other methods. The weight of the regularization term is used to prevent the network from overfitting. By adding a regularization term to the loss function, the complexity of the network parameters can be restricted. The optical feature similarity weights of the source imaging device and the target imaging device are parameters that measure the similarity degree of the optical characteristics between the source device and the target device, and these weights are used to adjust the weight allocation between different devices.

[0064] Preferably, the construction and optimization process of the weighted network loss function can be further refined. First, the image distortion parameters predicted by the model can be obtained through the forward propagation of the weighted network, and these parameters include distortion coefficients, edge sharpness, etc. The actually measured image distortion parameters can be obtained through experimental measurement or using a known distortion model. Mean square error MSE can be obtained by calculating the square difference between the model prediction value and the actual value, which reflects the accuracy of the model prediction. The weight of the regularization term can be adjusted according to the validation error during the training process to prevent the network from overfitting. The optical feature similarity weights of the source imaging device and the target imaging device can be calculated by comparing their optical feature matrices. For example, these weights can be obtained by calculating the cosine similarity between the feature matrices. During the training process, by minimizing the weighted network loss function, the parameters of the network can be adjusted to enable it to better learn the similarities and differences between different imaging devices, thereby improving the accuracy of distortion correction. Through this optimization process, the weighted network can provide more accurate weight allocation for distortion correction, improving the accuracy and robustness of the correction effect.

[0065] The above embodiments of the present invention have the following beneficial effects: 1. By using the rich historical data of the source imaging device to train the distortion correction model and combining the regional optical feature similarity weights of the target device, the problem of low correction accuracy caused by insufficient data of the target device is solved, cross-device distortion correction knowledge transfer is realized, and the image correction effect of the target device is improved.

[0066] 2. Adopting a multi-level adaptive learning mechanism to perform refined spatial transformation optimization for the optical features of different regions, overcomes the defect of insufficient adaptability of traditional methods in complex distortion scenarios, and can more accurately handle the correction problems of the central region, the edge region, and the distortion region.

[0067] 3. By constructing an image quality evaluation index and comprehensively quantifying and evaluating the correction result, spatial transformation features, and optical features, the problem that traditional methods are difficult to objectively measure the correction effect is solved, providing a reliable basis for the optimization and adjustment of distortion correction, and ensuring that the quality of the corrected image meets the actual application requirements.

[0068] Furthermore, the storage medium of the implementation manner of the present application stores program instructions capable of implementing all the above methods. Among them, the program instructions can be stored in the above storage medium in the form of a software product, including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in the various implementation manners of the present application. And the aforementioned storage medium includes: various media that can store program codes such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs, or terminal devices such as computers, servers, mobile phones, and tablets.

[0069] The above description is only some preferred implementation manners of the present invention and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in the implementation manners of the present invention is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the technical solutions formed by mutually replacing the above features with the (but not limited to) technical features with similar functions disclosed in the implementation manners of the present invention.

Claims

1. A distortion image correction method based on optical imaging, characterized in that, The method includes the following steps: Based on a training set including historical distortion image data and optical parameter data of a source imaging device and a target imaging device, a distortion correction model is trained, and optical features and optical feature similarity weights of each region of the source imaging device and the target imaging device are obtained; wherein, the amount of historical data of the source imaging device is more than that of the target imaging device to be corrected; An image correction model is trained based on the historical distortion image data of the source imaging device; Based on the optical feature similarity weights and the historical distortion image data of the target imaging device, using the image correction model, the corrected image results and spatial transformation features of each region of the target imaging device are obtained; Based on the corrected image results, spatial transformation features and the optical features of each region, an image quality evaluation index of the corresponding region is obtained, and the image quality evaluation indexes of each region are aggregated to obtain the distortion correction completion degree of the target imaging device.

2. The distortion image correction method based on optical imaging according to claim 1, wherein Calculate the image quality evaluation index of the corresponding region according to the following formula: ; where is the weight factor of the corrected image result, is the weight factor of the spatial transformation feature, is the weight factor of the optical feature, I is the corrected image result of the region, T is the spatial transformation feature of the region, and S is the optical feature of the region.

3. The distortion image correction method based on optical imaging according to claim 2, characterized in that The distortion correction model includes a feature network and a transformation network; Training the distortion correction model includes: Based on the training set, using the feature network to extract the optical features of the source imaging device and the target imaging device respectively; Based on the optical features, region division is performed, and the transformation network is used to perform spatial transformation learning on each region to obtain the comprehensive optical embedding features of each region of the source imaging device and the target imaging device; Based on the comprehensive optical embedding features of the source imaging device and the target imaging device respectively, using a weighted network and a weighted network loss function to learn region weights to obtain the similarity weights of each region of the source imaging device and the target imaging device; After training is completed, the comprehensive optical embedding features and similarity weights obtained by using the distortion correction model based on the training set are the optical features and optical feature similarity weights of each region of the source imaging device and the target imaging device.

4. A distortion image correction method based on optical imaging according to claim 3, characterized in that, Extracting the optical features includes: constructing a position correlation matrix based on the pixel positions of the source imaging device and the target imaging device respectively; constructing an optical point correlation matrix based on the optical point distributions of the source imaging device and the target imaging device respectively; constructing an edge similarity matrix based on the edge features of the source imaging device and the target imaging device respectively; constructing a distortion type correlation matrix based on the distortion mode data of the source imaging device and the target imaging device respectively; Based on the above multiple matrices, the optical features are obtained.

5. A distortion image correction method based on optical imaging according to claim 4, characterized in that, The transformation network sequentially uses multi-level adaptive learning based on structural similarity to perform spatial transformation learning on each region, including: using a multi-scale convolutional network to process the position correlation matrix, optical point correlation matrix, edge similarity matrix and distortion type correlation matrix to obtain a feature embedding vector corresponding to each matrix; using the self-attention mechanism of the feature fusion layer to update and fuse the feature embedding vectors of the source imaging device and the target imaging device respectively to obtain an updated feature embedding, and outputting a comprehensive optical embedding feature through an affine transformation; using domain-level adaptive learning, pixel-level adaptive learning and neighborhood-level adaptive learning based on structural similarity to perform spatial transformation learning on each region in sequence.

6. The distortion image correction method based on optical imaging according to claim 5, wherein The domain-level adaptive learning includes: extracting the embedding features of the central region, the edge region, and the distortion region from the updated feature embedding, and calculating the structural similarity of the three types of embedding features; calculating the loss function for the corresponding region type based on the structural similarity of the three types of embedding features; training the transformation network based on the domain-level overall loss function obtained from the loss functions of various region types.

7. A distortion image correction method based on optical imaging according to claim 6, characterized in that, The overall domain-level loss function is expressed as: ; where is the overall domain-level loss function, is the domain-level loss of the source imaging device, is the domain-level loss of the target imaging device, is the central region structure loss, is the edge region gradient loss, is the distortion region feature loss, is the weight coefficient, is the edge adjacency matrix, is the edge similarity matrix, are the distortion feature embeddings of the source device and the target device respectively, is the gradient operator, and SSIM is the structural similarity index.

8. A distortion image correction method based on optical imaging according to claim 5, characterized in that The pixel-level adaptive learning includes: calculating the structural similarity between the comprehensive optical embedding features of the source imaging device and the target imaging device at corresponding pixel positions; calculating the gradient magnitude difference between the comprehensive optical embedding features of the source imaging device and the target imaging device at corresponding pixel positions; constructing a pixel-level loss function based on the structural similarity and the gradient magnitude difference: ; where is the pixel-level feature vector of the source device, is the pixel-level feature vector of the target device, is the gradient operator, is the weight coefficient; training the transformation network based on the pixel-level loss function.

9. A distortion image correction method based on optical imaging according to claim 5, characterized in that, The neighborhood-level adaptive learning includes: obtaining a neighborhood feature vector by connecting the embedding features of an adjacent pixel pair from the source imaging device and the target imaging device; training the transformation network based on the neighborhood feature vector using a structural similarity loss function.

10. A distortion image correction method based on optical imaging according to any one of claims 3-9, characterized in that, The weighted network loss function is as follows: ; where MSE is the mean squared error, is the image distortion parameter predicted by the model, Y is the image distortion parameter measured actually, is the weight of the regularization term, is the optical feature similarity weight of the source imaging device, is the optical feature similarity weight of the target imaging device.

Images