A method and system for fisheye image correction based on synthetic distortion enhancement

Abstract

This invention belongs to the field of fisheye image correction. To address the problem that existing methods easily lead to image blurring or geometric distortion, it provides a fisheye image correction method and system based on synthetic distortion enhancement. The synthetic distortion enhancement-based fisheye image correction method includes performing a nonlinear geometric transformation on the original image using a dual midpoint circle mapping model to generate synthetic fisheye image samples with strong distortion features and clear structure, forming a synthetic fisheye image sample set. Based on the synthetic fisheye image sample set and combined with multiple loss functions, a distortion correction network model is trained. The trained distortion correction network model is then used to correct the fisheye image to be processed, thereby restoring the distorted region and outputting a distortion-free fisheye image. This improves the image restoration capability and model robustness under complex distortion conditions.

Description

Technical Field

[0001] This invention belongs to the field of fisheye image correction, and particularly relates to a fisheye image correction method and system based on synthetic distortion enhancement. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Fisheye cameras, with their ultra-wide field of view and compact size, have been widely used in various scenarios such as autonomous driving, intelligent monitoring, and virtual reality. However, due to their unique optical structure, fisheye lenses are prone to strong nonlinear distortion during imaging, especially noticeable in image edge regions, severely affecting the geometric representation of the image and the accuracy of subsequent visual understanding tasks. To address the distortion problem in fisheye images, existing image correction methods include those based on geometric modeling and those based on deep learning. Geometric modeling methods typically require precise camera calibration, estimating and correcting distortion by establishing imaging model parameters. While theoretically sound, these methods demand high calibration accuracy in practical applications and struggle to adapt to unknown or dynamically changing imaging environments. Deep learning-based fisheye image correction methods leverage the powerful feature extraction and fitting capabilities of convolutional neural networks to achieve end-to-end mapping from distorted to normal images. These methods offer greater flexibility and adaptability, but their performance largely depends on the diversity and representativeness of the training data. Because obtaining real fisheye images and their corresponding distortion parameters or labeled images is costly, currently available training datasets generally suffer from insufficient samples and incomplete coverage of distortion types, which limits the model's generalization ability in complex scenes.

[0004] Furthermore, existing image synthesis methods primarily employ simple projection transformations or low-order polynomial models when generating simulated fisheye images for training. These methods cannot fully simulate the nonlinear, high-intensity distortions produced by real-world fisheye lenses, resulting in insufficient recovery capabilities of the trained models for extreme distortions. Simultaneously, current network architectures have limited modeling capabilities for distorted regions, particularly in edge structure restoration and local detail preservation, which can easily lead to image blurring or geometric distortion. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a fisheye image correction method and system based on synthetic distortion enhancement, which can realistically simulate high-intensity distortion. At the same time, it introduces distortion perception and structure guidance mechanisms into the network structure to improve image restoration capability and model robustness under complex distortion conditions.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] The first aspect of the present invention provides a fisheye image correction method based on synthetic distortion enhancement.

[0008] In one or more embodiments, a fisheye image correction method based on synthetic distortion enhancement is provided, comprising:

[0009] The original image is subjected to nonlinear geometric transformation using a dual midpoint circle mapping model to generate synthetic fisheye image samples with strong distortion features and clear structure, thus forming a synthetic fisheye image sample set.

[0010] Based on the synthetic fisheye image sample set and multiple loss functions, a distortion correction network model is trained. The trained distortion correction network model is then used to correct the fisheye image to be processed, so as to restore the distorted area and output a distortion-free fisheye image.

[0011] The distortion correction network model adopts an encoder-decoder structure. Both the encoder and decoder incorporate a distortion perception module and a distortion adaptive Transformer module. The distortion perception module in the encoder extracts key distortion region features from the synthesized fisheye image sample / fisheye image to be processed. The distortion adaptive Transformer module in the encoder captures global structural relationship features in the synthesized fisheye image sample / fisheye image to be processed based on the key distortion region features. The distortion perception module in the decoder refines and enhances the key distortion region features, while the distortion adaptive Transformer module in the decoder optimizes the global structural relationship features.

[0012] As one implementation method, the process of performing a nonlinear geometric transformation on the original image using the double midpoint circle mapping model is as follows:

[0013] A geometric circular path tangent to the boundary is constructed in a two-dimensional image coordinate system. The intersection of the two circles is used as the mapping position of the pixel in the distorted image to realize the distortion transformation of the original image pixels.

[0014] In one implementation, the distortion perception module includes three feature extraction paths, one of which is used to extract the texture background of the synthetic fisheye image sample / original image, and the other two are used to extract the high-frequency features of the high-frequency image and the contour features of the edge-enhanced image, respectively.

[0015] As one implementation, the distortion perception module further includes a feature fusion submodule, which introduces a spatial attention mechanism and a channel attention mechanism. The spatial attention mechanism generates a position-sensitive attention map based on the differences in spatial positions in the input feature map, guiding the model to focus on regions with prominent structural deformation. The channel attention mechanism, on the other hand, weights different channels according to their importance in semantic expression.

[0016] In one implementation, the distortion-adaptive Transformer module uses depthwise separable convolutions instead of standard linear mapping layers to generate query, key, and value matrices, introducing local spatial information to enhance the ability to express edge structures and texture details.

[0017] As one implementation method, a learnable temperature factor is introduced during the process of multiplying the query with the key to form the attention weight matrix, which is used to adjust the numerical range and sensitivity of the inner product result.

[0018] As one implementation method, multiple loss functions include structural reconstruction loss, adversarial loss, perception and style enhancement loss, and multi-scale supervision loss. The various loss functions are combined in a weighted manner to form the total loss function of the distortion correction network model.

[0019] A second aspect of the present invention provides a fisheye image correction system based on synthetic distortion enhancement.

[0020] In one or more embodiments, a fisheye image correction system based on synthetic distortion enhancement includes:

[0021] The sample set construction module is used to perform nonlinear geometric transformation on the original image using the dual midpoint circle mapping model to generate synthetic fisheye image samples with strong distortion features and clear structure, thus forming a synthetic fisheye image sample set.

[0022] The model training module is used to train a distortion correction network model based on the synthetic fisheye image sample set and in conjunction with multiple loss functions. The trained distortion correction network model is used to correct the fisheye image to be processed, so as to restore the distorted area and output a distortion-free fisheye image.

[0023] The distortion correction network model adopts an encoder-decoder structure. Both the encoder and decoder incorporate a distortion perception module and a distortion adaptive Transformer module. The distortion perception module in the encoder extracts key distortion region features from the synthesized fisheye image sample / fisheye image to be processed. The distortion adaptive Transformer module in the encoder captures global structural relationship features in the synthesized fisheye image sample / fisheye image to be processed based on the key distortion region features. The distortion perception module in the decoder refines and enhances the key distortion region features, while the distortion adaptive Transformer module in the decoder optimizes the global structural relationship features.

[0024] A third aspect of the present invention provides a computer-readable storage medium.

[0025] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the fisheye image correction method based on synthetic distortion enhancement as described above.

[0026] A fourth aspect of the present invention provides an electronic device.

[0027] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the fisheye image correction method based on synthetic distortion enhancement as described above.

[0028] Compared with the prior art, the beneficial effects of the present invention are:

[0029] This invention relates to a fisheye image correction method and system based on synthetic distortion enhancement, aiming to address the problems of insufficient data coverage, limited modeling capabilities, and poor structural restoration effects in existing technologies under strong distortion scenarios. The method uses a dual midpoint circle mapping model to perform nonlinear transformations on standard images, generating synthetic images with diverse high-intensity distortion features. By introducing a distortion-aware dual attention mechanism and a distortion-adaptive Transformer module, key distortion features are extracted and global structural relationships are modeled. Finally, a multi-scale deep correction network integrating local perception and global modeling capabilities is constructed and trained end-to-end using multiple loss functions, achieving accurate restoration of distorted regions and high-quality image output. This method is suitable for geometric correction and visual quality improvement of fisheye images in complex scenarios. Attached Figure Description

[0030] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0031] Figure 1 This is a schematic diagram of the fisheye image correction method based on synthetic distortion enhancement according to an embodiment of the present invention;

[0032] Figure 2 This is a schematic diagram of the dual midpoint circle mapping model in the fisheye image correction method of this invention.

[0033] Figure 3 This is a schematic diagram of the distortion-perceiving dual attention mechanism in the fisheye image correction method of this invention.

[0034] Figure 4 This is a schematic diagram of the distortion adaptive Transformer module in the fisheye image correction method of this invention.

[0035] Figure 5 This is a schematic diagram of the fisheye image correction system based on synthetic distortion enhancement according to an embodiment of the present invention;

[0036] Figure 6 This is a comparison of the fisheye image correction method of this invention with other methods;

[0037] Figure 7 The results are the correction results of real fisheye images taken by the OV5640 220-degree fisheye camera based on self-network search in this embodiment of the invention. Detailed Implementation

[0038] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0039] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0040] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0041] Figure 1 This is a schematic diagram illustrating a fisheye image correction method based on synthetic distortion enhancement, according to an embodiment of the present invention. Combined with... Figure 1 Fisheye image correction methods based on synthetic distortion enhancement may include:

[0042] S101. The original image is subjected to nonlinear geometric transformation using a dual midpoint circle mapping model to generate synthetic fisheye image samples with strong distortion features and clear structure, thus forming a synthetic fisheye image sample set.

[0043] The process of performing nonlinear geometric transformation on the original image using the double midpoint circle mapping model is as follows:

[0044] A geometric circular path tangent to the boundary is constructed in a two-dimensional image coordinate system. The intersection of the two circles is used as the mapping position of the pixel in the distorted image to realize the distortion transformation of the original image pixels.

[0045] In this embodiment of the invention, a standard viewpoint image from a public dataset is selected as the original input image. The image is then subjected to nonlinear geometric transformation through a dual midpoint circle mapping model to simulate the strong distortion effect of a fisheye lens in the actual imaging process, thereby constructing a synthetic fisheye image with highly nonlinear characteristics and complex geometric distortion.

[0046] Specifically, such as Figure 2 As shown, let the coordinate system of a single image be XOY, and arbitrarily select a vertical line in the image. Pixel A and the horizontal line pixels on Construct and respectively 、 Tangent circles , The centers of their circles are respectively , The intersection point is denoted as q.

[0047] In the vertical direction, let the coordinates of point A be... Let point C be the foot of the perpendicular from point C to the bottom edge of the image (y=0). Construct a path through point C. And perpendicular to the line Tangent circles Its center is marked as The coordinates are obtained by the following formula:

[0048] ;

[0049] The radius of the circle It can be obtained from the following formula:

[0050] ;

[0051] Since point q∈ And satisfy the condition of point The vertical height is consistent, that is This allows us to calculate the distorted mapping point of point A in the vertical direction.

[0052] Similarly, in the horizontal direction, let the coordinates of point B be... Let point D be the foot of the perpendicular from the left boundary (x=0). Construct a path through... And with Tangent circles Its center is marked as The coordinates are obtained by the following formula:

[0053] ;

[0054] The radius of the circle It can be obtained from the following formula:

[0055] ;

[0056] Similarly, point q∈ and Thus, the distortion mapping point of point B in the horizontal direction can be obtained.

[0057] To maintain the geometric consistency and structural symmetry of the distortion mapping, select and Midpoint:

[0058] As input, and by calculating its mapping result in the distorted image, the final distorted pixel is defined by the joint mapping function:

[0059] ;

[0060] in This represents the distortion function based on the double midpoint circle mapping. Represents a circle , The intersection points, and the points that conform to the mapping direction, are used as the final output.

[0061] Based on the above geometric construction, a batch image processing strategy is adopted to uniformly enhance the distortion of a large-scale standard image dataset, resulting in pairs of distorted images and corresponding standard images. By adjusting the parameter r, the type and intensity of distortion are controlled to form a training image set containing different degrees of distortion.

[0062] S102, a distortion correction network model is trained based on the synthetic fisheye image sample set and multiple loss functions. The trained distortion correction network model is used to correct the fisheye image to be processed, so as to restore the distorted area and output a distortion-free fisheye image.

[0063] The distortion correction network model adopts an encoder-decoder structure. Both the encoder and decoder incorporate a distortion perception module and a distortion adaptive Transformer module. The distortion perception module in the encoder extracts key distortion region features from the synthesized fisheye image sample / fisheye image to be processed. The distortion adaptive Transformer module in the encoder captures global structural relationship features in the synthesized fisheye image sample / fisheye image to be processed based on the key distortion region features. The distortion perception module in the decoder refines and enhances the key distortion region features, while the distortion adaptive Transformer module in the decoder optimizes the global structural relationship features.

[0064] To address the issue that distorted regions in fisheye images typically exhibit complex visual features such as non-uniform deformation, blurred edges, and local structural distortion, this embodiment introduces a dual attention mechanism for distortion perception to achieve salient feature extraction and representation enhancement for key distorted regions.

[0065] like Figure 3 As shown, first input the distorted image. High-frequency images are obtained by performing high-pass filtering and edge detection respectively. With edge image High-frequency images preserve image texture variations, while edge images highlight structural contour information. These three types of images serve as the joint input to a multi-branch feature extraction network, each extracting multi-scale primary features through a lightweight convolutional module. Each branch network extracts deep semantic features by sharing some parameters and adjusting local adaptive paths, and then outputs these features to the feature fusion unit.

[0066] The distortion perception module of this invention includes three feature extraction paths. One feature extraction path is used to extract the texture background of the synthetic fisheye image sample / original image. The other two feature extraction paths are used to extract the high-frequency features of the high-frequency image and the contour features of the edge-enhanced image, respectively. The calculation formulas for the three paths are as follows:

[0067] ;

[0068] , and This represents three pathway characteristics; among them, This indicates element-wise multiplication. This represents the edge enhancement function.

[0069] The above three sets of features are concatenated along the channel dimension to form a multidimensional feature representation tensor that integrates spatial details and edge responses. Compression and normalization are then achieved through a fusion convolution module.

[0070] ;

[0071] in, For activation functions; This is the fusion function; It is a convolution function.

[0072] In this embodiment of the invention, the distortion perception module further includes a feature fusion submodule, which introduces a spatial attention mechanism and a channel attention mechanism. The spatial attention mechanism generates a position-sensitive attention map based on the differences in spatial positions in the input feature map, guiding the model to focus on regions with prominent structural deformation. The channel attention mechanism, on the other hand, weights different channels according to their importance in semantic expression.

[0073] In the fusion phase, a spatial attention mechanism is first applied to adaptively emphasize regions with significant distortion. This module assigns weights based on the importance of each pixel position, prioritizing the retention of features from regions with severe edge distortion and obvious texture breaks, while filtering out redundant or invalid background areas. Subsequently, a channel attention mechanism is introduced to reconstruct the feature map along the channel dimension with weights, guiding the model to focus on feature channels with strong expressive power and sensitive response to changes. The spatial and channel attention mechanisms are used in tandem to form a joint structure-texture enhancement path, achieving robust modeling of complex distortion regions without introducing additional structural biases.

[0074] In its implementation, the attention module employs residual connections and a sigmoid activation function to ensure gradient stability and smooth feature enhancement. (Attention weights and original feature map) Perform pixel-by-pixel fusion to form an enhanced distortion-aware feature map. The specific calculations are as follows:

[0075] ;

[0076] in, This represents the attention module function.

[0077] Finally, this feature map will be fed as input into the subsequent global structure modeling module to further model long-distance structural dependencies and global semantics.

[0078] To achieve global modeling and geometric consistency constraints of fisheye image structural information, this embodiment introduces a distortion-adaptive Transformer module to realize structural analysis of long-distance dependencies in the image and multi-scale feature fusion. This module makes targeted improvements to the traditional Transformer encoder structure, possessing global semantic modeling capabilities while enhancing the analysis and response capabilities to local structural distortion regions.

[0079] like Figure 4 As shown, in the process of constructing the attention mechanism, the Transformer encoder uses depthwise separable convolutions instead of traditional linear mapping layers to generate queries. ,key Sum A matrix is ​​used to introduce local spatial information to enhance the representation of edge structure and texture details. The specific calculation is as follows: ;

[0080] In the process of calculating attention weights, a learnable temperature factor is further introduced. This is used to adjust the scale and sensitivity of the query and key matrix multiplication result. Firstly, regarding... and Normalization is performed:

[0081] ;

[0082] Subsequently, the normalized attention weight matrix is ​​calculated. Its definition is as follows:

[0083] ;

[0084] This temperature factor can be dynamically optimized based on the training process, enabling the attention mechanism to respond more accurately to cross-regional distortion correlations.

[0085] Finally, based on the calculated attention weights AND-value matrix Complete feature aggregation: In the feedforward network section, to overcome the limitations of traditional fully connected structures in preserving structural information, this embodiment uses a gated convolutional feedforward network module instead of the standard feedforward sub-network. The gated convolutional feedforward network module consists of two parallel paths: one path processes features through an activation function, and the other serves as a gated control branch. Generate a weight map; the two features are multiplied element-wise before output to achieve dynamic feature adjustment, as calculated below:

[0086] ;

[0087] ;

[0088] To achieve effective restoration and pixel-level reconstruction of severely distorted regions in fisheye images, this embodiment constructs a distortion correction model with multi-scale collaborative modeling capabilities. This model employs an encoder-decoder architecture and integrates the aforementioned distortion perception module and distortion adaptive Transformer module at different resolution levels, forming a deep neural network with both structure-guided and distortion-modeling capabilities.

[0089] Specifically, the encoder of this model consists of multiple downsampling convolutional layers, each responsible for extracting image features at different scales. In the high-resolution six-layer shallow feature set, a distortion-aware module is introduced to enhance the capture of local edge deformation regions, distorted textures, and detail changes. In the low-resolution five-layer deep feature set, a distortion-adaptive Transformer module is embedded to model global structural dependencies and restore cross-regional geometric relationships, ensuring the network possesses overall structural reconstruction capabilities and globally consistent representation capabilities. In the decoder, a layer-by-layer upsampling strategy is employed, combined with a skip connection mechanism, to fuse the features extracted from each encoder layer back to the corresponding decoder layer, achieving cascaded integration of multi-scale semantic information and original structural features.

[0090] To achieve high-fidelity restoration and structural consistency preservation of distorted regions in fisheye images, this embodiment employs an end-to-end training strategy driven by multiple loss functions to globally optimize the aforementioned distortion correction network model.

[0091] Multiple loss functions include structural reconstruction loss, adversarial loss, perceptual and style enhancement loss, and multi-scale supervision loss;

[0092] Various loss functions are combined in a weighted manner to form the total loss function of the distortion correction network model. :

[0093] .

[0094] Structural reconstruction loss ;

[0095] Combat losses ;

[0096] Perception and style enhancement loss:

[0097] ;

[0098] Multi-scale monitoring loss ;

[0099] in, Indicates the correction result For the corresponding reference image, and They represent the first The feature map after layer correction compared to the original. For downsampling operation, Represents a convolution mapping. For feature splicing operations, and These are the loss weighting coefficients; Indicates the first jThe number of channels, height, and width of the layer feature map; Indicates the loss network number j Feature map of the layer; Indicates the first j The Gram matrix of the layer; Measuring the discriminator's ability to correctly identify real data; Measuring the ability of a generator to deceive a discriminator; For the discriminator function; This is a generator function.

[0100] During training, the model uses the Adam optimizer for parameter updates, with an initial learning rate set to 1e-4, and incorporates a learning rate decay strategy to adapt to the training needs at different stages. To avoid gradient explosion and overfitting, a gradient clipping mechanism and an early stopping strategy are introduced, and dynamic adjustments are made based on the validation set performance after each training round.

[0101] After training, the model performs fast forward inference on any input fisheye image during the testing phase to complete the structural restoration of the distorted region of the image and the generation of a distortion-free image.

[0102] The correction process in this invention has end-to-end processing capabilities. It requires only a single fisheye image as input, and the network can automatically complete operations such as distortion region identification, feature extraction, structural modeling, and image reconstruction, outputting a distortion-free image with excellent visual quality, clear edges, and consistent structure. Relying on pre-synthesized distortion enhancement and multi-task training strategies, this correction network still possesses good generalization ability and robustness when facing fisheye distortion images of different degrees and types, making it widely applicable to image preprocessing tasks in complex application scenarios such as autonomous driving, intelligent security, and virtual reality.

[0103] This invention uses SMIA-TV distortion as an objective indicator to measure the degree of image distortion and introduces the modulation transfer function (MTF) as an evaluation standard for image sharpness. The larger the absolute value of SMIA-TV distortion, the more significant the distortion; the larger the MTF value, the higher the image sharpness.

[0104] SMIA TV distortion, defined by the Standard Mobile Imaging Architecture (SMIA) organization, is a quantitative standard specifically designed to evaluate image distortion in mobile device cameras. Its analysis method compares the relative distortion of edge and center regions.

[0105] Table 1 compares the performance of the dual midpoint circular mapping model with other mainstream synthesis methods under the aforementioned metrics. The results show that the dual midpoint circular mapping model performs best in SMIA-TV distortion and outperforms other methods in sharpness evaluation, demonstrating that it can effectively maintain high image sharpness and structural fidelity while generating strongly distorted images. To visually demonstrate the effect, Figure 6 The paper presents a comparison of this method with other methods, demonstrating that the proposed dual midpoint circle mapping model exhibits more significant distortion features while maintaining the coherence of the image structure.

[0106] Table 1. Quantitative comparison of different distortion models;

[0107]

[0108] In this context, ↑ represents a larger value indicating better performance, and ↓ represents a smaller value indicating better performance.

[0109] To evaluate the performance of the proposed method in fisheye image correction, six representative comparative methods were selected, including the traditional geometric model SC, the regression-based DeepCalib, the generative adversarial network-based DR-GAN, SimFIR, and PCN, covering technical paths such as geometric modeling, generative modeling, and end-to-end regression. Quantitative evaluation used Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) to measure pixel and structural restoration capabilities, Multi-Scale Structural Similarity (MS-SSIM) and Composite Wavelet Structural Similarity (CW-SSIM) to reflect the consistency of multi-scale and edge structures in the image, and Generative Image Perceptual Quality (FID) and Perceptual Image Similarity (LPIPS) to evaluate the perceptual realism and detail quality of the image. Table 2 summarizes the evaluation results of each method on a synthetic dataset. The results show that the proposed method performs best in all metrics, outperforming existing methods and demonstrating significant comprehensive advantages. Figure 7 The invention demonstrates the correction results of real fisheye images obtained from web searches and captured by an OV5640 220-degree fisheye camera. Unlike other methods that suffer from problems such as inability to correct complex distortions, structural distortion, or artifacts, this invention can effectively repair structural distortions and maintain the geometric consistency and content integrity of the image without requiring camera parameters.

[0110] Table 2. Quantitative evaluation of different methods on synthetic datasets;

[0111]

[0112] In this context, ↑ represents a larger value indicating better performance, and ↓ represents a smaller value indicating better performance.

[0113] Figure 5This is a schematic diagram of a fisheye image correction system based on synthetic distortion enhancement, as described in an embodiment of the present invention. Figure 5 As shown, the fisheye image correction system based on synthetic distortion enhancement in this embodiment may include:

[0114] The sample set construction module 501 is used to perform nonlinear geometric transformation on the original image using the dual midpoint circle mapping model to generate synthetic fisheye image samples with strong distortion features and clear structure, thus forming a synthetic fisheye image sample set.

[0115] The model training module 502 is used to train a distortion correction network model based on the synthetic fisheye image sample set and in conjunction with multiple loss functions, and to use the trained distortion correction network model to correct the fisheye image to be processed, so as to restore the distorted region and output a distortion-free fisheye image.

[0116] The distortion correction network model adopts an encoder-decoder structure. Both the encoder and decoder incorporate a distortion perception module and a distortion adaptive Transformer module. The distortion perception module in the encoder extracts key distortion region features from the synthesized fisheye image sample / fisheye image to be processed. The distortion adaptive Transformer module in the encoder captures global structural relationship features in the synthesized fisheye image sample / fisheye image to be processed based on the key distortion region features. The distortion perception module in the decoder refines and enhances the key distortion region features, while the distortion adaptive Transformer module in the decoder optimizes the global structural relationship features.

[0117] It should be noted here that, Figure 5 Each module in the fisheye image correction system based on synthetic distortion enhancement corresponds one-to-one with each step in the fisheye image correction method based on synthetic distortion enhancement, and their specific implementation processes are the same, so they will not be described in detail here.

[0118] This invention relates to a fisheye image correction method and system based on synthetic distortion enhancement, aiming to address the problems of insufficient data coverage, limited modeling capabilities, and poor structural restoration effects in existing technologies under strong distortion scenarios. The method uses a dual midpoint circle mapping model to perform nonlinear transformations on standard images, generating synthetic images with diverse high-intensity distortion features. By introducing a distortion-aware dual attention mechanism and a distortion-adaptive Transformer module, key distortion features are extracted and global structural relationships are modeled. Finally, a multi-scale deep correction network integrating local perception and global modeling capabilities is constructed and trained end-to-end using multiple loss functions to achieve accurate restoration of distorted regions and high-quality image output. The fisheye image correction system based on synthetic distortion enhancement includes functional modules such as image generation, feature extraction, structural modeling, model training and inference, and image correction and reconstruction. It possesses good robustness and generalization performance and is suitable for geometric correction and visual quality improvement of fisheye images in complex scenarios.

[0119] In one or more embodiments, an electronic device is provided including a central processing unit (CPU) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage portion into a random access memory (RAM). Various programs and data required for system operation are also stored in the RAM. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0120] The following components are connected to the I / O interface: input sections including keyboards, mice, etc.; output sections including cathode ray tubes (CRTs), liquid crystal displays (LCDs), and speakers; storage sections including hard disks; and communication sections including network interface cards such as local area network (LAN) cards and modems. The communication sections perform communication processing via networks such as the Internet. Drives are also connected to the I / O interface as needed. Removable media, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive as needed so that computer programs read from them can be installed into the storage section as required.

[0121] When the central processing unit in the electronic device of this embodiment executes the program, it implements the steps in the above-described fisheye image correction method based on synthetic distortion enhancement.

[0122] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing a fisheye image correction method based on synthetic distortion enhancement. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit, it performs the various functions defined in the apparatus of this application.

[0123] The computer program instructions corresponding to the fisheye image correction method based on synthetic distortion enhancement can also be stored in a computer-readable storage medium that can guide a computer or other programmable data processing device to operate in a specific manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0124] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0125] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for correcting fisheye images based on synthetic distortion enhancement, characterized in that, include: The original image is subjected to nonlinear geometric transformation using a dual midpoint circle mapping model to generate synthetic fisheye image samples with strong distortion features and clear structure, thus forming a synthetic fisheye image sample set. Based on the synthetic fisheye image sample set and multiple loss functions, a distortion correction network model is trained. The trained distortion correction network model is then used to correct the fisheye image to be processed, so as to restore the distorted area and output a distortion-free fisheye image. The distortion correction network model adopts an encoder-decoder structure. Both the encoder and decoder incorporate a distortion-aware module and a distortion-adaptive Transformer module. The distortion-aware module in the encoder extracts key distortion region features from the synthesized fisheye image sample / fisheye image to be processed. The distortion-adaptive Transformer module in the encoder captures global structural relationship features in the synthesized fisheye image sample / fisheye image to be processed based on the key distortion region features. The distortion-aware module in the decoder refines and enhances the key distortion region features, while the distortion-adaptive Transformer module in the decoder optimizes the global structural relationship features. The process of performing a nonlinear geometric transformation on the original image using the double midpoint circle mapping model is as follows: In a two-dimensional image coordinate system, a geometric circular path tangent to the boundary is constructed, and the intersection of the two circles is used as the mapping position of the pixel in the distorted image to realize the distortion transformation of the original image pixels. The distortion perception module includes three feature extraction paths. One feature extraction path is used to extract the texture background of the synthetic fisheye image sample / original image, and the other two feature extraction paths are used to extract the high-frequency features of the high-frequency image and the contour features of the edge enhancement image, respectively. The distortion perception module also includes a feature fusion submodule, which introduces a spatial attention mechanism and a channel attention mechanism. The spatial attention mechanism generates a position-sensitive attention map based on the differences in spatial positions in the input feature map, guiding the model to focus on regions with prominent structural deformation. The channel attention mechanism, on the other hand, weights different channels according to their importance in semantic expression.

2. The fisheye image correction method based on synthetic distortion enhancement as described in claim 1, characterized in that, The distortion-adaptive Transformer module uses depthwise separable convolutions instead of standard linear mapping layers to generate query, key, and value matrices, introducing local spatial information to enhance the ability to express edge structures and texture details.

3. The fisheye image correction method based on synthetic distortion enhancement as described in claim 2, characterized in that, A learnable temperature factor is introduced during the process of multiplying the query with the key to form the attention weight matrix, which is used to adjust the numerical range and sensitivity of the inner product result.

4. The fisheye image correction method based on synthetic distortion enhancement as described in claim 1, characterized in that, Multiple loss functions include structural reconstruction loss, adversarial loss, perception and style enhancement loss, and multi-scale supervision loss. These loss functions are combined in a weighted manner to form the total loss function of the distortion correction network model.

5. A fisheye image correction system based on synthetic distortion enhancement, characterized in that, The method is implemented using the steps of the fisheye image correction method based on synthetic distortion enhancement as described in any one of claims 1-4, including: The sample set construction module is used to perform nonlinear geometric transformation on the original image using the dual midpoint circle mapping model to generate synthetic fisheye image samples with strong distortion features and clear structure, thus forming a synthetic fisheye image sample set. The model training module is used to train a distortion correction network model based on the synthetic fisheye image sample set and in conjunction with multiple loss functions. The trained distortion correction network model is used to correct the fisheye image to be processed, so as to restore the distorted area and output a distortion-free fisheye image. The distortion correction network model adopts an encoder-decoder structure. Both the encoder and decoder incorporate a distortion perception module and a distortion adaptive Transformer module. The distortion perception module in the encoder extracts key distortion region features from the synthesized fisheye image sample / fisheye image to be processed. The distortion adaptive Transformer module in the encoder captures global structural relationship features in the synthesized fisheye image sample / fisheye image to be processed based on the key distortion region features. The distortion perception module in the decoder refines and enhances the key distortion region features, while the distortion adaptive Transformer module in the decoder optimizes the global structural relationship features.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the fisheye image correction method based on synthetic distortion enhancement as described in any one of claims 1-4.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the fisheye image correction method based on synthetic distortion enhancement as described in any one of claims 1-4.

Images