Mixed shooting image correction method and system based on deep learning

Abstract

The invention discloses a deep learning-based mixed-shooting image correction method and system, and relates to the field of computer vision, and the method comprises the steps: carrying out the illumination normalization processing of a mixed-shooting image group, and constructing a Gaussian pyramid image group; inputting the Gaussian pyramid image group into a ResNet-34 network to extract a multi-scale feature map, and generating spatial position and channel statistical information; and converting the spatial position and the channel statistical information into a DNA sequence fragment through a DNA base mapping rule. Illumination normalization is achieved through Retinex decomposition and Gaussian filtering, the exposure difference of different devices is effectively eliminated, the input quality of follow-up processing is guaranteed, traditional feature description is converted into biological sequence comparison through DNA base coding, geometric parameters are optimized in combination with quantum annealing, and the accuracy of the method is improved. The matching precision of a low-texture area and the correction effect of a large-view-angle-difference scene are remarkably improved, and the sub-pixel-level matching precision is achieved based on weighted RANSAC and multi-scale pyramid fusion.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a method and system for correcting mixed-shot images based on deep learning. Background Art

[0002] With the popularization of multi-camera mobile devices and drone aerial photography technology, the correction and fusion technology of mixed images has become a research hotspot in the field of computer vision. Traditional methods are mainly based on feature point matching and geometric transformation models. For example, SIFT is combined with the RANSAC algorithm to achieve image registration. This technology extracts local feature descriptors, uses nearest neighbor search to match key points, and estimates the homography matrix or basic matrix through iterative optimization. Existing deep learning solutions further introduce convolutional neural networks to extract high-order features, improving the matching robustness in weak texture areas. When dealing with scenes with large perspective differences, inconsistent lighting or dynamic blur, existing technologies still rely on manually designed feature descriptors and fixed optimization strategies, failing to fully explore the correlation between cross-modal data.

[0003] Existing methods suffer from insufficient cross-modal feature comparability in the feature matching stage. When mixed images have significant illumination differences or resolution mismatches, traditional feature descriptors find it difficult to establish accurate similarity metrics, resulting in a decrease in matching accuracy. In low-light conditions, gradient information is susceptible to noise interference, while manually designed binary descriptors lose multi-scale statistical properties. The RANSAC algorithm uses a uniform sampling strategy and does not consider the semantic correlation between feature points, which may ignore high-confidence matching pairs. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a mixed-shot image correction method based on deep learning to solve the problems of inaccurate similarity measurement and decreased matching accuracy caused by illumination differences and resolution mismatch in cross-modal image feature matching.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: In a first aspect, the present invention provides a mixed-shot image correction method based on deep learning, which includes performing illumination normalization processing on a mixed-shot image group and constructing a Gaussian pyramid image group; Input the Gaussian pyramid image group into the ResNet-34 network to extract multi-scale feature maps and generate spatial position and channel statistics; The spatial position and channel statistical information are converted into DNA sequence fragments through DNA base mapping rules, and the Smith-Waterman algorithm is used to perform biological sequence alignment to obtain feature point matching pairs; Based on the feature point matching pairs, the initial homography matrix is ​​obtained and the basic matrix and essential matrix are obtained through the weighted RANSAC algorithm; Encode the basic matrix and the essential matrix into quantum bits, and obtain the projection transformation matrix through quantum annealing; The mixed image group is deformed by thin plate spline interpolation using the projection transformation matrix, and pyramid fusion is performed in the LAB color space. The fused image is then subjected to guided filtering to obtain the corrected image. As a preferred solution of the mixed-shot image correction method based on deep learning described in the present invention, wherein: performing illumination normalization processing on the mixed-shot image group and constructing a Gaussian pyramid image group includes the following steps: Receive original images taken under different exposure conditions from different devices, and perform logarithmic domain decomposition on the original images to obtain reflection components and illumination components; The illumination component is filtered using a Gaussian kernel to obtain a smoothed illumination component, and the reflection component is combined with the smoothed illumination component to obtain an illumination-consistent image group; Two-level anti-aliasing downsampling is performed on the image group with consistent illumination to construct a Gaussian pyramid image group.

[0007] As a preferred solution of the mixed-shot image correction method based on deep learning described in the present invention, the Gaussian pyramid image group is input into the ResNet-34 network to extract multi-scale feature maps and generate spatial position and channel statistical information, including the following steps: Input each layer of the pyramid image into the ResNet-34 network, output a multi-scale feature map, calculate the pixel gradient response value on the multi-scale feature map to determine the corner point position, and obtain a coordinate set; The mean and variance statistics are obtained at the two-dimensional spatial position of the multi-scale feature map to obtain channel statistical information.

[0008] As a preferred solution of the mixed-shot image correction method based on deep learning described in the present invention, the spatial position and channel statistical information are converted into DNA sequence fragments according to the DNA base mapping rule, and the Smith-Waterman algorithm is used to perform biological sequence alignment to obtain feature point matching pairs, including the following steps: The two-dimensional spatial position, channel statistics and multi-scale feature maps are spliced ​​and compressed through MLP to obtain the visual and geometric characteristics of the local area of ​​the image; Normalize the visual and geometric characteristics of the local area of ​​the image to obtain an integer vector, and convert the integer vector into a DNA sequence fragment according to the base rule; The Smith-Waterman algorithm is used to perform biological sequence alignment on the DNA sequence fragments of two images with consistent illumination to obtain feature point matching pairs.

[0009] As a preferred solution of the mixed-shot image correction method based on deep learning described in the present invention, wherein: the initial homography matrix is ​​obtained based on the feature point matching pair and the basic matrix and the essential matrix are obtained by the weighted RANSAC algorithm, including the following steps: Perform zero-mean scaling on the feature point matching pairs to obtain normalized coordinates and transformation matrix; Construct the least squares equation based on the normalized coordinates and decompose the singular values ​​to obtain the initial homography matrix; The DNA alignment score is extracted from the feature point matching pairs, and the projection deviation is selected from the initial homography matrix to obtain the geometric error; Calculate weighted RANSAC weights based on DNA alignment score and projection deviation; Sampling is performed according to the weighted RANSAC weight to obtain the basic matrix; By shooting the calibration plate and using Zhang Zhengyou calibration method to obtain the distortion parameters, the camera intrinsic parameter matrix is ​​obtained. Based on the camera intrinsic parameter matrix, the essential matrix is ​​obtained.

[0010] As a preferred solution of the mixed-shot image correction method based on deep learning described in the present invention, wherein: the basic matrix and the essential matrix are encoded as quantum bits, and the projection transformation matrix is ​​obtained through quantum annealing, including the following steps: Normalizing the fundamental matrix and the essential matrix to obtain a normalized vector, and encoding the normalized vector into a quantum bit; Based on the weighted RANSAC sampling weights, the camera intrinsic parameter matrix and the quantum bits, the total Hamiltonian is obtained; Use quantum annealing to process the total Hamiltonian to obtain a bit string, decode the bit string into continuous matrix parameters according to the linear mapping formula, and reconstruct the basic matrix and essential matrix; Decompose the basic matrix, extract the rotation matrix and translation vector, and combine them with the camera intrinsic parameter matrix to obtain the projection transformation matrix.

[0011] As a preferred solution of the mixed-shot image correction method based on deep learning described in the present invention, wherein: using the projection transformation matrix to perform thin plate spline interpolation deformation on the mixed-shot image group, and performing pyramid fusion in the LAB color space, the fused image is subjected to guided filtering processing to obtain a corrected image, including the following steps: The mixed image group is non-rigidly deformed using a projection transformation matrix to obtain a geometrically aligned image group, which is then converted from RGB to LAB color space to separate the luminance and chrominance components. Gaussian pyramids are constructed for the LAB color space respectively. The contribution weights of the image regions are obtained based on the DNA alignment scores and quantum bit strings. The Gaussian pyramids are weightedly fused to obtain a fused image. The fused image is subjected to edge smoothing processing to obtain a filtered LAB image; The filtered LAB image is converted back to the RGB space to obtain a corrected image.

[0012] In a second aspect, the present application provides a mixed image correction system based on deep learning, comprising, An image preprocessing module performs illumination normalization processing on the mixed image group and constructs a Gaussian pyramid image group; A feature extraction module inputs the Gaussian pyramid image group into a ResNet-34 network to extract multi-scale feature maps and generate spatial position and channel statistical information; A DNA encoding module converts the spatial position and channel statistical information into DNA sequence fragments through DNA base mapping rules, and adopts a Smith-Waterman algorithm for biological sequence alignment to obtain a feature point matching pair; A matrix generation module obtains an initial homography matrix based on the feature point matching pair and obtains a fundamental matrix and an essential matrix through a weighted RANSAC algorithm; A quantum optimization module encodes the fundamental matrix and the essential matrix into quantum bits, and obtains a projection transformation matrix through quantum annealing; A correction module performs thin-plate spline interpolation deformation on the mixed image group using the projection transformation matrix, and performs pyramid fusion in the LAB color space, and performs guided filtering processing on the fused image to obtain a corrected image.

[0013] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and wherein the computer program, when executed by the processor, implements any step of the mixed image correction method based on deep learning according to the first aspect of the present application.

[0014] In a fourth aspect, the present application provides a computer readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any step of the mixed image correction method based on deep learning according to the first aspect of the present application.

[0015] The present application has the following advantages: illumination normalization is achieved through Retinex decomposition and Gaussian filtering, effectively eliminating exposure differences of different devices and ensuring input quality for subsequent processing; traditional feature description is converted into biological sequence alignment through DNA base encoding, combined with quantum annealing to optimize geometric parameters, significantly improving matching accuracy in low-texture areas and correction effect in large-angle difference scenes; and sub-pixel level matching accuracy is achieved based on weighted RANSAC and multi-scale pyramid fusion. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 Flowchart of a mixed-shot image correction method based on deep learning; Figure 2 Schematic diagram of a mixed-shot image correction system based on deep learning; Figure 3 Schematic diagram of the corrected image; Figure 4 Schematic diagram of feature point matching pairs. DETAILED DESCRIPTION

[0018] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0019] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0020] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0021] Reference Figures 1 to 4 , is an embodiment of the present invention, which provides a mixed-shot image correction method based on deep learning, comprising the following steps: S1. Perform illumination normalization on the mixed image group and construct a Gaussian pyramid image group.

[0022] S1.1. Receive original images taken under different exposure conditions from different devices, and perform logarithmic domain decomposition on the original images to obtain reflection components and illumination components.

[0023] Furthermore, the original images taken under different device exposure conditions are decomposed in the logarithmic domain. After converting the original images into logarithmic space, the reflection component and illumination component are separated. The reflection component represents the inherent properties of the scene, and the illumination component reflects the ambient lighting conditions.

[0024] S1.2. Use a Gaussian kernel to filter the illumination component to obtain a smoothed illumination component, and combine the reflection component with the smoothed illumination component to obtain an illumination-consistent image group.

[0025] Furthermore, a two-dimensional Gaussian filter function is applied to the illumination component. The standard deviation of the Gaussian kernel is dynamically determined according to the image size. For an image with a width of W pixels and a height of H pixels, the standard deviation of the Gaussian kernel is max(W,H) / 50. Gaussian filtering is implemented using separable convolution. First, a one-dimensional Gaussian convolution is performed in the horizontal direction, and then a one-dimensional Gaussian convolution is performed in the vertical direction to obtain a smoothed illumination component. The smoothed illumination component is combined with the reflection component in the linear domain. The combination method is to perform exponential operations on the illumination component and the reflection component respectively and then multiply them to obtain an illumination consistent image group. The pixel value range of each image in the illumination consistent image group is normalized to [0,255], and the spatial resolution and number of color channels of the original image are kept unchanged.

[0026] S1.3. Perform two-level anti-aliasing downsampling on the image group with consistent illumination to construct a Gaussian pyramid image group.

[0027] Furthermore, a first level of downsampling is performed on each image in the illumination consistent image group. Before downsampling, the image is smoothed using a Gaussian kernel with a standard deviation of 1.0, and the image size is reduced to 1 / 2 of the original image. Then, a second level of downsampling is performed on the image obtained by the first level of downsampling. The image is also smoothed using a Gaussian kernel with a standard deviation of 1.0, and the image size is reduced to 1 / 4 of the original image. Each level of downsampling is implemented using a bilinear interpolation method to ensure the spatial continuity of the image, and finally a three-layer Gaussian pyramid image group including the original scale, 1 / 2 scale, and 1 / 4 scale is obtained.

[0028] S2. Input the Gaussian pyramid image group into the ResNet-34 network to extract multi-scale feature maps and generate spatial position and channel statistics.

[0029] S2.1. Input each layer of the pyramid image into the ResNet-34 network, output a multi-scale feature map, calculate the pixel gradient response value on the multi-scale feature map to determine the corner point position, and obtain a coordinate set.

[0030] Specifically, the expression is, ; in, R is the pixel gradient response value, M is a multi-scale feature map, is an empirical constant, is the determinant of the multi-scale feature map, is the trajectory of the multi-scale feature map.

[0031] Further, the original scale, 1 / 2 scale and 1 / 4 scale images in the Gaussian pyramid image group are input into the ResNet-34 network, the deformable convolution module is added to the ResNet-34 network after the conv3_x and conv4_x layers, the ResNet-34 network outputs multi-scale feature maps, the sizes of the feature maps are 1 / 8, 1 / 16 and 1 / 32 of the input image respectively, the Harris corner response values are calculated on the multi-scale feature maps, the first 500 points with the maximum response values are selected as the corner positions on each layer of the feature maps, the coordinates (x, y) of the corners and the pyramid level l are recorded to form a coordinate set.

[0032] S2.2, the mean and variance statistics are obtained in the two-dimensional spatial position of the multi-scale feature map to obtain the channel statistical information.

[0033] Further, the multi-scale feature maps output by the ResNet-34 network are calculated in the 5x5 neighborhood range of each feature point, and the channel statistics are calculated, for each channel of the 128-dimensional feature map, the mean value of all pixel values in the neighborhood is extracted as the mean value statistics, and the square average of the deviation of the pixel value from the mean value is obtained as the variance statistics, the range of the mean value statistics and the variance statistics is strictly limited in the 5x5 rectangular area around the two-dimensional spatial position of the feature map, and the maximum response value and the minimum response value of the feature point are recorded to form a five-dimensional channel statistical information vector containing the mean value, the variance, the maximum value, the minimum value and the median value.

[0034] S3, the spatial position and the channel statistical information are converted into DNA sequence fragments through the DNA base mapping rule, the Smith-Waterman algorithm is used for biological sequence alignment to obtain the feature point matching pair.

[0035] S3.1, the two-dimensional spatial position, the channel statistical information and the multi-scale feature map are spliced, and the visual and geometric characteristics of the local region of the image are obtained by MLP compression.

[0036] Further, the two-dimensional spatial coordinates of the feature points, the pyramid level, the statistical information vector and the 128-dimensional multi-scale feature map descriptor are spliced in the channel dimension to form a 136-dimensional composite vector, and then the composite vector is input into a two-layer fully connected neural network, the first fully connected layer maps the 136-dimensional input to 64 dimensions using the ReLU activation function, and the second fully connected layer maps the 64-dimensional input to 32 dimensions using the linear activation function to obtain the visual and geometric characteristics of the local region of the image.

[0037] S3.2, the visual and geometric characteristics of the local region of the image are normalized to obtain an integer vector, and the integer vector is converted into a DNA sequence fragment according to the base rule.

[0038] Furthermore, Min-Max normalization is performed on the visual and geometric characteristics of the local area of ​​the image to compress the numerical range to the interval [0,3]. The normalized floating-point values ​​are then converted to integer values ​​by rounding to obtain integer vectors. The integer values ​​are then converted into DNA base characters according to the base mapping rules, where 0 corresponds to base A, 1 corresponds to base T, 2 corresponds to base C, and 3 corresponds to base G. The 32 base characters are spliced ​​in sequence to form a DNA sequence fragment.

[0039] S3.3. Use the Smith-Waterman algorithm to perform biological sequence alignment on the DNA sequence fragments of the two images with consistent illumination to obtain feature point matching pairs.

[0040] Furthermore, a 32×32 scoring matrix is ​​first established, with the matching score set to +2, the mismatch score set to -1, and the gap penalty set to a linear function g(k)=-2k. The optimal alignment path is then searched from the lower right corner to the upper left corner. Sequence pairs with alignment scores higher than a threshold (e.g., score >40) are retained as candidate matches, and the index of the DNA sequence fragment is mapped back to the original feature point coordinates to obtain a preliminary set of feature point matching pairs. Finally, the spatial consistency of the matching pairs is verified, and abnormal matches with a reprojection error greater than 3 pixels are eliminated to form feature point matching pairs.

[0041] S4. Based on the feature point matching pairs, the initial homography matrix is ​​obtained and the basic matrix and the essential matrix are obtained through the weighted RANSAC algorithm.

[0042] S4.1. Perform zero-mean scaling on the feature point matching pairs to obtain normalized coordinates and transformation matrix.

[0043] Furthermore, the arithmetic mean of the x-coordinates and y-coordinates of all feature points in the matching pair set is calculated to obtain the x-coordinate mean μ_x=mean(x_i) and the y-coordinate mean μ_y=mean(y_i), where x_i and y_i represent the horizontal and vertical coordinate values ​​of the i-th feature point, respectively. For the coordinate mean (μ_x, μ_y) of all feature point matching pairs, the original coordinates (x, y) of each feature point are subtracted from the corresponding mean to achieve zero centering, and then the standard deviation of the coordinates (σ_x, σ_y) is calculated. The zero-centered coordinates are divided by the standard deviation to obtain the normalized coordinates. The mean and standard deviation during the scaling process are recorded to obtain the normalized coordinates and transformation matrix. S4.2. Construct a least squares equation based on the normalized coordinates and decompose the singular values ​​to obtain the initial homography matrix.

[0044] Furthermore, an overdetermined linear equation system is constructed using normalized coordinates. Two linear equations are generated for each pair of matching points to constrain the element relationship of the homography matrix. Then, the constructed coefficient matrix is ​​subjected to singular value decomposition, and the last column of the right singular matrix is ​​taken as the solution vector of the homography matrix. The solution vector is rearranged into a matrix form of three rows and three columns to obtain the homography matrix in the normalized coordinate system. The normalized homography matrix is ​​converted back to the original pixel coordinate system by left-multiplying and right-multiplying with the inverse matrix of the normalized transformation matrix to obtain the initial homography matrix.

[0045] S4.3. Extract the DNA alignment score from the feature point matching pairs, select the projection deviation from the initial homography matrix, and obtain the geometric error.

[0046] Furthermore, the DNA alignment score of each pair of feature points is read from the matching results generated by the Smith-Waterman algorithm. The DNA alignment score reflects the degree of sequence similarity. The initial homography matrix is ​​used to project the feature point coordinates of the first image to the coordinate system of the second image, and the Euclidean distance between the projected point and the actual matching point is obtained as the projection deviation. The DNA alignment score is combined with the projection deviation. The DNA alignment score represents the credibility of feature similarity, and the projection deviation represents the spatial geometric consistency. The two together constitute the geometric error evaluation index.

[0047] S4.4. Calculate weighted RANSAC weights based on DNA alignment score and projection deviation.

[0048] Specifically, the expression is, ; in, For the i Weighted RANSAC weights for matching points, For the i DNA comparison score for matching points, is the maximum value of the DNA alignment score, For the i The geometric error of the matching points, is the standard deviation of the Gaussian kernel, i The matching point index.

[0049] S4.5. Sampling is performed according to the weighted RANSAC weights to obtain the basic matrix.

[0050] Furthermore, a weighted sampling probability is derived based on the DNA alignment score and geometric error of the feature point matching pairs. The probability value is proportional to the DNA alignment score and inversely proportional to the geometric error. Eight pairs of matching points are randomly selected according to the weighted probability distribution to construct the basic matrix estimation equation. The basic matrix is ​​solved for the selected matching points using the eight-point method. The distance from all matching points to the extreme lines is used as the basis for inlier judgment. The weighted sampling and matrix estimation process is iteratively performed to retain the basic matrix with the largest number of inliers and the highest weighted inlier score.

[0051] S4.6. Use Zhang Zhengyou calibration method to obtain distortion parameters by photographing the calibration plate, obtain the camera intrinsic parameter matrix, and obtain the essential matrix based on the camera intrinsic parameter matrix.

[0052] Furthermore, a checkerboard calibration plate is used to capture multiple images at different positions and angles, the coordinates of the checkerboard corner points are extracted, and the Zhang Zhengyou calibration method is applied to solve the camera intrinsic parameter matrix. The intrinsic parameter matrix contains the focal length and principal point coordinate parameters, and the radial and tangential distortion coefficients are obtained at the same time. The solved camera intrinsic parameter matrix is ​​used to transform the basic matrix, and the essential matrix is ​​calculated through matrix multiplication. The essential matrix and the basic matrix satisfy the epipolar geometric constraint relationship, and the essential matrix is ​​obtained.

[0053] S5. Encode the basic matrix and the essential matrix into quantum bits, and obtain the projection transformation matrix through quantum annealing.

[0054] S5.1. Normalize the basic matrix and the essential matrix to obtain a normalized vector, and encode the normalized vector into a quantum bit.

[0055] Furthermore, S5.2. Based on the weighted RANSAC sampling weights, the camera intrinsic parameter matrix and the quantum bits, the total Hamiltonian is obtained.

[0056] Furthermore, all elements of the basic matrix and the essential matrix are expanded and spliced ​​into long vectors by rows, and Min-Max normalization is performed on each element in the vector to compress the numerical range to the interval [-1,1]. Then, each normalized floating-point value is mapped to a combination state of two quantum bits, where the interval [-1,-0.5) corresponds to the quantum bit combination |00>, the interval [-0.5,0) corresponds to |01>, the interval [0,0.5) corresponds to |10>, and the interval [0.5,1] corresponds to |11>. The matrix parameter encoding represented by the quantum bit sequence is generated to obtain the total Hamiltonian.

[0057] S5.3. Use quantum annealing to process the total Hamiltonian to obtain a bit string, decode the bit string into continuous matrix parameters according to the linear mapping formula, and reconstruct the basic matrix and the essential matrix.

[0058] Specifically, the expression is, ; in, For the k continuous matrix parameters, For the k The measurement result of the first qubit of the continuous matrix parameter, For the k The measurement result of the second quantum bit of the continuous matrix parameter.

[0059] Furthermore, the total Hamiltonian is initialized on the quantum annealing machine, and the annealing path and temperature scheduling parameters are set; the quantum annealing process is executed to evolve the system to the ground state, and the quantum bits are measured to obtain the optimal bit string; the bit string is divided into groups of two quantum bits, and each group is converted into a continuous value in the interval [-1,1] through a linear mapping formula. The converted values ​​are rearranged according to the original matrix structure to reconstruct the basic matrix and the essential matrix respectively.

[0060] S5.4. Decompose the basic matrix, extract the rotation matrix and translation vector, and combine them with the camera intrinsic parameter matrix to obtain the projection transformation matrix.

[0061] Furthermore, the basic matrix is ​​subjected to singular value decomposition to obtain the product form of three matrices; candidate solutions of the rotation matrix and translation vector are extracted from the decomposition result, and physically feasible rotation matrix and translation vector are selected through triangulation verification, and the rotation matrix and translation vector are combined into the camera extrinsic parameter matrix; the camera intrinsic parameter matrix is ​​multiplied with the extrinsic parameter matrix to obtain the complete projection transformation matrix.

[0062] S6. Perform thin plate spline interpolation deformation on the mixed image group using the projection transformation matrix, perform pyramid fusion in the LAB color space, and perform guided filtering on the fused image to obtain a corrected image.

[0063] S6.1. Perform non-rigid deformation on the mixed image group using a projective transformation matrix to obtain a geometrically aligned image group. Convert the geometrically aligned image group from RGB to LAB color space to separate the luminance and chrominance components.

[0064] Furthermore, a projection transformation matrix is ​​applied to each mixed image, and the RGB value of each pixel in the deformed image is calculated through bilinear interpolation to generate a geometrically aligned image group. The geometrically aligned image group is then converted from the RGB color space to the LAB color space. The conversion process uses the standard RGB to CIE XYZ and then to LAB color gamut mapping formula to separate the luminance component L and chrominance component AB in the LAB color space. The luminance component L represents the brightness and darkness information of the image, and the chrominance component AB contains the color information.

[0065] S6.2. Construct Gaussian pyramids for the LAB color space respectively, obtain the contribution weights of the image regions based on the DNA alignment scores and the quantum bit strings, perform weighted fusion on the Gaussian pyramids, and obtain a fused image.

[0066] Furthermore, two-level Gaussian pyramid decomposition is performed on the luminance component L and chrominance component AB of the LAB color space to generate image representations of different scales. The weight distribution of the image area in the pyramid fusion is obtained based on the DNA comparison score of the feature point matching pair and the parameter confidence after quantum bit decoding. At each pyramid level, the corresponding levels of multiple images are weighted averaged according to the weight distribution. The weight calculation of the luminance component L also takes local contrast into consideration. The weighted fused pyramid levels are merged through the Laplacian pyramid reconstruction algorithm to obtain the fused LAB image.

[0067] S6.3. Perform edge smoothing on the fused image to obtain a filtered LAB image.

[0068] Furthermore, the luminance component of the fused image is used as the guide image, and guided filtering is performed on the three LAB channels respectively. The kernel radius of the guided filtering is dynamically determined according to the image resolution. For an image with a width of W pixels, the kernel radius is W / 100. During the filtering process, the luminance component L is used to calculate the local linear coefficient, and the chrominance component is smoothed according to the same coefficient. The filtered LAB image maintains the original color distribution while eliminating artifacts at the fusion boundary.

[0069] S6.4. Convert the filtered LAB image back to RGB space to obtain a corrected image.

[0070] Furthermore, the luminance and chromaticity components of the LAB image are converted from CIE LAB to CIE XYZ standard color space. The nonlinear transformation formula defined by the International Commission on Illumination is used to transform the converted CIE XYZ tristimulus values ​​into RGB values ​​through a standard transformation matrix to ensure that the color gamut mapping conforms to the sRGB standard. The values ​​that exceed the standard RGB color gamut range are clipped, and the RGB components are limited to the range of 0 to 255 to generate a color-corrected RGB image.

[0071] This embodiment also provides a mixed-shot image correction system based on deep learning, including: an image preprocessing module that performs illumination normalization processing on the mixed-shot image group and constructs a Gaussian pyramid image group; The feature extraction module inputs the Gaussian pyramid image group into the ResNet-34 network to extract multi-scale feature maps and generate spatial position and channel statistics; The DNA encoding module converts spatial position and channel statistical information into DNA sequence fragments using DNA base mapping rules, and uses the Smith-Waterman algorithm to perform biological sequence alignment to obtain feature point matching pairs; The matrix generation module obtains the initial homography matrix based on the feature point matching pairs and obtains the basic matrix and essential matrix through the weighted RANSAC algorithm; The quantum optimization module encodes the basic matrix and the essential matrix into quantum bits and obtains the projection transformation matrix through quantum annealing; The correction module uses the projection transformation matrix to perform thin plate spline interpolation deformation on the mixed image group, and performs pyramid fusion in the LAB color space. The fused image is processed by guided filtering to obtain the corrected image.

[0072] This embodiment also provides a computer device, which is suitable for the mixed-shot image correction method based on deep learning, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the mixed-shot image correction method based on deep learning proposed in the above embodiment.

[0073] The computer device may be a terminal, comprising a processor, memory, a communication interface, a display, and an input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and computer programs. The internal memory provides an environment for the operating system and computer programs stored in the non-volatile storage media. The communication interface of the computer device is used to communicate with external terminals via wired or wireless communication. Wireless communication may be achieved via Wi-Fi, a carrier network, NFC (near-field communication), or other technologies. The display of the computer device may be a liquid crystal display or an electronic ink display. The input device may be a touchscreen overlay on the display, buttons, a trackball, or a touchpad on the computer device housing, or an external keyboard, touchpad, or mouse.

[0074] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the deep learning-based mixed-image correction method proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0075] In summary, the present invention achieves illumination normalization through Retinex decomposition and Gaussian filtering, effectively eliminating exposure differences between different devices and ensuring the input quality of subsequent processing. It converts traditional feature descriptions into biological sequence alignments through DNA base encoding, and combines quantum annealing to optimize geometric parameters, significantly improving the matching accuracy in low-texture areas and the correction effect in scenes with large viewing angle differences. Based on weighted RANSAC and multi-scale pyramid fusion, it achieves sub-pixel matching accuracy.

[0076] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A mixed-shot image correction method based on deep learning, characterized by: include, Perform illumination normalization on the mixed image group and construct a Gaussian pyramid image group; Input the Gaussian pyramid image group into the ResNet-34 network to extract multi-scale feature maps and generate spatial position and channel statistics; The spatial position and channel statistical information are converted into DNA sequence fragments through DNA base mapping rules, and the Smith-Waterman algorithm is used to perform biological sequence alignment to obtain feature point matching pairs; Based on the feature point matching pairs, the initial homography matrix is ​​obtained and the basic matrix and essential matrix are obtained through the weighted RANSAC algorithm; Encode the basic matrix and the essential matrix into quantum bits, and obtain the projection transformation matrix through quantum annealing; The mixed image group is deformed by thin plate spline interpolation using the projection transformation matrix, and pyramid fusion is performed in the LAB color space. The fused image is then subjected to guided filtering to obtain the corrected image.

2. The deep learning-based mixed-shot image correction method according to claim 1, wherein: Perform illumination normalization on the mixed image group and construct a Gaussian pyramid image group. The following steps are included: Receive original images taken under different exposure conditions from different devices, and perform logarithmic domain decomposition on the original images to obtain reflection components and illumination components; The illumination component is filtered using a Gaussian kernel to obtain a smoothed illumination component, and the reflection component is combined with the smoothed illumination component to obtain an illumination-consistent image group; Two-level anti-aliasing downsampling is performed on the image group with consistent illumination to construct a Gaussian pyramid image group.

3. The deep learning-based mixed-shot image correction method according to claim 2, wherein: Input the Gaussian pyramid image group into the ResNet-34 network to extract multi-scale feature maps and generate spatial position and channel statistics, including the following steps: Input each layer of the pyramid image into the ResNet-34 network, output a multi-scale feature map, calculate the pixel gradient response value on the multi-scale feature map to determine the corner point position, and obtain a coordinate set; The mean and variance statistics are obtained at the two-dimensional spatial position of the multi-scale feature map to obtain channel statistical information.

4. The deep learning-based mixed-shot image correction method according to claim 3, wherein: The spatial position and channel statistical information are converted into DNA sequence fragments through DNA base mapping rules, and the Smith-Waterman algorithm is used to perform biological sequence alignment to obtain feature point matching pairs, including the following steps: The two-dimensional spatial position, channel statistics and multi-scale feature maps are spliced ​​and compressed through MLP to obtain the visual and geometric characteristics of the local area of ​​the image; Normalize the visual and geometric characteristics of the local area of ​​the image to obtain an integer vector, and convert the integer vector into a DNA sequence fragment according to the base rule; The Smith-Waterman algorithm is used to perform biological sequence alignment on the DNA sequence fragments of two images with consistent illumination to obtain feature point matching pairs.

5. The deep learning-based mixed-shot image correction method according to claim 4, wherein: Based on the feature point matching pairs, the initial homography matrix is ​​obtained and the basic matrix and the essential matrix are obtained through the weighted RANSAC algorithm, including the following steps: Perform zero-mean scaling on the feature point matching pairs to obtain normalized coordinates and transformation matrix; Construct the least squares equation based on the normalized coordinates and decompose the singular values ​​to obtain the initial homography matrix; The DNA alignment score is extracted from the feature point matching pairs, and the projection deviation is selected from the initial homography matrix to obtain the geometric error; Calculate weighted RANSAC weights based on DNA alignment score and projection deviation; Sampling is performed according to the weighted RANSAC weight to obtain the basic matrix; By shooting the calibration plate and using Zhang Zhengyou calibration method to obtain the distortion parameters, the camera intrinsic parameter matrix is ​​obtained. Based on the camera intrinsic parameter matrix, the essential matrix is ​​obtained.

6. The deep learning-based mixed-shot image correction method according to claim 5, wherein: Encode the basic matrix and the essential matrix into quantum bits, and obtain the projection transformation matrix through quantum annealing, which includes the following steps: Normalizing the fundamental matrix and the essential matrix to obtain a normalized vector, and encoding the normalized vector into a quantum bit; Based on the weighted RANSAC sampling weights, the camera intrinsic parameter matrix and the quantum bits, the total Hamiltonian is obtained; Use quantum annealing to process the total Hamiltonian to obtain a bit string, decode the bit string into continuous matrix parameters according to the linear mapping formula, and reconstruct the basic matrix and essential matrix; Decompose the basic matrix, extract the rotation matrix and translation vector, and combine them with the camera intrinsic parameter matrix to obtain the projection transformation matrix.

7. The deep learning-based mixed-shot image correction method according to claim 6, wherein: The mixed image group is deformed by thin plate spline interpolation using the projection transformation matrix, and pyramid fusion is performed in the LAB color space. The fused image is subjected to guided filtering to obtain a corrected image, which includes the following steps: The mixed image group is non-rigidly deformed using a projection transformation matrix to obtain a geometrically aligned image group, which is then converted from RGB to LAB color space to separate the luminance and chrominance components. Gaussian pyramids are constructed for the LAB color space respectively. The contribution weights of the image regions are obtained based on the DNA alignment scores and quantum bit strings. The Gaussian pyramids are weightedly fused to obtain a fused image. Perform edge smoothing on the fused image to obtain the filtered LAB image; Convert the filtered LAB image back to RGB space to obtain the rectified image.

8. A deep learning-based mixed-shot image correction system, based on the deep learning-based mixed-shot image correction method according to any one of claims 1 to 7, characterized in that: include, The image preprocessing module performs illumination normalization on the mixed image group and constructs a Gaussian pyramid image group; The feature extraction module inputs the Gaussian pyramid image group into the ResNet-34 network to extract multi-scale feature maps and generate spatial position and channel statistics; The DNA encoding module converts spatial position and channel statistical information into DNA sequence fragments using DNA base mapping rules, and uses the Smith-Waterman algorithm to perform biological sequence alignment to obtain feature point matching pairs; The matrix generation module obtains the initial homography matrix based on the feature point matching pairs and obtains the basic matrix and essential matrix through the weighted RANSAC algorithm; The quantum optimization module encodes the basic matrix and the essential matrix into quantum bits and obtains the projection transformation matrix through quantum annealing; The correction module uses the projection transformation matrix to perform thin plate spline interpolation deformation on the mixed image group, and performs pyramid fusion in the LAB color space. The fused image is processed by guided filtering to obtain the corrected image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the mixed-shot image correction method based on deep learning are implemented in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the mixed-shot image correction method based on deep learning according to any one of claims 1 to 7 are implemented.

Images