A deep learning-based mixed image correction method and system

Abstract

The application discloses a kind of based on deep learning's mixed image correction method and system, it is related to computer vision field, including, mixed image group carries out illumination normalization processing and constructs Gaussian pyramid image group;Gaussian pyramid image group is input ResNet-34 network and extracts multi-scale feature map, and generates space position and channel statistical information;Space position and channel statistical information are converted into DNA sequence fragment by DNA base mapping rule.The application realizes illumination normalization by Retinex decomposition and Gaussian filtering, effectively eliminates the exposure difference of different devices, guarantees the input quality of subsequent processing, converts traditional feature description into biological sequence alignment by DNA base coding, combines quantum annealing to optimize geometric parameters, significantly improves the matching accuracy of low-texture area and the correction effect of large-angle difference scene, based on weighted RANSAC and multi-scale pyramid fusion, realizes sub-pixel level matching accuracy.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and in particular to a method and system for correcting mixed images based on deep learning. Background Technology

[0002] With the popularization of multi-camera mobile devices and drone aerial photography technology, the correction and fusion technology of mixed images has become a research hotspot in the field of computer vision. Traditional methods are mainly based on feature point matching and geometric transformation models, such as SIFT combined with RANSAC algorithm to achieve image registration. Such techniques extract local feature descriptors, use nearest neighbor search to match key points, and estimate homography matrix or fundamental matrix through iterative optimization. Existing deep learning solutions further introduce convolutional neural networks to extract higher-order features, which improves the matching robustness in weak texture regions. However, when dealing with large perspective differences, inconsistent lighting, or dynamic blur scenes, existing technologies still rely on manually designed feature descriptors and fixed optimization strategies, failing to fully explore the correlation of cross-modal data.

[0003] Existing methods suffer from insufficient cross-modal feature comparability during the feature matching stage. When there are significant differences in illumination or resolution mismatch in mixed images, traditional feature descriptors are unable to establish accurate similarity measures, leading to a decrease in matching accuracy. Under low illumination conditions, gradient information is easily affected by noise, and hand-designed binary descriptors lose multi-scale statistical properties. The RANSAC algorithm uses a uniform sampling strategy, which does not consider the semantic correlation between feature points and may ignore high-confidence matching pairs. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a deep learning-based mixed-image correction method to address the problems of inaccurate similarity measurement and decreased matching accuracy caused by differences in illumination and resolution mismatch in cross-modal image feature matching.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a method for correcting mixed images based on deep learning, which includes performing illumination normalization processing on a group of mixed images and constructing a Gaussian pyramid image group.

[0008] The Gaussian pyramid image set is input into the ResNet-34 network to extract multi-scale feature maps and generate spatial location and channel statistics information;

[0009] The spatial position and channel statistical information are converted into a DNA sequence fragment through a DNA base mapping rule, a Smith-Waterman algorithm is used for biological sequence alignment, and a feature point matching pair is obtained;

[0010] An initial homography matrix is derived based on the feature point matching pair, and a weighted RANSAC algorithm is used to obtain a fundamental matrix and an essential matrix;

[0011] The fundamental matrix and the essential matrix are encoded into qubits, and a projection transformation matrix is obtained through quantum annealing;

[0012] The projection transformation matrix is used for thin plate spline interpolation deformation of the mixed image group, pyramid fusion is performed in the LAB color space, guided filter processing is performed on the fused image, and a corrected image is obtained

[0013] As a preferred scheme of the mixed image correction method based on deep learning, wherein: the mixed image group is subjected to illumination normalization processing and a Gaussian pyramid image group is constructed, including the following steps,

[0014] Receiving original images taken under different device exposure conditions, and performing logarithmic domain decomposition on the original images to obtain reflection components and illumination components;

[0015] The illumination components are filtered using a Gaussian kernel to obtain smoothed illumination components, and the reflection components are combined with the smoothed illumination components to obtain an illumination consistent image group;

[0016] The illumination consistent image group is subjected to two-stage anti-aliasing down-sampling to construct a Gaussian pyramid image group.

[0017] As a preferred scheme of the mixed image correction method based on deep learning, wherein: the Gaussian pyramid image group is input into a ResNet-34 network to extract multi-scale feature maps and generate spatial position and channel statistical information, including the following steps,

[0018] Each layer of pyramid image is input into the ResNet-34 network, and multi-scale feature maps are output, the pixel point gradient response value is calculated on the multi-scale feature maps to determine the corner point position, and a coordinate set is obtained;

[0019] The mean and variance statistics are derived in the two-dimensional spatial position of the multi-scale feature map to obtain the channel statistical information.

[0020] As a preferred scheme of the mixed image correction method based on deep learning, wherein: the spatial position and channel statistical information are converted into a DNA sequence fragment through a DNA base mapping rule, a Smith-Waterman algorithm is used for biological sequence alignment, and a feature point matching pair is obtained, including the following steps,

[0021] The two-dimensional spatial position, channel statistical information and multi-scale feature map are spliced and compressed through an MLP to obtain visual and geometric characteristics of a local region of an image;

[0022] The visual and geometric characteristics of the local region of the image are normalized to obtain an integer vector, and the integer vector is converted into a DNA sequence fragment according to a base rule;

[0023] The DNA sequence fragments of the two images with consistent illumination are subjected to biological sequence alignment by using a Smith-Waterman algorithm to obtain a feature point matching pair.

[0024] As a preferred scheme of the mixed image correction method based on deep learning, wherein: based on the feature point matching pair, an initial homography matrix is obtained, and a fundamental matrix and an essential matrix are obtained by using a weighted RANSAC algorithm, including the following steps,

[0025] The feature point matching pair is subjected to zero-mean scaling to obtain normalized coordinates and a transformation matrix;

[0026] Based on the normalized coordinates, a least square equation is constructed, and singular values are decomposed to obtain the initial homography matrix;

[0027] The DNA alignment score is extracted from the feature point matching pair, and the projection deviation is selected from the initial homography matrix to obtain a geometric error;

[0028] Based on the DNA alignment score and the projection deviation, a weighted RANSAC weight is calculated;

[0029] The fundamental matrix is obtained by sampling according to the weighted RANSAC weight;

[0030] The distortion parameters are obtained by using the Zhang Zhengyou calibration method by shooting a calibration board to obtain an intrinsic matrix of the camera, and based on the intrinsic matrix of the camera, the essential matrix is obtained.

[0031] As a preferred scheme of the mixed image correction method based on deep learning, wherein: the fundamental matrix and the essential matrix are encoded into qubits, and a projection transformation matrix is obtained by quantum annealing, including the following steps,

[0032] The fundamental matrix and the essential matrix are subjected to normalization processing to obtain a normalized vector, and the normalized vector is encoded into qubits;

[0033] Based on the weighted RANSAC sampling weight, the intrinsic matrix of the camera and the qubits, a total Hamiltonian is obtained;

[0034] The total Hamiltonian is processed using quantum annealing to obtain a bit string, the bit string is decoded into a continuous matrix parameter according to a linear mapping formula, and the base matrix and the essential matrix are reconstructed;

[0035] The base matrix is decomposed, the rotation matrix and the translation vector are extracted, and the camera intrinsic matrix is combined to obtain a projection transformation matrix.

[0036] As a preferred scheme of the mixed image correction method based on deep learning, wherein: the projection transformation matrix is used for thin plate spline interpolation deformation of the mixed image group, pyramid fusion is carried out in the LAB color space, guided filter processing is carried out on the fused image, and a corrected image is obtained, including the following steps,

[0037] The projection transformation matrix is used for non-rigid deformation of the mixed image group to obtain a geometrically aligned image group, and the geometrically aligned image group is converted from RGB to LAB color space to separate the luminance and chrominance components;

[0038] Gaussian pyramids are constructed in the LAB color space, the contribution weight of the image region is obtained based on the DNA alignment score and the quantum bit string, the Gaussian pyramids are weighted and fused to obtain a fused image;

[0039] The fused image is subjected to edge smoothing processing to obtain a filtered LAB image;

[0040] The filtered LAB image is converted back to the RGB space to obtain a corrected image.

[0041] In a second aspect, the application provides a mixed image correction system based on deep learning, comprising,

[0042] The image preprocessing module performs illumination normalization processing on the mixed image group and constructs a Gaussian pyramid image group;

[0043] The feature extraction module inputs the Gaussian pyramid image group into the ResNet-34 network to extract multi-scale feature maps and generate spatial position and channel statistical information;

[0044] The DNA encoding module converts the spatial position and channel statistical information into a DNA sequence fragment through a DNA base mapping rule, and adopts the Smith-Waterman algorithm for biological sequence alignment to obtain a feature point matching pair;

[0045] The matrix generation module obtains an initial homography matrix based on the feature point matching pair and obtains a base matrix and an essential matrix through a weighted RANSAC algorithm;

[0046] The quantum optimization module encodes the base matrix and the essential matrix into quantum bits, and obtains a projection transformation matrix through quantum annealing;

[0047] The correction module performs thin plate spline interpolation deformation on the mixed image group by using the projection transformation matrix, performs pyramid fusion in the LAB color space, performs guided filtering processing on the fused image, and obtains a corrected image.

[0048] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and wherein the computer program, when executed by the processor, implements any step of the deep learning-based mixed image correction method according to the first aspect of the present application.

[0049] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any step of the deep learning-based mixed image correction method according to the first aspect of the present application.

[0050] The present application has the following beneficial effects: illumination normalization is achieved through Retinex decomposition and Gaussian filtering, effectively eliminating exposure differences of different devices and ensuring input quality for subsequent processing; traditional feature description is converted into biological sequence alignment through DNA base coding, and geometric parameters are optimized through quantum annealing, significantly improving matching accuracy in low-texture areas and correction effect in large-angle difference scenes; and sub-pixel level matching accuracy is achieved based on weighted RANSAC and multi-scale pyramid fusion. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0052] Fig. 1 Flowchart of the deep learning-based mixed image correction method;

[0053] Fig. 2 Schematic diagram of the deep learning-based mixed image correction system;

[0054] Fig. 3 Schematic diagram of the corrected image;

[0055] Fig. 4 Schematic diagram of the feature point matching pair. DETAILED DESCRIPTION

[0056] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings of the specification.

[0057] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be practiced without the specific details. In other instances, well-known methods have not been described in detail in order not to unnecessarily obscure aspects of the present application. The present application is not limited to the embodiments described herein which can be practiced with or without the same.

[0058] It is to be understood that the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. "Coupled" is defined as connected, whether directly or indirectly, with or without wiring or cable, or the like, or two or more elements are "coupled" if each is connected, if directly or through intervening elements, to the other or others under such a manner that a link exists between them.

[0059] Reference will now be made to Figs. 1-4 For one embodiment of the present application, the embodiment provides a deep learning-based multi-capture image correction method, comprising the following steps:

[0060] S1, performing illumination normalization processing on the multi-capture image group and constructing a Gaussian pyramid image group.

[0061] S1.1, receiving original images captured under different device exposure conditions, and performing log domain decomposition on the original images to obtain reflection components and illumination components.

[0062] Further, the original images captured under different device exposure conditions are decomposed in the log domain, and the reflection components and the illumination components are separated after the original images are converted to the log space. The reflection components represent the inherent properties of the scene, and the illumination components reflect the environmental lighting conditions.

[0063] S1.2, filtering the illumination components using a Gaussian kernel to obtain smoothed illumination components, and combining the reflection components with the smoothed illumination components to obtain an illumination-consistent image group.

[0064] Further, a two-dimensional Gaussian filter function is applied to the illumination components, and the standard deviation of the Gaussian kernel is dynamically determined according to the image size. For an image with a width of W pixels and a height of H pixels, the standard deviation of the Gaussian kernel is max(W, H) / 50. The Gaussian filtering is implemented using separable convolution, i.e., one-dimensional Gaussian convolution is performed in the horizontal direction first, and then one-dimensional Gaussian convolution is performed in the vertical direction to obtain the smoothed illumination components. The smoothed illumination components are combined with the reflection components in the linear domain. The combination method is to multiply the exponential operations of the illumination components and the reflection components to obtain the illumination-consistent image group. The pixel value range of each image in the illumination-consistent image group is normalized to [0, 255], and the spatial resolution and the number of color channels of the original image remain unchanged.

[0065] S1.3, performing two-stage anti-aliasing down-sampling on the illumination-consistent image group to construct a Gaussian pyramid image group.

[0066] Further, a first level down-sampling is performed on each image in the illumination consistent image group, and a Gaussian kernel with a standard deviation of 1.0 is used to smooth the image before the down-sampling, so that the image size is reduced to 1 / 2 of the original image. Then, a second level down-sampling is performed on the image obtained by the first level down-sampling, and a Gaussian kernel with a standard deviation of 1.0 is used to smooth the image after the down-sampling, so that the image size is reduced to 1 / 4 of the original image. Each level down-sampling is implemented by using a bilinear interpolation method to ensure the spatial continuity of the image. Finally, a three-layer Gaussian pyramid image group containing the original scale, 1 / 2 scale and 1 / 4 scale images is obtained.

[0067] S2, input the Gaussian pyramid image group into the ResNet-34 network to extract multi-scale feature maps and generate spatial position and channel statistical information.

[0068] S2.1, input each layer of the pyramid image into the ResNet-34 network to output multi-scale feature maps, calculate the pixel point gradient response value on the multi-scale feature maps to determine the corner point position, and obtain a coordinate set.

[0069] Specifically, the expression is,

[0070] ;

[0071] wherein, R is the pixel point gradient response value, M is the multi-scale feature map, is an empirical constant, is the determinant of the multi-scale feature map, is the trajectory of the multi-scale feature map.

[0072] Further, the original scale, 1 / 2 scale and 1 / 4 scale images in the Gaussian pyramid image group are respectively input into the ResNet-34 network. The ResNet-34 network adds a deformable convolution module after the conv3_x and conv4_x layers. The ResNet-34 network outputs multi-scale feature maps, and the feature map sizes are 1 / 8, 1 / 16 and 1 / 32 of the input image, respectively. The Harris corner point response value is calculated on the multi-scale feature maps. The first 500 points with the largest response value are selected as the corner point position on each layer of the feature map. The coordinates (x, y) of the corner point in the pyramid level l are recorded to form a coordinate set.

[0073] S2.2, obtain the mean and variance statistics in the two-dimensional spatial position of the multi-scale feature map to obtain the channel statistical information.

[0074] Further, the multi-scale feature maps output by the ResNet-34 network are calculated in a 5*5 neighborhood range of each feature point, and for each channel of the 128-dimensional feature map, the average value of all pixel values in the neighborhood is extracted as the mean statistical quantity, and the square average of the deviation of the pixel value from the mean is obtained as the variance statistical quantity. The range of the mean statistical quantity and the variance statistical quantity is strictly limited to a 5*5 rectangular region around the two-dimensional spatial position of the feature map. At the same time, the maximum response value and the minimum response value of the feature point are recorded to form a five-dimensional channel statistical information vector containing the mean, variance, maximum value, minimum value and median.

[0075] S3, convert the spatial position and channel statistical information into a DNA sequence fragment through the DNA base mapping rule, and perform biological sequence alignment using the Smith-Waterman algorithm to obtain a feature point matching pair.

[0076] S3.1, splice the two-dimensional spatial position, channel statistical information and multi-scale feature map, and compress through MLP to obtain the visual and geometric characteristics of the local region of the image.

[0077] Further, the two-dimensional spatial coordinates of the feature point, the pyramid level, the statistical information vector and the 128-dimensional multi-scale feature map descriptor are spliced in the channel dimension to form a 136-dimensional composite vector, and then the composite vector is input into a two-layer fully connected neural network. The first fully connected layer maps the 136-dimensional input to 64 dimensions using the ReLU activation function, and the second fully connected layer maps the 64-dimensional input to 32 dimensions using the linear activation function to obtain the visual and geometric characteristics of the local region of the image.

[0078] S3.2, normalize the visual and geometric characteristics of the local region of the image to obtain an integer vector, and convert the integer vector into a DNA sequence fragment according to the base rule.

[0079] Further, the visual and geometric characteristics of the local region of the image are subjected to Min-Max normalization, and the numerical range is compressed to the interval [0, 3]. Then the normalized floating point values are converted to integer values by rounding, and an integer vector is obtained. The integer values are converted into DNA base characters according to the base mapping rule, wherein 0 corresponds to base A, 1 corresponds to base T, 2 corresponds to base C, and 3 corresponds to base G. The 32 base characters are spliced in order to form a DNA sequence fragment.

[0080] S3.3, biological sequence alignment is performed on the DNA sequence fragments of the two images with consistent illumination using the Smith-Waterman algorithm to obtain a feature point matching pair.

[0081] Further, first, a 32x32 score matrix is established, the matching score is set to +2, the mismatch score is set to -1, and the gap penalty is set to a linear function g(k)=-2k. The optimal alignment path is traced back from the lower right corner to the upper left corner. Sequences with alignment scores higher than a threshold (for example, scores >40) are reserved as candidate matches. The indices of the DNA sequence fragments are mapped back to the original feature point coordinates to obtain a preliminary set of feature point matching pairs. Finally, the spatial consistency of the matching pairs is verified, and abnormal matches with a re-projection error greater than 3 pixels are removed to form the feature point matching pairs.

[0082] S4. Based on the feature point matching pairs, an initial homography matrix is obtained, and a weighted RANSAC algorithm is used to obtain the fundamental matrix and the essential matrix.

[0083] S4.1. The feature point matching pairs are zero-mean scaled to obtain normalized coordinates and a transformation matrix.

[0084] Further, the arithmetic mean values of the x coordinates and y coordinates of all feature points in the matching pair set are obtained, i.e., the x coordinate mean value μ_x=mean(x_i) and the y coordinate mean value μ_y=mean(y_i), where x_i and y_i represent the horizontal and vertical coordinate values of the i-th feature point. The original coordinates (x, y) of each feature point are subtracted from the coordinate mean values (μ_x, μ_y) to achieve zero centralization. Then, the standard deviations (σ_x, σ_y) of the coordinates are calculated. The zero-centered coordinates are divided by the standard deviations to obtain normalized coordinates. The mean values and standard deviations during scaling are recorded to obtain normalized coordinates and a transformation matrix.

[0085] S4.2. A least squares equation is constructed based on the normalized coordinates, and singular value decomposition is performed to obtain an initial homography matrix.

[0086] Further, an overdetermined linear equation set is constructed using the normalized coordinates. Two linear equations are generated for each matching point to constrain the element relationship of the homography matrix. Then, singular value decomposition is performed on the constructed coefficient matrix. The last column of the right singular matrix is taken as the solution vector of the homography matrix. The solution vector is rearranged into a three-row-three-column matrix form to obtain the homography matrix in the normalized coordinate system. The normalized homography matrix is converted back to the original pixel coordinate system by left multiplication and right multiplication of the inverse matrix of the normalization transformation matrix to obtain the initial homography matrix.

[0087] S4.3. The DNA alignment score is extracted from the feature point matching pairs, and the projection deviation is selected from the initial homography matrix to obtain the geometric error.

[0088] Furthermore, the DNA alignment score of each pair of feature points is read from the matching results generated by the Smith-Waterman algorithm. The DNA alignment score reflects the degree of sequence similarity. The coordinates of the feature points of the first image are projected onto the coordinate system of the second image using the initial homography matrix. The Euclidean distance between the projected point and the actual matching point is obtained as the projection deviation. The DNA alignment score is combined with the projection deviation. The DNA alignment score represents the reliability of feature similarity, and the projection deviation represents the spatial geometric consistency. Together, they constitute the geometric error evaluation index.

[0089] S4.4 Calculate the weighted RANSAC weights based on DNA alignment scores and projection bias.

[0090] Specifically, the expression is,

[0091] ;

[0092] in, For the first i The weighted RANSAC weights of the matching points, For the first i DNA alignment scores at matching points The maximum value in the DNA alignment score. For the first i Geometric error of the matching point, The standard deviation of the Gaussian kernel. i For the index of the matching point.

[0093] S4.5. Sample according to the weighted RANSAC weights to obtain the basic matrix.

[0094] Furthermore, a weighted sampling probability is derived based on the DNA alignment score and geometric error of the feature point matching pairs. The probability value is directly proportional to the DNA alignment score and inversely proportional to the geometric error. Eight pairs of matching points are randomly selected according to the weighted probability distribution to construct the basic matrix estimation equation. The basic matrix is ​​solved using the eight-point method for the selected matching points. The distance from all matching points to the epipolar line is used as the basis for judging inliers. The weighted sampling and matrix estimation process is iteratively executed, and the basic matrix with the largest number of inliers and the highest weighted inlier score is retained.

[0095] S4.6. By shooting the calibration board and using Zhang Zhengyou's calibration method, the distortion parameters are obtained, the camera intrinsic parameter matrix is ​​obtained, and the essential matrix is ​​obtained based on the camera intrinsic parameter matrix.

[0096] Further, multiple images are taken at different positions and angles using a chessboard calibration board, corner point coordinates of the chessboard are extracted, Zhang Zhengyou calibration method is applied to solve the camera intrinsic matrix, the intrinsic matrix includes focal length and principal point coordinate parameters, radial and tangential distortion coefficients are obtained at the same time, the solved camera intrinsic matrix is used to convert the fundamental matrix, the essential matrix is calculated through matrix multiplication, the essential matrix and the fundamental matrix satisfy the epipolar geometry constraint relationship, and the essential matrix is obtained.

[0097] S5.1, the fundamental matrix and the essential matrix are normalized to obtain a normalized vector, and the normalized vector is encoded into quantum bits.

[0098] S5.1, the fundamental matrix and the essential matrix are normalized to obtain a normalized vector, and the normalized vector is encoded into quantum bits.

[0099] Further,

[0100] S5.2, based on the weighted RANSAC sampling weight, the camera intrinsic matrix and the quantum bits, a total Hamiltonian is obtained.

[0101] Further, all elements of the fundamental matrix and the essential matrix are expanded and spliced into a long vector by row, Min-Max normalization is performed on each element in the vector, the numerical range is compressed to the interval [-1, 1], then each normalized floating point value is mapped to the combined state of two quantum bits, wherein the interval [-1, -0.5) corresponds to the quantum bit combination |00>, the interval [-0.5, 0) corresponds to |01>, the interval [0, 0.5) corresponds to |10>, and the interval [0.5, 1] corresponds to |11>, a matrix parameter encoding represented by a quantum bit sequence is generated, and a total Hamiltonian is obtained.

[0102] S5.3, the total Hamiltonian is processed using quantum annealing to obtain a bit string, the bit string is decoded into a continuous matrix parameter according to the linear mapping formula, and the fundamental matrix and the essential matrix are reconstructed.

[0103] Specifically, the expression is,

[0104] ;

[0105] Wherein, is the first quantum bit of the m-th continuous matrix parameter, k is the first quantum bit of the m-th continuous matrix parameter, is the second quantum bit of the m-th continuous matrix parameter, k is the second quantum bit of the m-th continuous matrix parameter. k is the second quantum bit of the m-th continuous matrix parameter.

[0106] ​Further, initialize the total Hamiltonian on the quantum annealer, set the annealing path and temperature schedule parameters; perform the quantum annealing process to evolve the system to the ground state, measure the quantum bits to obtain the optimal bit string; divide the bit string into groups of two qubits, convert each group to a continuous value in the interval [-1, 1] through a linear mapping formula, rearrange the converted values according to the original matrix structure, and reconstruct the fundamental matrix and the essential matrix respectively.

[0107] S5.4, decompose the fundamental matrix, extract the rotation matrix and the translation vector, and combine them with the camera intrinsic matrix to obtain the projection transformation matrix.

[0108] Further, singular value decomposition is performed on the fundamental matrix to obtain the product form of the three matrices; candidate solutions of the rotation matrix and the translation vector are extracted from the decomposition results, and the physically feasible rotation matrix and translation vector are selected through triangularization verification; the rotation matrix and the translation vector are combined into the camera extrinsic matrix; and the camera intrinsic matrix and the extrinsic matrix are multiplied to obtain the complete projection transformation matrix.

[0109] S6, use the projection transformation matrix to perform thin plate spline interpolation deformation on the mixed image group, perform pyramid fusion in the LAB color space, and perform guided filtering on the fused image to obtain the corrected image.

[0110] S6.1, use the projection transformation matrix to perform non-rigid deformation on the mixed image group to obtain a geometrically aligned image group, convert the geometrically aligned image group from RGB to LAB color space, and separate the luminance and chrominance components.

[0111] Further, apply the projection transformation matrix to each mixed image, calculate the RGB values of each pixel point of the deformed image through bilinear interpolation, generate a geometrically aligned image group, and then convert the geometrically aligned image group from RGB color space to LAB color space. The conversion process uses the standard RGB to CIE XYZ and then to LAB color gamut mapping formula. In the LAB color space, separate the luminance component L and the chrominance component AB. The luminance component L represents the brightness information of the image, and the chrominance component AB contains color information.

[0112] S6.2, construct a Gaussian pyramid in the LAB color space, obtain the contribution weight of the image region based on the DNA alignment score and the quantum bit string, perform weighted fusion on the Gaussian pyramid, and obtain the fused image.

[0113] Further, two-level Gaussian pyramid decomposition is performed on the luminance component L and the chrominance component AB of the LAB color space respectively to generate image representations of different scales, and according to the DNA alignment score of the feature point matching pair and the parameter confidence after quantum bit decoding, the weight distribution of the image region in the pyramid fusion is obtained; on each pyramid level, the corresponding levels of multiple images are weighted and averaged according to the weight distribution, wherein the weight calculation of the luminance component L considers the local contrast, and the pyramid levels after weighted fusion are combined through the Laplacian pyramid reconstruction algorithm to obtain the fused LAB image.

[0114] S6.3, edge smoothing processing is performed on the fused image to obtain a filtered LAB image.

[0115] Further, the luminance component of the fused image is taken as a guide image, and guide filtering is performed on the three channels of LAB respectively, and the kernel radius of the guide filtering is dynamically determined according to the image resolution, and for an image with a width of W pixels, the kernel radius is W / 100, and in the filtering process, the luminance component L is used to calculate the local linear coefficient, and the chrominance component is smoothed according to the same coefficient, and the filtered LAB image retains the original color distribution while eliminating the artifacts at the fusion boundary.

[0116] S6.4, the filtered LAB image is converted back to the RGB space to obtain a corrected image.

[0117] Further, the luminance component and the chrominance component of the LAB image are subjected to standard color space conversion from CIE LAB to CIE XYZ, the converted CIE XYZ three stimulus values are converted into RGB values through a standard conversion matrix, the color gamut mapping is ensured to comply with the sRGB standard, the values exceeding the standard RGB color gamut range are clipped, the RGB components are limited within the range of 0 to 255, and a color-corrected RGB image is generated.

[0118] The embodiment also provides a mixed shot image correction system based on deep learning, comprising: an image preprocessing module, which performs illumination normalization processing on a mixed shot image group and constructs a Gaussian pyramid image group;

[0119] A feature extraction module inputs the Gaussian pyramid image group into a ResNet-34 network to extract multi-scale feature maps and generate spatial position and channel statistical information;

[0120] A DNA encoding module converts the spatial position and channel statistical information into DNA sequence fragments through a DNA base mapping rule, and adopts a Smith-Waterman algorithm to perform biological sequence alignment to obtain a feature point matching pair;

[0121] The matrix generating module obtains an initial homography matrix based on the matching of the feature points, and obtains a fundamental matrix and an essential matrix through a weighted RANSAC algorithm;

[0122] The quantum optimization module encodes the fundamental matrix and the essential matrix into quantum bits, and obtains a projection transformation matrix through quantum annealing.

[0123] The correction module performs thin plate spline interpolation deformation on the mixed image group by using the projection transformation matrix, performs pyramid fusion in the LAB color space, performs guided filter processing on the fused image, and obtains a corrected image.

[0124] The embodiment also provides a computer device suitable for the deep learning-based mixed image correction method, which comprises a memory and a processor.

[0125] The computer device can be a terminal, and the computer device comprises a processor, a memory, a communication interface, a display screen and an input device which are connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, a trackball or a touchpad arranged on the shell of the computer device. In addition, the input device can also be an external keyboard, a touchpad or a mouse, etc.

[0126] The embodiment also provides a storage medium on which a computer program is stored, the program being executed by a processor to implement the method for correcting a mixed shot image based on deep learning proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.

[0127] To sum up, the method for correcting a mixed shot image based on deep learning is implemented by the Retinex decomposition and the Gaussian filtering to realize illumination normalization, effectively eliminate exposure differences of different devices, guarantee input quality of subsequent processing, convert a traditional feature description into a biological sequence alignment by DNA base coding, combine quantum annealing to optimize geometric parameters, significantly improve matching accuracy of a low-texture region and correction effect of a large-angle difference scene, and realize sub-pixel level matching accuracy based on weighted RANSAC and multi-scale pyramid fusion.

[0128] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application but not limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application, and all of them should be covered in the scope of the claims of the present application.

Claims

1. A deep learning-based mixed image correction method, characterized in that: Comprising, illumination normalization processing is performed on the mixed image group, and a Gaussian pyramid image group is constructed; the Gaussian pyramid image group is input into a ResNet-34 network to extract multi-scale feature maps and generate spatial position and channel statistical information; the spatial position and channel statistical information are converted into DNA sequence fragments through DNA base mapping rules, biological sequence alignment is performed using the Smith-Waterman algorithm, and a feature point matching pair is obtained; an initial homography matrix is obtained based on the feature point matching pair, and a fundamental matrix and an essential matrix are obtained through a weighted RANSAC algorithm; the fundamental matrix and the essential matrix are encoded into qubits, and a projection transformation matrix is obtained through quantum annealing; thin-plate spline interpolation deformation is performed on the mixed image group using the projection transformation matrix, pyramid fusion is performed in the LAB color space, guided filtering is performed on the fused image, and a corrected image is obtained; the spatial position and channel statistical information are converted into DNA sequence fragments through DNA base mapping rules, biological sequence alignment is performed using the Smith-Waterman algorithm, and a feature point matching pair is obtained, including the following steps, two-dimensional spatial position, channel statistical information, and multi-scale feature maps are spliced and compressed through MLP to obtain the visual and geometric characteristics of the local region of the image; the visual and geometric characteristics of the local region of the image are normalized to obtain an integer vector, which is converted into a DNA sequence fragment according to the base rules; biological sequence alignment is performed on the DNA sequence fragments of the two image groups with consistent illumination using the Smith-Waterman algorithm to obtain a feature point matching pair; an initial homography matrix is obtained based on the feature point matching pair, and a fundamental matrix and an essential matrix are obtained through a weighted RANSAC algorithm, including the following steps, the feature point matching pair is scaled to zero mean to obtain normalized coordinates and a transformation matrix; a least squares equation is constructed based on the normalized coordinates, and singular values are decomposed to obtain an initial homography matrix; the DNA alignment score is extracted from the feature point matching pair, and the projection deviation is selected from the initial homography matrix to obtain a geometric error; based on the DNA alignment score and the projection deviation, the weighted RANSAC weight is calculated; sampling is performed according to the weighted RANSAC weight to obtain a fundamental matrix; distortion parameters are obtained by shooting a calibration board using the Zhang Zhengyou calibration method to obtain a camera intrinsic matrix, and the essential matrix is obtained based on the camera intrinsic matrix; the fundamental matrix and the essential matrix are encoded into qubits, and a projection transformation matrix is obtained through quantum annealing, including the following steps, the fundamental matrix and the essential matrix are normalized to obtain a normalized vector, which is encoded into qubits; based on the weighted RANSAC sampling weight, the camera intrinsic matrix, and the qubits, a total Hamiltonian is obtained; the total Hamiltonian is processed using quantum annealing to obtain a bit string, which is decoded into a continuous matrix parameter according to a linear mapping formula to reconstruct the fundamental matrix and the essential matrix; the fundamental matrix is decomposed to extract a rotation matrix and a translation vector, which are combined with the camera intrinsic matrix to obtain a projection transformation matrix. 2.The deep learning-based refashioning method of claim 1, wherein: performing illumination normalization on the mixed image set and constructing a Gaussian pyramid image set, comprising the following steps, receiving original images taken under different device exposure conditions, and performing log domain decomposition on the original images to obtain reflection components and illumination components; performing filtering processing on the illumination components using a Gaussian kernel to obtain smoothed illumination components, and combining the reflection components with the smoothed illumination components to obtain an illumination-consistent image set; performing two-level anti-aliasing down-sampling on the illumination-consistent image set to construct a Gaussian pyramid image set. 3.The deep learning-based image blending correction method of claim 2, wherein: inputting the Gaussian pyramid image set into a ResNet-34 network to extract multi-scale feature maps and generate spatial position and channel statistical information, comprising the following steps, inputting each layer of the pyramid image into the ResNet-34 network to output multi-scale feature maps, calculating pixel point gradient response values on the multi-scale feature maps to determine corner point positions, and obtaining a coordinate set; obtaining mean and variance statistics in the two-dimensional spatial position of the multi-scale feature maps to obtain channel statistical information. 4.The deep learning-based image blending correction method of claim 1, wherein: performing thin-plate spline interpolation deformation on the mixed image set using the projection transformation matrix, performing pyramid fusion in the LAB color space, performing guided filtering processing on the fused image to obtain a corrected image, comprising the following steps, performing non-rigid deformation on the mixed image set using the projection transformation matrix to obtain a geometrically aligned image set, converting the geometrically aligned image set from RGB to LAB color space to separate the luminance and chrominance components; constructing a Gaussian pyramid for the LAB color space, obtaining contribution weights of image regions based on DNA alignment scores and quantum bit strings, and performing weighted fusion on the Gaussian pyramid to obtain a fused image; performing edge smoothing processing on the fused image to obtain a filtered LAB image; converting the filtered LAB image back to the RGB space to obtain a corrected image.

5. A deep learning-based mixed image correction system based on the deep learning-based mixed image correction method according to any one of claims 1 to 4, characterized by: comprising, an image preprocessing module, performing illumination normalization on the mixed image set and constructing a Gaussian pyramid image set; a feature extraction module, inputting the Gaussian pyramid image set into a ResNet-34 network to extract multi-scale feature maps and generate spatial position and channel statistical information; a DNA encoding module, converting the spatial position and channel statistical information into DNA sequence fragments through DNA base mapping rules, and performing biological sequence alignment using the Smith-Waterman algorithm to obtain feature point matching pairs; a matrix generation module, obtaining an initial homography matrix based on the feature point matching pairs and obtaining a fundamental matrix and an essential matrix through a weighted RANSAC algorithm; a quantum optimization module, encoding the fundamental matrix and the essential matrix into quantum bits, and obtaining a projection transformation matrix through quantum annealing; a correction module, performing thin-plate spline interpolation deformation on the mixed image set using the projection transformation matrix, performing pyramid fusion in the LAB color space, and performing guided filtering processing on the fused image to obtain a corrected image. 6.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is characterized in that: The processor executes the computer program to realize the steps of the deep learning-based mixed image correction method of any one of claims 1-4.

7. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program is executed by the processor to realize the steps of the deep learning-based mixed image correction method of any one of claims 1-4.

Images