Image correction method and device, storage medium and computer equipment

Abstract

The invention discloses an image correction method and device, a storage medium and computer equipment, relates to the technical field of digital medical treatment and the technical field of financial science and technology, and mainly aims to improve the correction efficiency and correction precision of images. Comprising the steps of obtaining a to-be-corrected image, inputting the to-be-corrected image into a preset matrix prediction model for matrix prediction, and obtaining a global perspective transformation matrix corresponding to the to-be-corrected image; performing global correction on the to-be-corrected image by using the global perspective transformation matrix to obtain a globally corrected image; and carrying out grid division on the globally corrected image, determining an image edge grid in each divided grid, respectively determining a local perspective transformation matrix corresponding to each image edge grid, and carrying out image edge correction on the globally corrected image based on the local perspective transformation matrix to obtain a corrected image. The image correction method and device are suitable for scenes where images are corrected.

Description

Technical Field

[0001] This invention relates to the fields of digital medical technology and financial technology, and in particular to an image correction method, apparatus, storage medium and computer equipment. Background Technology

[0002] Before 3D modeling or OCR recognition, perspective transformation correction is performed on images primarily to eliminate geometric distortions caused by the shooting angle. This restores the target object or text in the image to its shape under a standard viewpoint, thereby improving the accuracy and efficiency of subsequent processing. For example, in the medical field, tilted X-ray images need to be corrected to restore the true shape of the skeletal structure, improving the accuracy of AI-assisted diagnosis; in the fintech field, tilted invoice images need to be corrected to a horizontal viewpoint to ensure that OCR can accurately identify key fields such as amount and account number.

[0003] Currently, feature points are typically manually labeled from distorted images and ideal standard reference images. A transformation matrix is ​​then determined based on these manually labeled feature points, and this matrix is ​​used to correct the entire distorted image. However, this manual labeling method is time-consuming and labor-intensive. Furthermore, in actual shooting scenarios, image distortion is often quite complex, and the degree and form of distortion may differ in different regions of the image. Therefore, using a uniform transformation matrix to correct the entire distorted image cannot provide precise correction for different regions of the image. Summary of the Invention

[0004] This invention provides an image correction method, apparatus, storage medium, and computer device, which mainly improves the correction efficiency and accuracy of images.

[0005] According to a first aspect of the present invention, an image correction method is provided, comprising:

[0006] The image to be corrected is obtained, and the image to be corrected is input into a preset matrix prediction model to perform matrix prediction, thereby obtaining the global perspective transformation matrix corresponding to the image to be corrected.

[0007] The global perspective transformation matrix is ​​used to perform global correction on the image to be corrected, resulting in a globally corrected image.

[0008] The globally corrected image is divided into grids, and image edge grids are determined in each grid. The local perspective transformation matrix corresponding to each image edge grid is determined. Based on the local perspective transformation matrix, image edge correction is performed on the globally corrected image to obtain the corrected image.

[0009] Optionally, before inputting the image to be corrected into a preset matrix prediction model for matrix prediction to obtain the global perspective transformation matrix corresponding to the image to be corrected, the method further includes:

[0010] Construct a prediction model with a pre-defined initial matrix;

[0011] Obtain a sample dataset, wherein the sample dataset includes sample perspective distortion images with annotation information, and the annotation information is an actual perspective transformation matrix that can accurately correct the sample perspective distortion images;

[0012] The sample dataset is divided into a training set and a test set. The training set is used to train the preset initial matrix prediction model, and the test set is used to test the trained preset initial matrix prediction model. Finally, the trained preset initial matrix prediction model that meets the test conditions is taken as the preset matrix prediction model.

[0013] Optionally, the method for obtaining the sample perspective distortion image includes:

[0014] Obtain the orthophoto image of the sample;

[0015] In the orthophoto image of the sample, multiple sample feature points are determined, and perturbation points are randomly determined in a preset neighborhood centered on each of the sample feature points.

[0016] Based on the position coordinates of each sample feature point and the position coordinates of the corresponding perturbation point, a perturbation perspective transformation matrix is ​​constructed, and the perturbation perspective transformation matrix is ​​used to perform a perturbation transformation on the sample orthophoto image to obtain the sample perspective distortion image.

[0017] Optionally, the method for determining the actual perspective transformation matrix includes:

[0018] Multiple feature points are extracted from the perspective distortion image of the sample and the corresponding orthophoto image of the sample, and cross-image matching is performed on the feature points in the perspective distortion image of the sample and the feature points in the orthophoto image of the sample. Based on the matching results, a preset number of feature point pairs are determined.

[0019] Based on the feature point position coordinates in each of the feature point pairs, the actual perspective transformation matrix for correcting the sample perspective distortion image to the sample orthophoto image is determined.

[0020] Optionally, the globally corrected image is divided into grids, including:

[0021] Determine the image feature vector corresponding to the globally corrected image, and determine the feature dimension, the dimension value of each feature dimension, and the dimension weight of each feature dimension of the image feature vector;

[0022] The image complexity of the globally corrected image is determined based on the feature dimension, the dimension value, and the dimension weight.

[0023] The grid density is determined based on the image complexity, and the globally corrected image is then divided into grids based on the grid density.

[0024] Optionally, the local perspective transformation matrix corresponding to each image edge grid is determined, including:

[0025] In each divided grid, a center grid is determined, and based on the position coordinates of the four corner points in the center grid and the position coordinates of the four corner points in each edge grid, a local perspective transformation matrix is ​​constructed between the center grid and each edge grid.

[0026] Optionally, based on the local perspective transformation matrix, image edge correction is performed on the globally corrected image to obtain a corrected image, including:

[0027] The distance between each image edge grid and the image center grid is determined, and based on the distance, the matrix weights of the local perspective transformation matrix corresponding to each image edge grid are determined.

[0028] Based on the matrix weights, the local perspective transformation matrices corresponding to each image edge grid are weighted and fused to obtain a comprehensive perspective transformation matrix. Based on the comprehensive perspective transformation matrix, image edge correction is performed on the globally corrected image to obtain the corrected image.

[0029] According to a second aspect of the present invention, an image correction apparatus is provided, comprising:

[0030] The matrix prediction unit is used to acquire the image to be corrected, input the image to be corrected into a preset matrix prediction model to perform matrix prediction, and obtain the global perspective transformation matrix corresponding to the image to be corrected.

[0031] A global correction unit is used to perform global correction on the image to be corrected using the global perspective transformation matrix to obtain a globally corrected image.

[0032] The local correction unit is used to divide the globally corrected image into grids, determine the image edge grids in each grid, and determine the local perspective transformation matrix corresponding to each image edge grid. Based on the local perspective transformation matrix, the image edge is corrected on the globally corrected image to obtain the corrected image.

[0033] According to a third aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the above-described image correction method.

[0034] According to a fourth aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described image correction method.

[0035] According to the image correction method, apparatus, storage medium, and computer device provided by the present invention, compared with the current method of determining the transformation matrix based on manually annotated feature points and using the transformation matrix to correct the entire distorted image, the present invention directly uses a model to obtain the global perspective transformation matrix and uses the global perspective transformation matrix to perform global perspective transformation on the image. This avoids the time-consuming and labor-intensive problem of manually extracting feature points to obtain the perspective transformation matrix, thereby improving the image correction efficiency and accuracy. Then, the image is divided into grids to determine the image edge grids and the local perspective transformation matrix of the image edge grids. Finally, the local perspective transformation matrix is ​​used to correct the image edges. Thus, the present invention achieves global and local image correction by combining the global perspective transformation matrix and the local perspective transformation matrix. Global correction may not be able to handle locally curved edges. By dividing the grid and optimizing the edge grid locally, local deformations (such as curved pages or edge distortion of circular objects) can be finely corrected. That is, global correction prioritizes the treatment of major distortions, while local correction supplements details, reducing the complexity of the problem in stages and improving the overall correction accuracy. Attached Figure Description

[0036] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0037] Figure 1 A flowchart of an image correction method provided by an embodiment of the present invention is shown;

[0038] Figure 2 A flowchart of another image correction method provided by an embodiment of the present invention is shown;

[0039] Figure 3 A schematic diagram of the structure of an image correction device provided in an embodiment of the present invention is shown;

[0040] Figure 4 A schematic diagram of another image correction device provided in an embodiment of the present invention is shown;

[0041] Figure 5A schematic diagram of the physical structure of a computer device provided in an embodiment of the present invention is shown. Detailed Implementation

[0042] The present invention will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the present application can be combined with each other.

[0043] Currently, the method of determining the transformation matrix based on manually labeled feature points and then using this transformation matrix to correct the entire distorted image is time-consuming and labor-intensive. In addition, in actual shooting scenarios, image distortion is often quite complex, and the degree and form of distortion in different regions of the image may vary. Therefore, using a uniform transformation matrix to correct the entire distorted image cannot accurately correct different regions of the image.

[0044] To address the above problems, embodiments of the present invention provide an image correction method, such as... Figure 1 As shown, the method includes:

[0045] 101. Obtain the image to be corrected, input the image to be corrected into the preset matrix prediction model for matrix prediction, and obtain the global perspective transformation matrix corresponding to the image to be corrected.

[0046] The images to be corrected can be CT images in the medical field, insurance contract images in the financial field, or lease contract images. These images are deformed due to the tilt of the shooting angle.

[0047] In this embodiment of the invention, the image to be corrected is obtained from the database. Then, a preset matrix prediction model is used to predict the global perspective transformation matrix for the image to be corrected. The global perspective transformation matrix is ​​then used to perform a global perspective transformation on the image to be corrected to obtain an orthophoto. This embodiment of the invention directly predicts the global perspective transformation matrix through a model, which can improve the prediction accuracy and efficiency of the perspective transformation matrix and save the time of manually determining the perspective transformation matrix.

[0048] 102. Use the global perspective transformation matrix to perform global correction on the image to be corrected, and obtain the globally corrected image.

[0049] In this embodiment of the invention, if the pixel of the image to be corrected is (x, y), the global perspective transformation matrix... The correction method is as follows:

[0050]

[0051] Where (x″,y″) are the pixel coordinates in the globally corrected image, ω′ is the homogeneous coordinate transformation coefficient, and (x′,y′) are the homogeneous coordinates of the pixel in the globally corrected image.

[0052] 103. Divide the globally corrected image into grids, determine the image edge grids in each grid, and determine the local perspective transformation matrix corresponding to each image edge grid. Based on the local perspective transformation matrix, perform image edge correction on the globally corrected image to obtain the corrected image.

[0053] In this embodiment of the invention, the globally corrected image is divided into multiple uniform grids. Grids with at least one vertex located at the image boundary are defined as edge grids. Simultaneously, a center grid located at the center of the image is determined within each grid. Four points on the center grid are used as target keypoints, and four points on the edge grids are used as source keypoints. Based on the mapping relationship between the target keypoints and the source keypoints, the following equation is constructed:

[0054]

[0055] Where (x′,y′) are the coordinates of the target keypoint, (x,y) are the coordinates of the source keypoint, and a 11 a 12 a 13 a 21 a 22 a 23 a 31 a 32 a 33 The elements of the local perspective transformation matrix to be solved are represented by the coordinates of the four target keypoints and the four source keypoints, respectively. These elements can be used to solve the local perspective transformation matrix, thus obtaining the local perspective transformation matrix for each edge grid. Finally, the edge grid is subjected to secondary correction using the local perspective transformation matrix to eliminate residual deformation from the global correction. Therefore, this embodiment of the invention performs local correction on top of global image correction. Local correction eliminates local deformation from global correction, thereby improving the image correction accuracy. Furthermore, during the local correction process, by setting a local perspective transformation matrix for each edge grid to perform deformation correction, it can specifically eliminate wavy distortion, trapezoidal distortion, and other deformations, further improving the image correction accuracy.

[0056] According to the image correction method provided by this invention, compared with the current method of determining the transformation matrix based on manually annotated feature points and then using the transformation matrix to correct the entire distorted image, this invention directly uses a model to obtain the global perspective transformation matrix and uses this global perspective transformation matrix to perform a global perspective transformation on the image. This avoids the time-consuming and labor-intensive problem of manually extracting feature points to obtain the perspective transformation matrix, thereby improving the image correction efficiency and accuracy. The image is then divided into grids to determine the image edge grids, and the local perspective transformation matrix of the image edge grids is determined. Finally, the local perspective transformation matrix is ​​used to correct the image edges. Thus, this invention achieves both global and local image correction by combining the global and local perspective transformation matrices. Global correction may not be able to handle locally curved edges; by dividing the image into grids and optimizing the edge grids locally, local deformations (such as curved pages or edge distortion of circular objects) can be precisely corrected. In other words, global correction prioritizes the handling of major distortions, while local correction supplements details, reducing the complexity of the problem in stages and improving the overall correction accuracy.

[0057] Furthermore, to better illustrate the above image correction process, as a refinement and extension of the above embodiments, this invention provides another image correction method, such as... Figure 2 As shown, the method includes:

[0058] 201. Construct a prediction model with a preset initial matrix.

[0059] 202. Obtain the sample dataset, which includes sample perspective distortion images with annotation information. The annotation information is the actual perspective transformation matrix that can accurately correct the sample perspective distortion images.

[0060] In this embodiment of the invention, in order to train and construct a preset matrix prediction model, it is first necessary to obtain a sample perspective distortion image as training data. Based on this, step 202 specifically includes: obtaining a sample orthophoto image; determining multiple sample feature points in the sample orthophoto image, and randomly determining perturbation points in a preset neighborhood centered on each sample feature point; constructing a perturbation perspective transformation matrix based on the position coordinates of each sample feature point and the position coordinates of the corresponding perturbation point, and using the perturbation perspective transformation matrix to perform a perturbation transformation on the sample orthophoto image to obtain the sample perspective distortion image.

[0061] The preset neighborhood is set according to actual needs. Specifically, multiple orthophoto images of samples without geometric distortion are downloaded from the network. Taking one target sample orthophoto image as an example, at least four sample feature points with significant geometric structure are identified in the target sample orthophoto image. Perturbation points for the corresponding sample feature points are identified within a region centered on each sample feature point and with a radius defined by a specified length. Thus, the perturbation points for each sample feature point can be determined in the above manner. For example, if each sample feature point i among the four sample feature points is denoted as (x... ci ,y ci Each perturbation feature point is denoted as (x). ri ,y ri The perturbation perspective transformation matrix to be solved is: The elements of the perturbation perspective transformation matrix are then determined as follows:

[0062]

[0063] Each sample feature point i(x) ci ,y ci ), and its corresponding perturbation feature point (x) ri ,y ri Substituting into the above formula, we can obtain the perturbation perspective transformation matrix. Then, we use the perturbation perspective transformation matrix to perform perturbation transformation on the sample orthophoto image, that is, multiply each pixel in the sample orthophoto image by the perturbation perspective transformation matrix to obtain each transformed pixel. Then, each transformed pixel constitutes the sample perspective distortion image.

[0064] Furthermore, after determining the sample perspective distortion image, it is also necessary to determine the actual perspective transformation matrix for correcting the sample perspective distortion image to an orthophoto image. Based on this, the method includes: extracting multiple feature points in the sample perspective distortion image and the corresponding sample orthophoto image respectively, and performing cross-image matching on the feature points in the sample perspective distortion image and the feature points in the sample orthophoto image; determining a preset number of feature point pairs based on the matching results; and determining the actual perspective transformation matrix for correcting the sample perspective distortion image to the sample orthophoto image based on the feature point position coordinates in each feature point pair.

[0065] Specifically, the first step is to extract multiple feature points with significant geometric structures from the sample perspective distortion image and the corresponding sample orthophoto image. Taking the extraction of feature points from the sample perspective distortion image as an example, the specific method for extracting feature points includes: using the sample perspective distortion image as the bottom pyramid layer image of the image pyramid to be constructed; using each pyramid layer in the image pyramid to be constructed, except for the bottom pyramid layer, as a target pyramid layer; based on the number of rows and columns of the image in the previous pyramid layer corresponding to the target pyramid layer, sequentially performing Gaussian blur processing and downsampling processing on the image in the previous pyramid layer to obtain the target pyramid layer image; constructing the image pyramid corresponding to each image based on the bottom pyramid layer image and each of the target pyramid layer images; processing the images in two adjacent pyramid layers of the image pyramid to images of the same resolution, and subtracting the images of the same resolution in the two adjacent pyramid layers to obtain the difference of Gaussian pyramid. The method for processing the images in two adjacent pyramid layers of the image pyramid to images of the same resolution includes: determining the low-resolution images in the two adjacent pyramid layers. The system comprises a high-resolution image layer and a low-resolution image layer. Based on the number of rows and columns in the high-resolution image layer and the number of rows and columns in the low-resolution image layer, a magnification factor is determined for the image in the low-resolution image layer. Based on the magnification factor, the image in the low-resolution image layer is upsampled to obtain an image of the same resolution as the image in the high-resolution image layer. Each difference pyramid layer in the Gaussian difference pyramid (excluding the first and last layers) is designated as a target difference pyramid layer, and each pixel in the image of the target difference pyramid layer is designated as a target pixel. In the difference pyramid layers adjacent to the target difference pyramid layer, each adjacent evaluation pixel within a preset difference neighborhood corresponding to the target pixel is determined, and in the target difference pyramid layer, each same-layer evaluation pixel within a preset same-layer neighborhood corresponding to the target pixel is determined. If the target pixel is the maximum or minimum pixel value among the adjacent evaluation pixels and the same-layer evaluation pixels, then the target pixel is determined as a feature point with significant geometric structure in the sample perspective distortion image.

[0066] The preset difference neighborhood and preset same-layer neighborhood are set according to actual needs. Specifically, in order to extract feature points in the sample perspective distortion image, it is first necessary to construct an influence pyramid for the sample perspective distortion image. The specific construction method of the image pyramid is as follows: First, the sample perspective distortion image is used as the bottom layer image of the image pyramid. This layer image has the highest resolution and is the clearest. Then, Gaussian blur processing is performed on the bottom layer image. After Gaussian blur processing, downsampling processing is performed. The downsampling process is to merge every 2×2 pixels in the bottom layer image into a new pixel. All new pixels constitute the image in the next layer of the pyramid corresponding to the bottom layer. If the next layer of the pyramid corresponding to the bottom layer is determined as the second pyramid layer. The image in the second pyramid layer is then subjected to Gaussian blurring. After Gaussian blurring, downsampling is performed. The downsampling process involves merging every 2×2 pixels in the image of the second pyramid layer into a new pixel. All these new pixels form the image in the next pyramid layer, i.e., the third pyramid layer. This process is repeated layer by layer, applying Gaussian blurring and downsampling to the image in the previous pyramid layer to obtain the image in the current pyramid layer. This continues until the resolution of the image in the latest pyramid layer meets the requirements. Finally, the image pyramid of the sample perspective distortion image is formed by all the pyramid layers.

[0067] Furthermore, after constructing the image pyramid for each image, a Gaussian difference pyramid needs to be constructed. The specific construction scheme is as follows: Taking two adjacent pyramid layers of the first and second pyramid layers as examples, since the images in the first and second pyramid layers have different resolutions, a high-resolution image layer and a low-resolution image layer are first determined in the first and second pyramid layers. The resolution of the images in the high-resolution and low-resolution image layers, i.e., the number of rows and columns, is then determined. Based on the resolutions of the images in the high-resolution and low-resolution image layers, the magnification factor of the image in the low-resolution image layer is determined. Based on this magnification factor, the image in the low-resolution image layer is upsampled. Specific upsampling methods include bilinear interpolation, bicubic interpolation, and nearest neighbor method. This process brings the resolution of the image in the low-resolution image layer to the same level as the resolution of the image in the high-resolution image layer. Thus, by following the above method, the images in each adjacent pyramid layer of the image pyramid can be processed to achieve the same resolution. The image is first treated as an image of the same resolution. Then, the images of the same resolution in two adjacent pyramid layers are subtracted to obtain the Gaussian difference pyramid. The Gaussian difference pyramid is then labeled from bottom to top as the 1st difference pyramid layer, the 2nd difference pyramid layer, the 3rd difference pyramid layer, etc. Starting from the 2nd difference pyramid layer, the neighboring evaluation pixels and the same-layer evaluation pixels of each pixel in the 2nd difference pyramid layer are determined. Specifically, taking pixel a in the 2nd difference pyramid layer as an example, the pixels in the preset difference neighborhood corresponding to pixel a are determined in the 1st and 3rd difference pyramid layers, i.e., the neighboring evaluation pixels. At the same time, the pixels in the preset same-layer neighborhood corresponding to pixel a are determined in the 2nd difference pyramid layer, i.e., the same-layer evaluation pixels. Then, it is determined whether pixel a is the pixel with the largest pixel value among the neighboring evaluation pixels and the same-layer evaluation pixels, or the pixel with the smallest pixel value. If so, pixel a is determined as a feature point in the sample perspective distortion image. When extracting feature points using the difference of Gaussian pyramid in this embodiment of the invention, the extracted feature points have scale invariance because image information at different scales is taken into account. That is, the same feature points can be accurately identified at different scales, thus improving the accuracy of feature point extraction in the image.

[0068] Furthermore, following the above method, feature points in the sample orthophoto image can be determined. Then, cross-image similarity matching is performed on each feature point in the sample perspective distortion image and the corresponding sample orthophoto image to identify feature point pairs with similar geometric structures in the two images. The number of these feature point pairs is at least four. Then, based on the transformation relationship between the coordinates of the feature points in the sample perspective distortion image and the coordinates of the feature points in the sample orthophoto image, the actual perspective transformation matrix for correcting the sample perspective distortion image to the sample orthophoto image is determined. Finally, multiple sample perspective distortion images and their corresponding actual perspective transformation matrices constitute a sample dataset.

[0069] 203. Divide the sample dataset into a training set and a test set. Use the training set to train the preset initial matrix prediction model and use the test set to test the trained preset initial matrix prediction model. Finally, use the trained preset initial matrix prediction model that meets the test conditions as the preset matrix prediction model.

[0070] Specifically, the model is trained and tested using a sample dataset. This can begin by dividing the dataset into training and test sets using random or specific strategies (such as stratified sampling). The model is then trained using the training set and tested on the test set to evaluate its performance on unseen data. Metrics such as mCP, precision, and recall are calculated and recorded on the test set. If the model performance does not meet requirements, the training phase can be returned for further iterations or adjustments. This process yields a pre-defined matrix prediction model that meets the desired performance.

[0071] 204. Obtain the image to be corrected, input the image to be corrected into the preset matrix prediction model for matrix prediction, and obtain the global perspective transformation matrix corresponding to the image to be corrected.

[0072] 205. Use the global perspective transformation matrix to perform global correction on the image to be corrected, and obtain the globally corrected image.

[0073] The image to be corrected is directly input into the preset matrix prediction model. The preset matrix prediction model can automatically output the global perspective transformation matrix. Then, the global perspective transformation matrix is ​​used to perform global correction on the image to be corrected through the cv2.warpPerspective function.

[0074] 206. Divide the globally corrected image into grids, determine the image edge grids in each grid, and determine the local perspective transformation matrix corresponding to each image edge grid. Based on the local perspective transformation matrix, perform image edge correction on the globally corrected image to obtain the corrected image.

[0075] In this embodiment of the invention, after global correction, in order to eliminate possible edge distortions in the image, local edge correction is also required. Before local correction, the globally corrected image needs to be meshed. Based on this, step 206 specifically includes: determining the image feature vector corresponding to the globally corrected image, and determining the feature dimension, the dimension value of each feature dimension, and the dimension weight of each feature dimension; determining the image complexity of the globally corrected image based on the feature dimension, the dimension value, and the dimension weight; determining the mesh density based on the image complexity, and meshing the globally corrected image based on the mesh density.

[0076] Specifically, the image complexity is determined according to the following formula:

[0077]

[0078] Complexity img Let f be the image complexity, d be the total number of feature dimensions, and f be the image complexity. i For each feature dimension i, w i Dimension weights are assigned to each feature dimension i. Then, if the grid complexity is higher, the grid density is greater, and the globally corrected image is divided into grids based on the grid density. If at least one point in each grid is located at the image edge, then that grid is determined as an edge grid. Further, a local perspective transformation matrix needs to be determined for each edge grid. Based on this, the method includes: determining the image center grid in each divided grid, and constructing a local perspective transformation matrix between the image center grid and each image edge grid based on the position coordinates of the four corner points in the image center grid and the position coordinates of the four corner points in each image edge grid.

[0079] Specifically, the perspective transformation between the central grid and the edge grid is described by the homography matrix H. The corner coordinates (x, y) of the central grid and the corner coordinates (x′, y′) of the edge grid satisfy the following form:

[0080]

[0081] Among them, h 11 ......h 32The elements of matrix H are represented by the corner points of the center grid and the corner points of the edge grid, respectively. These are then substituted into the above formula to solve for the matrix elements, thereby obtaining the local perspective transformation matrix of each image edge grid. Further, local correction of the image edges is required based on the local perspective transformation matrix of each image edge grid. Therefore, the method includes: determining the distance between each image edge grid and the image center grid, and determining the matrix weights of the local perspective transformation matrices corresponding to each image edge grid based on the distances; weighting and fusing the local perspective transformation matrices corresponding to each image edge grid based on the matrix weights to obtain a comprehensive perspective transformation matrix; and performing image edge correction on the globally corrected image based on the comprehensive perspective transformation matrix to obtain the corrected image.

[0082] Specifically, the distance between the center point of each edge grid and the center point of the central grid is determined. Weights are assigned to the local perspective transformation matrix of each edge grid based on these distances; for example, the larger the distance, the smaller the corresponding weight. Based on these weights, each local perspective transformation matrix is ​​weighted and summed to obtain a comprehensive perspective transformation matrix common to all edge grids. This comprehensive perspective transformation matrix is ​​then used to correct the image of each edge grid. Specifically, each pixel in each edge grid is multiplied by the comprehensive perspective transformation matrix to obtain the corrected pixels for each edge grid. These corrected pixels constitute the corrected edge grid, thus achieving image edge correction on the globally corrected image. Since the local perspective transformation matrix of a single edge grid may be abnormal due to corner detection errors, weighted fusion can suppress noise effects, thereby improving the accuracy of local image correction.

[0083] According to another image correction method provided by the present invention, compared with the current method of determining the transformation matrix based on manually annotated feature points and using the transformation matrix to correct the entire distorted image, the present invention directly uses a model to obtain the global perspective transformation matrix and uses the global perspective transformation matrix to perform global perspective transformation on the image. This avoids the time-consuming and labor-intensive problem of manually extracting feature points to obtain the perspective transformation matrix, thereby improving the image correction efficiency and accuracy. Then, the image is divided into grids to determine the image edge grids and the local perspective transformation matrix of the image edge grids. Finally, the local perspective transformation matrix is ​​used to correct the image edges. Thus, the present invention achieves global and local image correction by combining the global perspective transformation matrix and the local perspective transformation matrix. Global correction may not be able to handle locally curved edges. By dividing the grid and optimizing the edge grid locally, local deformations (such as curved pages or edge distortion of circular objects) can be finely corrected. That is, global correction prioritizes the treatment of major distortions, while local correction supplements details, reducing the complexity of the problem in stages and improving the overall correction accuracy.

[0084] Furthermore, as Figure 1 In specific implementation, embodiments of the present invention provide an image correction device, such as... Figure 3 As shown, the device includes: a matrix prediction unit 31, a global correction unit 32, and a local correction unit 33.

[0085] The matrix prediction unit 31 can be used to acquire the image to be corrected, input the image to be corrected into a preset matrix prediction model for matrix prediction, and obtain the global perspective transformation matrix corresponding to the image to be corrected.

[0086] The global correction unit 32 can be used to perform global correction on the image to be corrected using the global perspective transformation matrix to obtain a globally corrected image.

[0087] The local correction unit 33 can be used to divide the globally corrected image into grids, determine the image edge grids in each grid, and determine the local perspective transformation matrix corresponding to each image edge grid. Based on the local perspective transformation matrix, the image edge is corrected on the globally corrected image to obtain the corrected image.

[0088] In specific application scenarios, in order to construct a pre-defined matrix prediction model, such as Figure 4 As shown, the device also includes a construction unit 34.

[0089] The construction unit 34 can be used to construct a preset initial matrix prediction model; obtain a sample dataset, wherein the sample dataset includes sample perspective distortion images with annotation information, the annotation information being an actual perspective transformation matrix capable of accurately correcting the sample perspective distortion images; divide the sample dataset into a training set and a test set, use the training set to train the preset initial matrix prediction model, and use the test set to test the trained preset initial matrix prediction model, and finally use the trained preset initial matrix prediction model that meets the test conditions as the preset matrix prediction model.

[0090] In specific application scenarios, in order to obtain sample perspective distortion images, the construction unit 34 includes an acquisition module 341, a first determination module 342, and a disturbance module 343.

[0091] The acquisition module 341 can be used to acquire orthophotos of samples.

[0092] The first determining module 342 can be used to determine multiple sample feature points in the sample orthophoto image, and randomly determine perturbation points in a preset neighborhood centered on each sample feature point.

[0093] The perturbation module 343 can be used to construct a perturbation perspective transformation matrix based on the position coordinates of each sample feature point and the position coordinates of the corresponding perturbation point, and use the perturbation perspective transformation matrix to perform perturbation transformation on the sample orthophoto image to obtain the sample perspective distortion image.

[0094] In specific application scenarios, in order to obtain the actual perspective transformation matrix, the first determining module 342 can also be used to extract multiple feature points in the sample perspective distortion image and the corresponding sample orthophoto image respectively, and perform cross-image matching on the feature points in the sample perspective distortion image and the feature points in the sample orthophoto image. Based on the matching results, a preset number of feature point pairs are determined. Based on the feature point position coordinates in each feature point pair, the actual perspective transformation matrix for correcting the sample perspective distortion image to the sample orthophoto image is determined.

[0095] In specific application scenarios, in order to perform grid division on the globally corrected image, the local correction unit 33 includes a second determination module 331 and a grid division module 332.

[0096] The second determining module 331 can be used to determine the image feature vector corresponding to the globally corrected image, and to determine the feature dimension, the dimension value of each feature dimension, and the dimension weight of each feature dimension of the image feature vector.

[0097] The second determining module 331 can also be used to determine the image complexity of the globally corrected image based on the feature dimension, the dimension value, and the dimension weight.

[0098] The grid division module 332 can be used to determine the grid density based on the image complexity, and to perform grid division on the globally corrected image based on the grid density.

[0099] In specific application scenarios, in order to determine the local perspective transformation matrix corresponding to each image edge grid, the local correction unit 33 also includes a construction module 333.

[0100] The construction module 333 can be used to determine the image center grid in each divided grid, and based on the position coordinates of the four corner points in the image center grid and the position coordinates of the four corner points in each image edge grid, respectively construct the local perspective transformation matrix between the image center grid and each image edge grid.

[0101] In specific application scenarios, in order to perform local correction on the image, the local correction unit 33 further includes a fusion module 334.

[0102] The second determining module 331 can also be used to determine the distance between each of the image edge grids and the image center grid, and based on the distance, determine the matrix weight of the local perspective transformation matrix corresponding to each of the image edge grids.

[0103] The fusion module 334 can be used to perform weighted fusion of the local perspective transformation matrix corresponding to each image edge grid based on the matrix weights to obtain a comprehensive perspective transformation matrix, and perform image edge correction on the globally corrected image based on the comprehensive perspective transformation matrix to obtain a corrected image.

[0104] It should be noted that other corresponding descriptions of the functional modules involved in the image correction device provided in this embodiment of the invention can be found in [reference]. Figure 1 The corresponding description of the method shown will not be repeated here.

[0105] Based on the above, Figure 1 Accordingly, this embodiment of the invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the following steps: acquiring an image to be corrected; inputting the image to be corrected into a preset matrix prediction model for matrix prediction to obtain a global perspective transformation matrix corresponding to the image to be corrected; performing global correction on the image to be corrected using the global perspective transformation matrix to obtain a globally corrected image; dividing the globally corrected image into grids; determining image edge grids in each grid and determining a local perspective transformation matrix corresponding to each image edge grid; and performing image edge correction on the globally corrected image based on the local perspective transformation matrix to obtain a corrected image.

[0106] Based on the above, Figure 1 The method shown and as Figure 3 The embodiment of the device shown in the invention also provides a physical structure diagram of a computer device, such as... Figure 5As shown, the computer device includes: a processor 41, a memory 42, and a computer program stored in the memory 42 and executable on the processor. Both the memory 42 and the processor 41 are mounted on a bus 43. When the processor 41 executes the program, it performs the following steps: acquiring an image to be corrected; inputting the image to be corrected into a preset matrix prediction model for matrix prediction to obtain a global perspective transformation matrix corresponding to the image to be corrected; performing global correction on the image to be corrected using the global perspective transformation matrix to obtain a globally corrected image; dividing the globally corrected image into grids; determining image edge grids in each grid division; determining the local perspective transformation matrix corresponding to each image edge grid; and performing image edge correction on the globally corrected image based on the local perspective transformation matrix to obtain a corrected image.

[0107] Through the technical solution of this invention, the present invention directly uses a model to obtain a global perspective transformation matrix, and uses this global perspective transformation matrix to perform a global perspective transformation on the image. This avoids the time-consuming and labor-intensive problem of manually extracting feature points to obtain the perspective transformation matrix, thereby improving the image correction efficiency and accuracy. Then, the image is divided into grids to determine the image edge grids, and the local perspective transformation matrix of the image edge grids is determined. Finally, the local perspective transformation matrix is ​​used to correct the image edges. Thus, this invention achieves global and local image correction by combining the global perspective transformation matrix and the local perspective transformation matrix. Global correction may not be able to handle locally curved edges. By dividing the grid and optimizing the edge grid locally, local deformations (such as curved pages or edge distortion of circular objects) can be finely corrected. That is, global correction prioritizes the treatment of major distortions, while local correction supplements details, reducing the complexity of the problem in stages and improving the overall correction accuracy.

[0108] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0109] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image correction method, characterized in that, include: The image to be corrected is obtained, and the image to be corrected is input into a preset matrix prediction model to perform matrix prediction, thereby obtaining the global perspective transformation matrix corresponding to the image to be corrected. The global perspective transformation matrix is ​​used to perform global correction on the image to be corrected, resulting in a globally corrected image. The globally corrected image is divided into grids, and image edge grids are determined in each grid. The local perspective transformation matrix corresponding to each image edge grid is determined. Based on the local perspective transformation matrix, image edge correction is performed on the globally corrected image to obtain the corrected image.

2. The method according to claim 1, characterized in that, Before inputting the image to be corrected into a preset matrix prediction model for matrix prediction to obtain the global perspective transformation matrix corresponding to the image to be corrected, the method further includes: Construct a prediction model with a pre-defined initial matrix; Obtain a sample dataset, wherein the sample dataset includes sample perspective distortion images with annotation information, and the annotation information is an actual perspective transformation matrix that can accurately correct the sample perspective distortion images; The sample dataset is divided into a training set and a test set. The training set is used to train the preset initial matrix prediction model, and the test set is used to test the trained preset initial matrix prediction model. Finally, the trained preset initial matrix prediction model that meets the test conditions is taken as the preset matrix prediction model.

3. The method according to claim 2, characterized in that, The method for obtaining the sample perspective distortion image includes: Obtain the orthophoto image of the sample; In the orthophoto image of the sample, multiple sample feature points are determined, and perturbation points are randomly determined in a preset neighborhood centered on each of the sample feature points. Based on the position coordinates of each sample feature point and the position coordinates of the corresponding perturbation point, a perturbation perspective transformation matrix is ​​constructed, and the perturbation perspective transformation matrix is ​​used to perform a perturbation transformation on the sample orthophoto image to obtain the sample perspective distortion image.

4. The method according to claim 3, characterized in that, The method for determining the actual perspective transformation matrix includes: Multiple feature points are extracted from the perspective distortion image of the sample and the corresponding orthophoto image of the sample, and cross-image matching is performed on the feature points in the perspective distortion image of the sample and the feature points in the orthophoto image of the sample. Based on the matching results, a preset number of feature point pairs are determined. Based on the feature point position coordinates in each of the feature point pairs, the actual perspective transformation matrix for correcting the sample perspective distortion image to the sample orthophoto image is determined.

5. The method according to claim 1, characterized in that, The globally corrected image is divided into grids, including: Determine the image feature vector corresponding to the globally corrected image, and determine the feature dimension, the dimension value of each feature dimension, and the dimension weight of each feature dimension of the image feature vector; The image complexity of the globally corrected image is determined based on the feature dimension, the dimension value, and the dimension weight. The grid density is determined based on the image complexity, and the globally corrected image is then divided into grids based on the grid density.

6. The method according to claim 1, characterized in that, Determine the local perspective transformation matrix corresponding to each image edge grid, including: In each divided grid, a center grid is determined, and based on the position coordinates of the four corner points in the center grid and the position coordinates of the four corner points in each edge grid, a local perspective transformation matrix is ​​constructed between the center grid and each edge grid.

7. The method according to claim 1, characterized in that, Based on the local perspective transformation matrix, image edge correction is performed on the globally corrected image to obtain a corrected image, including: The distance between each image edge grid and the image center grid is determined, and based on the distance, the matrix weights of the local perspective transformation matrix corresponding to each image edge grid are determined. Based on the matrix weights, the local perspective transformation matrices corresponding to each image edge grid are weighted and fused to obtain a comprehensive perspective transformation matrix. Based on the comprehensive perspective transformation matrix, image edge correction is performed on the globally corrected image to obtain the corrected image.

8. An image correction device, characterized in that, include: The matrix prediction unit is used to acquire the image to be corrected, input the image to be corrected into a preset matrix prediction model to perform matrix prediction, and obtain the global perspective transformation matrix corresponding to the image to be corrected. A global correction unit is used to perform global correction on the image to be corrected using the global perspective transformation matrix to obtain a globally corrected image. The local correction unit is used to divide the globally corrected image into grids, determine the image edge grids in each grid, and determine the local perspective transformation matrix corresponding to each image edge grid. Based on the local perspective transformation matrix, the image edge is corrected on the globally corrected image to obtain the corrected image.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images