Image correction method, device, storage medium and electronic device

Abstract

The present disclosure relates to an image correction method and device, a storage medium and an electronic device to reduce the deformation of the corrected image and improve the display effect of the corrected image. The method comprises: obtaining a target image to be corrected; classifying straight lines in the target image to obtain a horizontal line set and a vertical line set; filtering the horizontal lines in the horizontal line set to remove outlier horizontal lines in the horizontal line set to obtain a target horizontal line set, and filtering the vertical lines in the vertical line set to remove outlier vertical lines in the vertical line set to obtain a target vertical line set; determining a target correction transformation matrix according to the straight lines in the target horizontal line set and the target vertical line set, and performing transformation processing on the target image by using the target correction transformation matrix to obtain a corrected image.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to image correction methods, apparatus, storage media and electronic devices. Background Technology

[0002] Image correction can correct images that have perspective distortion due to shooting from above or below. By finding the geometric relationships of the objects in the image, the transformation matrix is ​​calculated, and the original image is transformed according to the transformation matrix to obtain the corrected image.

[0003] Image correction in related technologies mainly involves first detecting straight lines in the image, then detecting vanishing points using the detected straight lines, obtaining the correction transformation matrix using the detected vanishing points, and finally using the correction transformation matrix to transform the image. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides an image correction method, apparatus, storage medium, and electronic device.

[0005] According to a first aspect of the present disclosure, an image correction method is provided, comprising:

[0006] Obtain the target image to be corrected;

[0007] The straight lines in the target image are classified to obtain a set of horizontal lines and a set of vertical lines;

[0008] The horizontal lines in the set of horizontal lines are filtered to remove outliers, resulting in a target set of horizontal lines. Similarly, the vertical lines in the set of vertical lines are filtered to remove outliers, resulting in a target set of vertical lines.

[0009] The target correction transformation matrix is ​​determined based on the straight lines in the target horizontal line set and the target vertical line set, and the target image is transformed using the target correction transformation matrix to obtain the corrected image.

[0010] Optionally, the step of filtering the horizontal lines in the set of horizontal lines to remove outliers and obtain the target set of horizontal lines includes:

[0011] Select any two horizontal lines from the set of horizontal lines as the initial target horizontal lines, and perform the following procedure:

[0012] Determine the distance between the vanishing point of the target horizontal line and each other horizontal line in the set of horizontal lines. If the distance is less than a preset distance, classify the other horizontal lines into a candidate set that includes the target horizontal line, and select any two horizontal lines other than the target horizontal line in the set of horizontal lines as new target horizontal lines, until every horizontal line in the set of horizontal lines is selected as a target horizontal line.

[0013] Among the multiple candidate sets obtained, the set with the most horizontal lines is determined as the target horizontal line set.

[0014] Optionally, the step of filtering the vertical lines in the set of vertical lines to remove outliers and obtain the target set of vertical lines includes:

[0015] Select any two vertical lines from the set of vertical lines as the initial target vertical lines, and perform the following procedure:

[0016] Determine the distance between the vanishing point of the target vertical line and each other vertical line in the set of vertical lines. If the distance is less than a preset distance, classify the other vertical lines into a candidate set that includes the target vertical line, and select any two vertical lines other than the target vertical line in the set of vertical lines as new target vertical lines, until every vertical line in the set of vertical lines is selected as a target vertical line.

[0017] Among the multiple candidate sets obtained, the set with the most vertical lines is determined as the target vertical line set.

[0018] Optionally, determining the target correction transformation matrix based on the straight lines in the target horizontal line set and the target vertical line set includes:

[0019] Using a preset perspective transformation matrix as the initial correction transformation matrix, the correction transformation process is iteratively performed on the lines in the set of horizontal lines and the set of vertical lines until the preset number of iterations is reached, thereby obtaining the target correction transformation matrix. The correction transformation process includes:

[0020] The straight lines in the set of horizontal lines and the set of vertical lines are transformed according to the correction transformation matrix of this iteration process;

[0021] Based on the dot product of the transformed horizontal line and the vertical axis in the target image plane and the dot product of the transformed vertical line and the horizontal axis in the target image plane, the target optimization function is determined, and the correction transformation matrix is ​​updated with the goal of minimizing the target optimization function.

[0022] The updated correction transformation matrix is ​​used as the correction transformation matrix for the next iteration.

[0023] Optionally, the preset perspective transformation matrix is ​​determined in the following way:

[0024] Based on preset horizontal transformation parameters, the height of the target image, and the width of the target image, a horizontal transformation matrix is ​​determined, wherein the horizontal transformation parameters are used to perform corrective transformation along the horizontal axis.

[0025] Based on preset vertical transformation parameters, the height of the target image, and the width of the target image, a vertical transformation matrix is ​​determined, wherein the vertical transformation parameters are used to perform corrective transformation along the vertical axis.

[0026] Based on preset planar rotation parameters, the height of the target image, and the width of the target image, a planar transformation matrix is ​​determined, wherein the planar rotation parameters are used to perform rotation transformation on the plane of the target image;

[0027] The product of the horizontal transformation matrix, the vertical transformation matrix, and the planar transformation matrix is ​​used as the preset perspective transformation matrix;

[0028] Updating the correction transformation matrix includes:

[0029] Update at least one of the horizontal transformation parameters, vertical transformation parameters, and planar rotation parameters of the correction transformation matrix.

[0030] Optionally, the target correction transformation matrix includes a horizontal transformation matrix and a vertical transformation matrix, and the step of determining the target correction transformation matrix based on the straight lines in the target horizontal line set and the target vertical line set includes:

[0031] The vanishing point of each horizontal line in the target horizontal line set is determined to be at the angle between the vanishing point of each horizontal line in the target vertical line set and the target image plane to obtain the horizontal perspective angle; and the vanishing point of each vertical line in the target vertical line set is determined to be at the angle between the vanishing point of each vertical line in the target vertical line set and the target image plane to obtain the vertical perspective angle.

[0032] Based on the numerical relationship between the horizontal perspective angle and the first preset angle, and the numerical relationship between the vertical perspective angle and the second preset angle, the correction direction for correcting the target image is determined, and the correction direction includes horizontal correction or vertical correction.

[0033] In the horizontal transformation matrix and the vertical transformation matrix, the target correction transformation matrix is ​​determined according to the correction direction.

[0034] Optionally, classifying the straight lines in the target image to obtain a set of horizontal lines and a set of vertical lines includes:

[0035] Line detection is performed on the target image, and for each detected line, the angle between the line and the horizontal axis in the target image plane and the angle between the line and the vertical axis in the target image plane are determined.

[0036] If the angle between the line and the horizontal axis is less than the angle between the line and the vertical axis, the line is classified into the set of horizontal lines. If the angle between the line and the horizontal axis is greater than or equal to the angle between the line and the vertical axis, the line is classified into the set of vertical lines.

[0037] According to a first aspect of the present disclosure, an image correction apparatus is provided, the apparatus comprising:

[0038] The acquisition module is configured to acquire the target image to be corrected.

[0039] The classification module is configured to classify the straight lines in the target image to obtain a set of horizontal lines and a set of vertical lines;

[0040] The filtering module is configured to filter the horizontal lines in the set of horizontal lines to remove outlier horizontal lines in the set of horizontal lines, thereby obtaining a target set of horizontal lines; and to filter the vertical lines in the set of vertical lines to remove outlier vertical lines in the set of vertical lines, thereby obtaining a target set of vertical lines.

[0041] The correction module is configured to determine a target correction transformation matrix based on the straight lines in the target horizontal line set and the target vertical line set, and to perform transformation processing on the target image using the target correction transformation matrix to obtain a corrected image.

[0042] Optionally, the filtering module is configured to:

[0043] Select any two horizontal lines from the set of horizontal lines as the initial target horizontal lines, and perform the following procedure:

[0044] Determine the distance between the vanishing point of the target horizontal line and each other horizontal line in the set of horizontal lines. When the distance is less than a preset distance, classify the other horizontal lines into a candidate set that includes the target horizontal line, and select any two horizontal lines other than the target horizontal line in the set of horizontal lines as new target horizontal lines, until every horizontal line in the set of horizontal lines is selected as a target horizontal line.

[0045] Among the multiple candidate sets obtained, the set with the most horizontal lines is determined as the target horizontal line set.

[0046] Optionally, the filtering module is configured to:

[0047] Select any two vertical lines from the set of vertical lines as the initial target vertical lines, and perform the following procedure:

[0048] Determine the distance between the vanishing point of the target vertical line and each other vertical line in the set of vertical lines. When the distance is less than a preset distance, classify the other vertical lines into a candidate set that includes the target vertical line, and select any two vertical lines other than the target vertical line in the set of vertical lines as new target vertical lines, until every vertical line in the set of vertical lines is selected as a target vertical line.

[0049] Among the multiple candidate sets obtained, the set with the most vertical lines is determined as the target vertical line set.

[0050] Optionally, the correction module is configured as follows:

[0051] Using a preset perspective transformation matrix as the initial correction transformation matrix, the correction transformation process is iteratively performed on the lines in the set of horizontal lines and the set of vertical lines until the preset number of iterations is reached, thereby obtaining the target correction transformation matrix. The correction transformation process includes:

[0052] The straight lines in the set of horizontal lines and the set of vertical lines are transformed according to the correction transformation matrix of this iteration process;

[0053] Based on the dot product of the transformed horizontal line and the vertical axis in the target image plane and the dot product of the transformed vertical line and the horizontal axis in the target image plane, the target optimization function is determined, and the correction transformation matrix is ​​updated with the goal of minimizing the target optimization function.

[0054] The updated correction transformation matrix is ​​used as the correction transformation matrix for the next iteration.

[0055] Optionally, the apparatus further includes a matrix determination module for determining a preset perspective transformation matrix correction, the matrix determination module being configured to:

[0056] Based on preset horizontal transformation parameters, the height of the target image, and the width of the target image, a horizontal transformation matrix is ​​determined, wherein the horizontal transformation parameters are used to perform corrective transformation along the horizontal axis.

[0057] Based on preset vertical transformation parameters, the height of the target image, and the width of the target image, a vertical transformation matrix is ​​determined, wherein the vertical transformation parameters are used to perform corrective transformation along the vertical axis.

[0058] Based on preset planar rotation parameters, the height of the target image, and the width of the target image, a planar transformation matrix is ​​determined, wherein the planar rotation parameters are used to perform rotation transformation on the plane of the target image;

[0059] The product of the horizontal transformation matrix, the vertical transformation matrix, and the planar transformation matrix is ​​used as the preset perspective transformation matrix;

[0060] The correction module is configured as follows:

[0061] Update at least one of the horizontal transformation parameters, vertical transformation parameters, and planar rotation parameters of the correction transformation matrix.

[0062] Optionally, the target correction transformation matrix includes a horizontal transformation matrix and a vertical transformation matrix, and the correction module includes:

[0063] The perspective angle determination module is configured to determine the angle between the vanishing point of each horizontal line in the target horizontal line set and the target image plane to obtain the horizontal perspective angle, and to determine the angle between the vanishing point of each vertical line in the target vertical line set and the target image plane to obtain the vertical perspective angle.

[0064] The correction direction determination module is configured to determine the correction direction for correcting the target image based on the numerical relationship between the horizontal perspective angle and the first preset angle, and the numerical relationship between the vertical perspective angle and the second preset angle. The correction direction includes horizontal correction and vertical correction.

[0065] The matrix determination module is configured to determine the target correction transformation matrix based on the correction direction in the horizontal transformation matrix and the vertical transformation matrix.

[0066] Optionally, the classification module is configured as follows:

[0067] Perform line detection on the target image, and for each detected line, determine the angle between the line and the horizontal axis, and the angle between the line and the vertical axis;

[0068] When the angle between the line and the horizontal axis is less than the angle between the line and the vertical axis, the line is classified into the horizontal line set. When the angle between the line and the horizontal axis is greater than or equal to the angle between the line and the vertical axis, the line is classified into the vertical line set.

[0069] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0070] processor;

[0071] Memory used to store processor-executable instructions;

[0072] The processor is configured as follows:

[0073] Obtain the target image to be corrected;

[0074] The straight lines in the target image are classified to obtain a set of horizontal lines and a set of vertical lines;

[0075] The horizontal lines in the set of horizontal lines are filtered to remove outliers, resulting in a target set of horizontal lines. Similarly, the vertical lines in the set of vertical lines are filtered to remove outliers, resulting in a target set of vertical lines.

[0076] The target correction transformation matrix is ​​determined based on the straight lines in the target horizontal line set and the target vertical line set, and the target image is transformed using the target correction transformation matrix to obtain the corrected image.

[0077] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the steps of the image correction method according to any one of the first aspects.

[0078] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0079] Lines in the target image to be corrected can be classified into horizontal and vertical sets. These sets are then filtered to remove outliers (noise lines), resulting in target horizontal and vertical line sets. Finally, the target correction transformation matrix is ​​calculated based on these sets to perform image correction. Compared to related techniques, filtering the line detection results reduces the impact of outliers on the transformation matrix calculation, improving its accuracy and thus reducing image distortion after correction, ultimately enhancing the display quality.

[0080] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0081] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0082] Figure 1 This is a flowchart illustrating an image correction method according to an exemplary embodiment.

[0083] Figure 2 This is a schematic diagram illustrating the angles between a straight line and the horizontal and vertical axes of the target image plane during the straight line classification process in an image correction method according to an exemplary embodiment.

[0084] Figure 3 This is a flowchart illustrating an image correction method according to another exemplary embodiment.

[0085] Figure 4This is a block diagram illustrating an image correction device according to another exemplary embodiment.

[0086] Figure 5 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0087] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0088] As mentioned in the background section, image correction in related technologies mainly involves first detecting straight lines in the image, then performing vanishing point detection on the detected straight lines, then obtaining the correction transformation matrix using the detected vanishing points, and finally using the correction transformation matrix to transform the image.

[0089] The inventors discovered that, according to the relevant technology, directly using the results of line detection to obtain the correction transformation matrix can lead to inaccurate vanishing point detection results because the results of line detection may contain a lot of noise. Therefore, using these noisy lines for vanishing point detection will result in inaccurate vanishing point detection results. Consequently, image correction based on the homography matrix obtained from the vanishing point detection results will result in severe distortion of the corrected image, affecting the display effect of the corrected image.

[0090] In view of this, the present disclosure provides a new image correction method to solve the problems existing in the related art, reduce image distortion after image correction, and improve the display effect after image correction.

[0091] First, let's describe the possible implementation scenarios of the image correction method provided in this disclosure. This image correction method can be applied to different types of electronic devices, such as mobile phones and tablets, allowing these devices to perform image correction. For example, by integrating this image correction method into a mobile phone's photo album function, when a user takes a photo and the subject suffers from perspective distortion due to a tilted shooting angle (e.g., a building is distorted due to shooting from below), the photo album function can be used to perform subsequent processing and optimization of the photo based on the user's image correction selection. This not only makes photo enhancement very convenient but also reduces distortion in the corrected image.

[0092] Figure 1 This is a flowchart illustrating an image correction method according to an exemplary embodiment, such as... Figure 1As shown, this image correction method can be applied to different types of electronic devices, including:

[0093] In step S11, the target image to be corrected is obtained.

[0094] In step S12, the straight lines in the target image are classified to obtain a set of horizontal lines and a set of vertical lines.

[0095] In step S13, the horizontal lines in the horizontal line set are filtered to remove outliers and obtain the target horizontal line set. Similarly, the vertical lines in the vertical line set are filtered to remove outliers and obtain the target vertical line set.

[0096] In step S14, the target correction transformation matrix is ​​determined based on the straight lines in the target horizontal line set and the target vertical line set, and the target image is transformed using the target correction transformation matrix to obtain the corrected image.

[0097] The above method classifies straight lines in the target image to be corrected, obtaining sets of horizontal and vertical lines. These sets are then filtered to remove outliers (noise lines), resulting in target horizontal and vertical line sets. Finally, the target correction transformation matrix is ​​calculated based on these sets to perform image correction. Compared to related techniques, filtering the line detection results reduces the impact of outliers (noise lines) on the transformation matrix calculation, improving the accuracy of the transformation matrix solution. This reduces image distortion after correction and improves the display effect.

[0098] To enable those skilled in the art to better understand the image correction method provided in this disclosure, the above steps are illustrated in detail below.

[0099] For example, in step S11, the target image to be corrected can be an image captured by an electronic device, or the target image can be an image downloaded by the electronic device via the network, or the target image can be a video frame image extracted from a video, etc. The embodiments of this disclosure do not limit the acquisition method and specific form of the target image.

[0100] After obtaining the target image to be corrected, the straight lines in the target image can be classified to obtain a set of horizontal lines and a set of vertical lines. Then, the correction transformation matrix can be obtained based on the set of horizontal lines and the set of vertical lines to perform image correction.

[0101] For example, line detection can be performed on the target image first to determine all straight lines included in the target image. For instance, the EDLines line detection algorithm and the LSD line detection algorithm can be used to detect lines in the target image. Then, for all the lines obtained from the line detection, line classification can be performed, classifying all lines in the target image as either horizontal or vertical lines.

[0102] In one possible approach, line detection can be performed on the target image, and for each detected line, the angle between the line and the horizontal axis in the target image plane, and the angle between the line and the vertical axis in the target image plane, can be determined. If the angle between the line and the horizontal axis is less than the angle between the line and the vertical axis, the line is classified into the horizontal line set. If the angle between the line and the horizontal axis is greater than or equal to the angle between the line and the vertical axis, the line is classified into the vertical line set.

[0103] For example, the top-left vertex of the target image can be used as the origin, the direction to the right of the top-left vertex can be defined as the positive X-axis, and the direction downwards from the top-left vertex can be defined as the negative Y-axis, thus establishing a coordinate axis. Of course, coordinate axes within the target image plane can also be established in other ways, and this embodiment of the present disclosure does not limit this. After establishing the coordinate axes within the target image plane, the horizontal axis within the target plane can be any horizontal axis that coincides with or is parallel to the X-axis of the coordinate axis, and the vertical axis within the target plane can be any vertical axis that coincides with or is parallel to the Y-axis of the coordinate axis, and this embodiment of the present disclosure does not limit this.

[0104] For any detected line, we can first determine the angle between the line and the horizontal axis of the target image plane, and the angle between the line and the vertical axis of the target image plane. Then, based on the relationship between these two angles, the lines in the target image are classified as horizontal or vertical lines. For example, the horizontal axis of the target plane is as follows: Figure 2 The X-axis is shown, and the vertical axis in the target plane is as follows: Figure 2 As shown, the angle between line L and the X-axis is 'a', and the angle between line L and the Y-axis is 'b'. Since angle 'a' is less than angle 'b', line L can be classified as a horizontal line. Conversely, if the angle between line L and the X-axis is greater than or equal to the angle between line L and the Y-axis, then line L can be classified as a vertical line.

[0105] After classifying the straight lines in the target image into sets of horizontal and vertical lines, in order to reduce the influence of noisy straight lines on the subsequent solution of the correction transformation matrix and reduce the deformation of the image after correction, the straight lines in the sets of horizontal and vertical lines can be filtered.

[0106] In one possible approach, filtering the lines in the set of horizontal lines to remove outliers and obtain the target set of horizontal lines can be done by: selecting any two lines in the set of horizontal lines as initial target horizontal lines, and performing the following process: determining the distance between the vanishing point of the target horizontal line and every other horizontal line in the set of horizontal lines; if the distance is less than a preset distance, classifying the other horizontal lines into a candidate set that includes the target horizontal line, and selecting any two horizontal lines other than the target horizontal line in the set of horizontal lines as new target horizontal lines, until every horizontal line in the set of horizontal lines is selected as a target horizontal line; finally, determining the set with the most horizontal lines among the multiple candidate sets as the target set of horizontal lines.

[0107] The preset distance can be set according to the actual situation, and this embodiment does not limit it.

[0108] It should be understood that a vanishing point is a point where two or more straight lines converge towards the distant horizon. In this embodiment of the present disclosure, to filter outliers from the set of horizontal lines, two horizontal lines can be randomly selected first, and their intersection point outside the plane can be calculated. This intersection point is the vanishing point of the two horizontal lines. Then, the distances between the other horizontal lines in the set (excluding the two selected lines) and the vanishing point can be calculated. If the distance between the other horizontal lines and the vanishing point is less than a preset distance, then the other horizontal lines are considered to belong to the same set of horizontal lines as the two horizontal lines corresponding to the vanishing point, and can be classified into a candidate set including the two horizontal lines corresponding to the vanishing point. If the distance between the other horizontal lines and the vanishing point is greater than or equal to the preset distance, then the other horizontal lines are considered outliers, and do not belong to the same set as the two horizontal lines corresponding to the vanishing point. Thus, the remaining horizontal lines in the set can be classified according to the vanishing points of the two selected lines to obtain a candidate set. Afterward, two lines are randomly selected again, and the above process is repeated until every horizontal line in the set has been selected for vanishing point calculation. Therefore, we can obtain multiple candidate sets S(L1), S(L2), ..., S(Ln), where n is a positive integer greater than 2. Finally, we can determine the set with the most horizontal lines from the multiple candidate sets as the target set of horizontal lines.

[0109] Similarly, in one possible approach, the lines in the set of vertical lines are filtered to remove outliers, resulting in a target set of vertical lines. This can be achieved by selecting any two lines from the set of vertical lines as initial target vertical lines and performing the following steps: determining the distance between the vanishing point of the target vertical line and every other vertical line in the set of vertical lines; if the distance is less than a preset distance, classifying the other vertical lines into a candidate set that includes the target vertical line; and selecting any two vertical lines from the set of vertical lines other than the target vertical line as new target vertical lines, until every vertical line in the set of vertical lines is selected as a target vertical line. Finally, the set with the most vertical lines among the multiple candidate sets is determined as the target set of vertical lines.

[0110] By using the above method, outlier lines (i.e., noisy lines) can be filtered out from the set of horizontal lines and the set of vertical lines respectively, thereby reducing the impact of outlier lines on the solution of the transformation matrix, improving the accuracy of the solution of the transformation matrix, and thus reducing the image distortion after image correction and improving the display effect of the image after correction.

[0111] After filtering out the outliers in the sets of horizontal and vertical lines, thus obtaining the target sets of horizontal and vertical lines, the target correction transformation matrix can be determined based on the lines in the target sets of horizontal and vertical lines.

[0112] In one possible approach, determining the target correction transformation matrix based on the lines in the target horizontal and vertical line sets can be achieved as follows: First, determine the angle between the vanishing point of each horizontal line in the target horizontal line set and the target image plane to obtain the horizontal perspective angle. Then, determine the angle between the vanishing point of each vertical line in the target vertical line set and the target image plane to obtain the vertical perspective angle. Next, based on the numerical relationship between the horizontal perspective angle and a first preset angle, and the numerical relationship between the vertical perspective angle and a second preset angle, determine the correction direction for correcting the target image. This correction direction can include either horizontal or vertical correction. Finally, in the horizontal and vertical transformation matrices, determine the target correction transformation matrix based on the correction direction.

[0113] It should be understood that in practical applications, image correction typically includes both horizontal and vertical perspective correction. Therefore, the direction of image correction can be determined first, and then the corresponding correction method can be selected based on that direction, rather than performing both horizontal and vertical perspective correction on every single image. This reduces computational load and improves the efficiency of image correction.

[0114] For example, if the vanishing point coordinates of each horizontal line in the target horizontal line set are (x, y, z), the angle between the vanishing point of each horizontal line and the target plane can be calculated using the following formula to obtain the horizontal perspective angle:

[0115]

[0116] Similarly, the vertical perspective angle can be calculated using the formula described above after obtaining the vanishing point coordinates of each vertical line in the target vertical line set.

[0117] After obtaining the horizontal and vertical perspective angles, the correction direction for correcting the target image can be determined based on the numerical relationship between the horizontal perspective angle and a first preset angle, and the numerical relationship between the vertical perspective angle and a second preset angle. The first and second preset angles can be the same. For example, both the first and second preset angles can be set to 15 degrees. In this case, if the horizontal perspective angle is less than 15 degrees, the correction direction is determined to be horizontal correction; otherwise, horizontal correction is not required. If the vertical perspective angle is less than 15 degrees, the correction direction is determined to be vertical correction; otherwise, vertical correction is not required. If both the horizontal and vertical perspective angles are less than 15 degrees, the correction direction is determined to be both horizontal and vertical correction. If both the horizontal and vertical perspective angles are greater than 15 degrees, no correction processing is required.

[0118] In other words, when the target correction transformation matrix includes both horizontal and vertical transformation matrices, the target correction transformation matrix can be determined based on the correction direction to improve the efficiency of image correction.

[0119] In one possible approach, determining the target correction transformation matrix based on the lines in the target horizontal and vertical line sets can be an iterative process. For example, a preset perspective transformation matrix can be used as the initial correction transformation matrix, and the correction transformation process can be iteratively performed on the lines in the horizontal and vertical line sets until the preset number of iterations is reached, resulting in the target correction transformation matrix. The correction transformation process includes: transforming the lines in the horizontal and vertical line sets according to the correction transformation matrix of the current iteration; determining the target optimization function based on the dot product of the transformed horizontal lines with the horizontal axis of the target image plane and the dot product of the transformed vertical lines with the vertical axis of the target image plane; updating the correction transformation matrix with the goal of minimizing the target optimization function; and then using the updated correction transformation matrix as the correction transformation matrix for the next iteration.

[0120] The preset number of times can be set according to the actual situation, and this embodiment does not limit it.

[0121] In one possible approach, the preset perspective transformation matrix can be determined as follows: First, a horizontal transformation matrix is ​​determined based on preset horizontal transformation parameters, the height of the target image, and the width of the target image. These horizontal transformation parameters are used for corrective transformation along the horizontal axis. Then, a vertical transformation matrix is ​​determined based on preset vertical transformation parameters, the height of the target image, and the width of the target image. These vertical transformation parameters are used for corrective transformation along the vertical axis. Finally, a planar transformation matrix is ​​determined based on preset planar rotation parameters, the height of the target image, and the width of the target image. These planar rotation parameters are used for rotational transformation of the target image plane. Finally, the product of the horizontal transformation matrix, the vertical transformation matrix, and the planar transformation matrix can be used as the preset perspective transformation matrix. Correspondingly, updating the corrective transformation matrix can be done by updating at least one of the horizontal transformation parameters, the vertical transformation parameters, and the planar rotation parameters of the corrective transformation matrix.

[0122] It should be understood that the image correction in this embodiment can be decomposed into three parts: perspective correction in the horizontal axis direction, perspective correction in the vertical axis direction, and in-plane rotation correction. The purpose of the horizontal axis correction is to ensure that all horizontal lines are parallel and perpendicular to the vertical axis after correction, and the purpose of the vertical axis correction is to ensure that all vertical lines are parallel and perpendicular to the horizontal axis after correction. Furthermore, since the camera is often not parallel to the horizon during photography, in-plane rotation can also be performed to align the lines in the image with the horizon.

[0123] For example, perspective correction on the horizontal and vertical axes simulates the effect of a shifting lens. Straight lines in one direction can be corrected using lens shift parameters shift_x and shift_y, respectively, and the camera's misalignment with the horizon can be corrected using the plane rotation parameter angle. In this case, the process of solving for the correction coefficients can include: first, initializing shift_x, shift_y, and angle to 0, i.e., setting the preset horizontal transformation parameters, preset vertical transformation parameters, and preset plane rotation parameters to 0. Then, based on the preset horizontal transformation parameters, the height of the target image, and the width of the target image, the horizontal transformation matrix M can be determined. x :

[0124]

[0125] Where e_x = exp(shift_x), height represents the height of the target image, and width represents the width of the target image.

[0126] Similarly, the horizontal transformation matrix M can be determined based on the preset vertical transformation parameters, the height of the target image, and the width of the target image. y :

[0127]

[0128] Where e_y = exp(shift_y).

[0129] In addition, the plane transformation matrix M can be determined based on preset plane rotation parameters, the height of the target image, and the width of the target image. r :

[0130]

[0131] Among them, t x =-0.5×height×cos(angle)+0.5×width×sin(angle)+0.5×height, t y =-0.5×height×sin(angle)-0.5×width×cos(angle)+0.5×width.

[0132] Using the above method, the horizontal transformation matrix, vertical transformation matrix, and planar transformation matrix can be obtained. Then, the product of these three matrices can be used as the preset perspective transformation matrix. Specifically, the preset perspective transformation matrix can be determined as follows:

[0133] M = M x ×M y ×M r

[0134] Then, the preset perspective transformation matrix can be used as the initial correction transformation matrix to iteratively perform the correction transformation process on the lines in the target horizontal and vertical line sets until the preset number of iterations is reached. In each iteration, the lines in the horizontal and vertical line sets can be transformed according to the correction transformation matrix of this iteration. Then, the target optimization function is determined based on the dot product of the transformed horizontal line with the vertical axis in the target image plane and the dot product of the transformed vertical line with the horizontal axis in the target image plane.

[0135] It should be understood that when calculating the dot product of the transformed horizontal line and the vertical axis, the dot product is 0 if the transformed line is perpendicular to the horizontal axis, and vice versa. Similarly, when calculating the dot product of the transformed vertical line and the horizontal axis, the dot product is 0 if the transformed line is perpendicular to the vertical axis, and vice versa. Therefore, in this embodiment, the target optimization function can be determined based on the dot product of the transformed horizontal line and the vertical axis in the target image plane, and the dot product of the transformed vertical line and the horizontal axis in the target image plane. For example, the dot product of the transformed horizontal line and the vertical axis in the target image plane, and the dot product of the transformed vertical line and the horizontal axis in the target image plane can be directly used as the target optimization function. Alternatively, the target optimization function can be determined by the weighted sum of the dot product of the transformed horizontal line and the vertical axis in the target image plane, where the weights can be the lengths of the corresponding lines.

[0136] Then, with the goal of minimizing the objective optimization function, the correction transformation matrix is ​​updated. For example, at least one of the horizontal transformation parameters, vertical transformation parameters, and planar rotation parameters of the correction transformation matrix can be updated. Then, based on the updated correction transformation parameters, the process of solving the correction transformation matrix is ​​repeated, and the straight lines in the target horizontal and vertical line sets are corrected according to the newly solved correction transformation matrix. The correction transformation matrix is ​​then updated again based on the results of the correction transformation. This process is iterated continuously until the preset number of iterations is reached. The correction transformation matrix at the point where the iteration stops is then determined as the final target correction transformation matrix used for image correction.

[0137] The above method can be used to further optimize the target correction transformation matrix for image correction through iterative solution, thereby further reducing the distortion of the image after correction and improving the display effect of the image after correction.

[0138] After obtaining the target correction transformation matrix, the target image can be transformed according to this matrix to obtain the corrected image. Since noisy lines are filtered out before solving for the target correction transformation matrix, the influence of noisy lines on the solution is reduced, thereby minimizing image distortion after correction and improving the display effect.

[0139] The image correction method provided in this disclosure will now be described through another exemplary embodiment.

[0140] Reference Figure 3 The image correction method may include:

[0141] In step S301, the target image to be corrected is acquired.

[0142] In step S302, line detection is performed on the target image to obtain all straight lines in the target image.

[0143] In step S303, for each detected straight line, the angle between the straight line and the horizontal axis in the target image plane, and the angle between the straight line and the vertical axis in the target image plane are determined.

[0144] In step S304, if the angle between the line and the horizontal axis is less than the angle between the line and the vertical axis, the line is classified into the set of horizontal lines; if the angle between the line and the horizontal axis is greater than or equal to the angle between the line and the vertical axis, the line is classified into the set of vertical lines.

[0145] In step S305, the straight lines in the horizontal line set are filtered to remove outliers, resulting in the target horizontal line set. Similarly, the vertical lines in the vertical line set are filtered to remove outliers, resulting in the target vertical line set. It should be understood that this filtering process has already been explained above and will not be repeated here.

[0146] In step S306, the vanishing point of each horizontal line in the target horizontal line set is determined to be at the angle between the vanishing point and the target image plane to obtain the horizontal perspective angle, and the vanishing point of each vertical line in the target vertical line set is determined to be at the angle between the vanishing point and the target image plane to obtain the vertical perspective angle.

[0147] In step S307, the correction direction for correcting the target image is determined based on the numerical relationship between the horizontal perspective angle and the first preset angle, and the numerical relationship between the vertical perspective angle and the second preset angle.

[0148] In step S308, an initial correction transformation matrix is ​​determined based on the correction direction and a preset perspective transformation matrix. It should be understood that if the correction direction is lateral correction, the vertical transformation matrix corresponding to vertical correction in the preset perspective transformation matrix can be set to 1, i.e., M as mentioned above. y The value is 1. If the correction direction is vertical, the horizontal transformation matrix corresponding to the horizontal correction in the preset perspective transformation matrix can be set to 1, i.e., M in the above text. x The value is 1.

[0149] In step S309, a correction transformation process is performed on the straight lines in the horizontal and vertical line sets according to the correction transformation matrix.

[0150] In step S310, it is determined whether the number of iterations has reached the preset number. If so, proceed to step S312; otherwise, proceed to step S311.

[0151] In step S311, the correction transformation matrix is ​​updated based on the correction transformation result, and the process returns to step S309. It should be understood that this update process has been explained above and will not be repeated here.

[0152] In step S312, the target image is transformed according to the target correction transformation matrix after the iteration to obtain the corrected image.

[0153] The specific implementation methods for each of the above steps have been described in detail above and will not be repeated here. It should also be understood that, for the sake of simplicity, the above method embodiments are described as a series of actions; however, those skilled in the art should understand that this disclosure is not limited to the order of actions described above. Furthermore, those skilled in the art should also understand that the embodiments described above are preferred embodiments, and the steps involved are not necessarily essential to this disclosure.

[0154] The above method removes outlier lines (i.e., noise lines) from the line detection results, yielding a target horizontal line set and a target vertical line set. Then, the target correction transformation matrix is ​​calculated based on these sets for image correction. Compared to related techniques, this method reduces the impact of outlier lines on the transformation matrix calculation, improving its accuracy and thus reducing image distortion and enhancing the display quality after correction. Furthermore, it allows for iterative calculation of the correction transformation matrix, further optimizing it compared to a one-time solution, further improving accuracy and minimizing distortion, ultimately enhancing the display quality.

[0155] Based on the same concept, this disclosure also provides an image correction device that can be incorporated into or entirely of an electronic device through software, hardware, or a combination of both. (See also...) Figure 4 The image correction device 400 includes an acquisition module 401, a classification module 402, a filtering module 403, and a correction module 404.

[0156] The acquisition module 401 is configured to acquire the target image to be corrected.

[0157] The classification module 402 is configured to classify the straight lines in the target image to obtain a set of horizontal lines and a set of vertical lines.

[0158] The filtering module 403 is configured to filter the horizontal lines in the set of horizontal lines to remove outliers from the set of horizontal lines, thereby obtaining a target set of horizontal lines, and to filter the vertical lines in the set of vertical lines to remove outliers from the set of vertical lines, thereby obtaining a target set of vertical lines.

[0159] The correction module 404 is configured to determine a target correction transformation matrix based on the straight lines in the target horizontal line set and the target vertical line set, and to perform transformation processing on the target image using the target correction transformation matrix to obtain a corrected image.

[0160] Optionally, the filtering module 403 is configured as follows:

[0161] Select any two horizontal lines from the set of horizontal lines as the initial target horizontal lines, and perform the following procedure:

[0162] Determine the distance between the vanishing point of the target horizontal line and each other horizontal line in the set of horizontal lines. When the distance is less than a preset distance, classify the other horizontal lines into a candidate set that includes the target horizontal line, and select any two horizontal lines other than the target horizontal line in the set of horizontal lines as new target horizontal lines, until every horizontal line in the set of horizontal lines is selected as a target horizontal line.

[0163] Among the multiple candidate sets obtained, the set with the most horizontal lines is determined as the target horizontal line set.

[0164] Optionally, the filtering module 403 is configured as follows:

[0165] Select any two vertical lines from the set of vertical lines as the initial target vertical lines, and perform the following procedure:

[0166] Determine the distance between the vanishing point of the target vertical line and each other vertical line in the set of vertical lines. When the distance is less than a preset distance, classify the other vertical lines into a candidate set that includes the target vertical line, and select any two vertical lines other than the target vertical line in the set of vertical lines as new target vertical lines, until every vertical line in the set of vertical lines is selected as a target vertical line.

[0167] Among the multiple candidate sets obtained, the set with the most vertical lines is determined as the target vertical line set.

[0168] Optionally, the correction module 404 is configured as follows:

[0169] Using a preset perspective transformation matrix as the initial correction transformation matrix, the correction transformation process is iteratively performed on the lines in the set of horizontal lines and the set of vertical lines until the preset number of iterations is reached, thereby obtaining the target correction transformation matrix. The correction transformation process includes:

[0170] The straight lines in the set of horizontal lines and the set of vertical lines are transformed according to the correction transformation matrix of this iteration process;

[0171] Based on the dot product of the transformed horizontal line and the vertical axis in the target image plane and the dot product of the transformed vertical line and the horizontal axis in the target image plane, the target optimization function is determined, and the correction transformation matrix is ​​updated with the goal of minimizing the target optimization function.

[0172] The updated correction transformation matrix is ​​used as the correction transformation matrix for the next iteration.

[0173] Optionally, the device 400 further includes a matrix determination module for determining a preset perspective transformation matrix correction, the matrix determination module being configured to:

[0174] Based on preset horizontal transformation parameters, the height of the target image, and the width of the target image, a horizontal transformation matrix is ​​determined, wherein the horizontal transformation parameters are used to perform corrective transformation along the horizontal axis.

[0175] Based on preset vertical transformation parameters, the height of the target image, and the width of the target image, a vertical transformation matrix is ​​determined, wherein the vertical transformation parameters are used to perform corrective transformation along the vertical axis.

[0176] Based on preset planar rotation parameters, the height of the target image, and the width of the target image, a planar transformation matrix is ​​determined, wherein the planar rotation parameters are used to perform rotation transformation on the plane of the target image;

[0177] The product of the horizontal transformation matrix, the vertical transformation matrix, and the planar transformation matrix is ​​used as the preset perspective transformation matrix;

[0178] The correction module is configured as follows:

[0179] Update at least one of the horizontal transformation parameters, vertical transformation parameters, and planar rotation parameters of the correction transformation matrix.

[0180] Optionally, the target correction transformation matrix includes a horizontal transformation matrix and a vertical transformation matrix, and the correction module 404 is configured as follows:

[0181] The vanishing point of each horizontal line in the target horizontal line set is determined to be at the angle between the vanishing point of each horizontal line in the target vertical line set and the target image plane to obtain the horizontal perspective angle; and the vanishing point of each vertical line in the target vertical line set is determined to be at the angle between the vanishing point of each vertical line in the target vertical line set and the target image plane to obtain the vertical perspective angle.

[0182] Based on the numerical relationship between the horizontal perspective angle and the first preset angle, and the numerical relationship between the vertical perspective angle and the second preset angle, the correction direction for correcting the target image is determined, and the correction direction includes horizontal correction and vertical correction.

[0183] In the horizontal transformation matrix and the vertical transformation matrix, the target correction transformation matrix is ​​determined according to the correction direction.

[0184] Optionally, the classification module 402 is configured as follows:

[0185] Perform line detection on the target image, and for each detected line, determine the angle between the line and the horizontal axis, and the angle between the line and the vertical axis;

[0186] When the angle between the line and the horizontal axis is less than the angle between the line and the vertical axis, the line is classified into the horizontal line set. When the angle between the line and the horizontal axis is greater than or equal to the angle between the line and the vertical axis, the line is classified into the vertical line set.

[0187] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0188] Based on the same concept, this disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the image correction method provided in this disclosure.

[0189] Based on the same concept, this disclosure also provides an electronic device, including:

[0190] processor;

[0191] Memory used to store processor-executable instructions;

[0192] The processor is configured as follows:

[0193] Obtain the target image to be corrected;

[0194] The straight lines in the target image are classified to obtain a set of horizontal lines and a set of vertical lines;

[0195] The horizontal lines in the set of horizontal lines are filtered to remove outliers, resulting in a target set of horizontal lines. Similarly, the vertical lines in the set of vertical lines are filtered to remove outliers, resulting in a target set of vertical lines.

[0196] The target correction transformation matrix is ​​determined based on the straight lines in the target horizontal line set and the target vertical line set, and the target image is transformed using the target correction transformation matrix to obtain the corrected image.

[0197] Of course, the processor of the electronic device can also execute the steps of any of the above image correction methods, and this disclosure does not limit this.

[0198] In one possible manner, the block diagram of the electronic device is as follows: Figure 5 As shown. It should be understood that the electronic device 500 can be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0199] Reference Figure 5 The electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0200] Processing component 802 typically controls the overall operation of electronic device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the image correction method described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0201] Memory 804 is configured to store various types of data to support the operation of electronic device 800. Examples of this data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0202] Power component 806 provides power to various components of electronic device 800. Power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800.

[0203] Multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0204] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when electronic device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0205] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0206] Sensor assembly 814 includes one or more sensors for providing state assessments of various aspects of electronic device 800. For example, sensor assembly 814 can detect the on / off state of electronic device 800, the relative positioning of components such as the display and keypad of electronic device 800, changes in position of electronic device 800 or a component of electronic device 800, the presence or absence of user contact with electronic device 800, orientation or acceleration / deceleration of electronic device 800, and temperature changes of electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0207] Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. Electronic device 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0208] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the image correction method described above.

[0209] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of an electronic device 800 to complete the image correction method described above. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0210] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a programmable device, the computer program having a code portion for performing the image correction method described above when executed by the programmable device.

[0211] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0212] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image correction method, characterized in that, The method includes: Obtain the target image to be corrected; The straight lines in the target image are classified to obtain a set of horizontal lines and a set of vertical lines; The horizontal lines in the set of horizontal lines are filtered to remove outliers, resulting in a target set of horizontal lines. Similarly, the vertical lines in the set of vertical lines are filtered to remove outliers, resulting in a target set of vertical lines. The filtering process for the horizontal lines in the set of horizontal lines to remove outliers and obtain the target set of horizontal lines includes: selecting any two horizontal lines from the set of horizontal lines as initial target horizontal lines, and performing the following steps: determining the distance between the vanishing point of the target horizontal line and every other horizontal line in the set of horizontal lines; if the distance is less than a preset distance, classifying the other horizontal lines into a candidate set including the target horizontal lines; and selecting any two horizontal lines from the set of horizontal lines other than the target horizontal lines as new target horizontal lines, until every horizontal line in the set of horizontal lines is selected as a target horizontal line; and determining the set with the most horizontal lines among the obtained candidate sets as the target set of horizontal lines. The target correction transformation matrix is ​​determined based on the straight lines in the target horizontal line set and the target vertical line set, and the target image is transformed using the target correction transformation matrix to obtain the corrected image.

2. The method according to claim 1, characterized in that, The step of filtering the vertical lines in the set of vertical lines to remove outliers and obtain the target set of vertical lines includes: Select any two vertical lines from the set of vertical lines as the initial target vertical lines, and perform the following procedure: Determine the distance between the vanishing point of the target vertical line and each other vertical line in the set of vertical lines. If the distance is less than a preset distance, classify the other vertical lines into a candidate set that includes the target vertical line, and select any two vertical lines other than the target vertical line in the set of vertical lines as new target vertical lines, until every vertical line in the set of vertical lines is selected as a target vertical line. Among the multiple candidate sets obtained, the set with the most vertical lines is determined as the target vertical line set.

3. The method according to any one of claims 1-2, characterized in that, The step of determining the target correction transformation matrix based on the straight lines in the target horizontal line set and the target vertical line set includes: Using a preset perspective transformation matrix as the initial correction transformation matrix, the correction transformation process is iteratively performed on the lines in the set of horizontal lines and the set of vertical lines until the preset number of iterations is reached, thereby obtaining the target correction transformation matrix. The correction transformation process includes: The straight lines in the set of horizontal lines and the set of vertical lines are transformed according to the correction transformation matrix of this iteration process; Based on the dot product of the transformed horizontal line and the vertical axis in the target image plane and the dot product of the transformed vertical line and the horizontal axis in the target image plane, the target optimization function is determined, and the correction transformation matrix is ​​updated with the goal of minimizing the target optimization function. The updated correction transformation matrix is ​​used as the correction transformation matrix for the next iteration.

4. The method according to claim 3, characterized in that, The preset perspective transformation matrix is ​​determined in the following way: A horizontal transformation matrix is ​​determined based on preset horizontal transformation parameters, the height of the target image, and the width of the target image. The horizontal transformation parameters are used to perform corrective transformation along the horizontal axis. Based on preset vertical transformation parameters, the height of the target image, and the width of the target image, a vertical transformation matrix is ​​determined, wherein the vertical transformation parameters are used to perform corrective transformation along the vertical axis. Based on preset planar rotation parameters, the height of the target image, and the width of the target image, a planar transformation matrix is ​​determined, wherein the planar rotation parameters are used to perform rotation transformation on the plane of the target image; The product of the horizontal transformation matrix, the vertical transformation matrix, and the planar transformation matrix is ​​used as the preset perspective transformation matrix; Updating the correction transformation matrix includes: Update at least one of the horizontal transformation parameters, vertical transformation parameters, and planar rotation parameters of the correction transformation matrix.

5. The method according to any one of claims 1-2, characterized in that, The target correction transformation matrix includes a horizontal transformation matrix and a vertical transformation matrix. Determining the target correction transformation matrix based on the straight lines in the target horizontal line set and the target vertical line set includes: The vanishing point of each horizontal line in the target horizontal line set is determined to be at the angle between the vanishing point of each horizontal line in the target vertical line set and the target image plane to obtain the horizontal perspective angle; and the vanishing point of each vertical line in the target vertical line set is determined to be at the angle between the vanishing point of each vertical line in the target vertical line set and the target image plane to obtain the vertical perspective angle. Based on the numerical relationship between the horizontal perspective angle and the first preset angle, and the numerical relationship between the vertical perspective angle and the second preset angle, the correction direction for correcting the target image is determined, and the correction direction includes horizontal correction or vertical correction. In the horizontal transformation matrix and the vertical transformation matrix, the target correction transformation matrix is ​​determined according to the correction direction.

6. The method according to any one of claims 1-2, characterized in that, The process of classifying the straight lines in the target image to obtain a set of horizontal lines and a set of vertical lines includes: Line detection is performed on the target image, and for each detected line, the angle between the line and the horizontal axis in the target image plane and the angle between the line and the vertical axis in the target image plane are determined. If the angle between the line and the horizontal axis is less than the angle between the line and the vertical axis, the line is classified into the set of horizontal lines. If the angle between the line and the horizontal axis is greater than or equal to the angle between the line and the vertical axis, the line is classified into the set of vertical lines.

7. An image correction device, characterized in that, The device includes: The acquisition module is configured to acquire the target image to be corrected. The classification module is configured to classify the straight lines in the target image to obtain a set of horizontal lines and a set of vertical lines; The filtering module is configured to filter the horizontal lines in the set of horizontal lines to remove outlier horizontal lines in the set of horizontal lines, thereby obtaining a target set of horizontal lines; and to filter the vertical lines in the set of vertical lines to remove outlier vertical lines in the set of vertical lines, thereby obtaining a target set of vertical lines. The filtering module is configured to: select any two horizontal lines from the set of horizontal lines as initial target horizontal lines, and perform the following process: determine the distance between the vanishing point of the target horizontal line and each other horizontal line in the set of horizontal lines; when the distance is less than a preset distance, classify the other horizontal lines into a candidate set including the target horizontal line, and select any two horizontal lines other than the target horizontal line from the set of horizontal lines as new target horizontal lines, until every horizontal line in the set of horizontal lines is selected as a target horizontal line; determine the set with the most horizontal lines among the multiple candidate sets as the target horizontal line set; The correction module is configured to determine a target correction transformation matrix based on the straight lines in the target horizontal line set and the target vertical line set, and to perform transformation processing on the target image using the target correction transformation matrix to obtain a corrected image.

8. The apparatus according to claim 7, characterized in that, The filtering module is configured as follows: Select any two vertical lines from the set of vertical lines as the initial target vertical lines, and perform the following procedure: Determine the distance between the vanishing point of the target vertical line and each other vertical line in the set of vertical lines. When the distance is less than a preset distance, classify the other vertical lines into a candidate set that includes the target vertical line, and select any two vertical lines other than the target vertical line in the set of vertical lines as new target vertical lines, until every vertical line in the set of vertical lines is selected as a target vertical line. Among the multiple candidate sets obtained, the set with the most vertical lines is determined as the target vertical line set.

9. The apparatus according to any one of claims 7-8, characterized in that, The correction module is configured as follows: Using a preset perspective transformation matrix as the initial correction transformation matrix, the correction transformation process is iteratively performed on the lines in the set of horizontal lines and the set of vertical lines until the preset number of iterations is reached, thereby obtaining the target correction transformation matrix. The correction transformation process includes: The straight lines in the set of horizontal lines and the set of vertical lines are transformed according to the correction transformation matrix of this iteration process; Based on the dot product of the transformed horizontal line and the vertical axis in the target image plane and the dot product of the transformed vertical line and the horizontal axis in the target image plane, the target optimization function is determined, and the correction transformation matrix is ​​updated with the goal of minimizing the target optimization function. The updated correction transformation matrix is ​​used as the correction transformation matrix for the next iteration.

10. The apparatus according to claim 9, characterized in that, The device further includes a matrix determination module for determining the preset perspective transformation matrix correction, the matrix determination module being configured to: A horizontal transformation matrix is ​​determined based on preset horizontal transformation parameters, the height of the target image, and the width of the target image. The horizontal transformation parameters are used to perform corrective transformation along the horizontal axis. Based on preset vertical transformation parameters, the height of the target image, and the width of the target image, a vertical transformation matrix is ​​determined, wherein the vertical transformation parameters are used to perform corrective transformation along the vertical axis. Based on preset planar rotation parameters, the height of the target image, and the width of the target image, a planar transformation matrix is ​​determined, wherein the planar rotation parameters are used to perform rotation transformation on the plane of the target image; The product of the horizontal transformation matrix, the vertical transformation matrix, and the planar transformation matrix is ​​used as the preset perspective transformation matrix; The correction module is configured as follows: Update at least one of the horizontal transformation parameters, vertical transformation parameters, and planar rotation parameters of the correction transformation matrix.

11. The apparatus according to any one of claims 7-8, characterized in that, The target correction transformation matrix includes a horizontal transformation matrix and a vertical transformation matrix, and the correction module is configured as follows: The vanishing point of each horizontal line in the target horizontal line set is determined to be at the angle between the vanishing point of each horizontal line in the target vertical line set and the target image plane to obtain the horizontal perspective angle; and the vanishing point of each vertical line in the target vertical line set is determined to be at the angle between the vanishing point of each vertical line in the target vertical line set and the target image plane to obtain the vertical perspective angle. Based on the numerical relationship between the horizontal perspective angle and the first preset angle, and the numerical relationship between the vertical perspective angle and the second preset angle, the correction direction for correcting the target image is determined, and the correction direction includes horizontal correction and vertical correction. In the horizontal transformation matrix and the vertical transformation matrix, the target correction transformation matrix is ​​determined according to the correction direction.

12. The apparatus according to any one of claims 7-8, characterized in that, The classification module is configured as follows: Line detection is performed on the target image, and for each detected line, the angle between the line and the horizontal axis in the target image plane and the angle between the line and the vertical axis in the target image plane are determined. When the angle between the line and the horizontal axis is less than the angle between the line and the vertical axis, the line is classified into the horizontal line set. When the angle between the line and the horizontal axis is greater than or equal to the angle between the line and the vertical axis, the line is classified into the vertical line set.

13. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured as follows: Obtain the target image to be corrected; The straight lines in the target image are classified to obtain a set of horizontal lines and a set of vertical lines; The horizontal lines in the set of horizontal lines are filtered to remove outliers, resulting in a target set of horizontal lines. Similarly, the vertical lines in the set of vertical lines are filtered to remove outliers, resulting in a target set of vertical lines. The filtering process for the horizontal lines in the set of horizontal lines to remove outliers and obtain the target set of horizontal lines includes: selecting any two horizontal lines from the set of horizontal lines as initial target horizontal lines, and performing the following steps: determining the distance between the vanishing point of the target horizontal line and every other horizontal line in the set of horizontal lines; if the distance is less than a preset distance, classifying the other horizontal lines into a candidate set including the target horizontal lines; and selecting any two horizontal lines from the set of horizontal lines other than the target horizontal lines as new target horizontal lines, until every horizontal line in the set of horizontal lines is selected as a target horizontal line; and determining the set with the most horizontal lines among the obtained candidate sets as the target set of horizontal lines. The target correction transformation matrix is ​​determined based on the straight lines in the target horizontal line set and the target vertical line set, and the target image is transformed using the target correction transformation matrix to obtain the corrected image.

14. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method described in any one of claims 1 to 6.

Images