Image tilt correction method and device, electronic equipment and storage medium
By preprocessing the image and horizontal projection variance calculation, the problem of small angle tilt image correction is solved, the rapid and accurate rotation of the image is achieved, the accuracy of text detection and information extraction is improved, and a solid foundation is provided for business activities.
Patent Information
- Application Number
- CN202411129479.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-16
- Publication Date
- 2024-12-06
AI Technical Summary
The prior art cannot effectively correct the tilt correction of images with small angle tilts, affecting the extraction and recognition effects of text images.
By preprocessing the image to be corrected, the shadowed image is obtained and the rotation and horizontal projection variance calculation is performed within the preset range to determine the optimal rotation angle to achieve accurate tilt correction of the image.
Quickly and accurately rotate text images at small angles to horizontal states, improving the accuracy and efficiency of text detection recognition and information extraction, and is suitable for multiple business and image processing fields.
Smart Images

Figure CN119091444A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to an image tilt correction method, device, electronic equipment and storage medium. Background Art
[0002] Extracting specific information from complex documents is particularly important in many business activities, such as information extraction from damage assessment photos in the financial field. This is a critical and complex process due to the wide variety of business documents and templates, such as contracts, invoices, reports or news articles.
[0003] In the current document information extraction (DIE) process of text images, the tilt of the image may affect the difficulty and accuracy of extraction. The tilt of the image will affect the coordinate relationship of the context and weaken the context relationship, and is not conducive to maximizing the effect of algorithms such as OCR.
[0004] At present, there are many deep learning-based algorithms that can achieve large-angle text image tilt correction (such as 90°, 180°), but they are powerless for small-angle tilt (within 15°). Summary of the invention
[0005] The purpose of the present invention is to overcome the above technical deficiencies and provide an image tilt correction method, device, electronic device and storage medium to solve the technical problem that the prior art cannot perform tilt correction on images tilted at a small angle.
[0006] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:
[0007] In a first aspect, the present invention provides an image tilt correction method, comprising the following steps:
[0008] Acquire an image to be corrected, and preprocess the image to be corrected to obtain a shadow-removed image;
[0009] Rotating the shadow-removed image several times within a preset range, and calculating the variance of the shadow-removed image in horizontal projection after each rotation;
[0010] Comparing the calculated several variances to obtain a comparison result;
[0011] Based on the comparison result, the image to be corrected or the image with shadow removed is rotated to complete the tilt correction of the image to be corrected.
[0012] In some embodiments, the acquiring of the image to be corrected and preprocessing of the image to be corrected to obtain the image with shadows removed include:
[0013] Acquire an image to be corrected, and perform RGB channel separation on the image to be corrected to obtain an R channel image, a G channel image, and a B channel image;
[0014] Performing dilation processing, median filtering processing, and absolute difference processing on the R channel image, the G channel image, and the B channel image respectively;
[0015] The processed R channel image, G channel image and B channel image are merged to obtain the shadow-removed image.
[0016] In some embodiments, the calculation method of the variance of the shadow-removed image on the horizontal projection is:
[0017] Obtaining a horizontal projection value of each row of the shadow-removed image, wherein the shadow-removed image is an image with m rows and n columns;
[0018] Calculate the average of the horizontal projection values of each row;
[0019] Based on the average value and the horizontal projection value of each row, the variance of the shadow-removed image in the horizontal projection is calculated.
[0020] In some embodiments, the calculation formula of the variance of the shadow-removed image in horizontal projection is:
[0021]
[0022] Wherein, Var is the variance of the shadow-removed image in horizontal projection, xi is the horizontal projection value of the i-th row of the shadow-removed image, μ is the average value of the horizontal projection values of each row, and k is the k-th row of the shadow-removed image.
[0023] In some embodiments, the preset range is -20° to 20°.
[0024] In some embodiments, rotating the image to be corrected or the image with shadow removed based on the comparison result to complete the tilt correction of the image to be corrected includes:
[0025] Based on the comparison result, obtaining the rotation angle of the shadow-removed image when the variance is maximum;
[0026] Based on the fact that the maximum variance is the rotation angle of the shadow-removed image, the image to be corrected or the shadow-removed image is rotated to complete the tilt correction of the image to be corrected.
[0027] In some embodiments, the image tilt correction method further includes:
[0028] When the tilt correction of the image to be corrected is achieved by rotating the shadow-removed image, image enhancement processing is performed on the image after the tilt correction.
[0029] In a second aspect, the present invention further provides an image tilt correction device, comprising:
[0030] A preprocessing module is used to obtain an image to be corrected and preprocess the image to be corrected to obtain an image with shadows removed;
[0031] A variance calculation module, used to rotate the shadow-removed image several times within a preset range, and calculate the variance of the shadow-removed image in horizontal projection after each rotation;
[0032] A comparison module, used for comparing the calculated variances to obtain a comparison result;
[0033] The correction module is used to rotate the image to be corrected or the image with shadow removed based on the comparison result to complete the tilt correction of the image to be corrected.
[0034] In a third aspect, the present invention further provides an electronic device, comprising: a processor and a memory;
[0035] The memory stores a computer program executable by the processor;
[0036] When the processor executes the computer program, the steps in the image tilt correction method described above are implemented.
[0037] In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the image tilt correction method as described above.
[0038] Compared with the prior art, the image tilt correction method, device, electronic device and storage medium provided by the present invention first obtain the image to be corrected, pre-process the image to be corrected to obtain a shadow-removed image; then, within a preset range, the shadow-removed image is rotated several times, and the variance of the shadow-removed image on the horizontal projection after each rotation is calculated; then, the calculated variances are compared to obtain a comparison result; finally, based on the comparison result, the image to be corrected or the shadow-removed image is rotated to complete the tilt correction of the image to be corrected. The present invention can quickly and accurately rotate a text image with a small angle to the horizontal by pre-processing the image and calculating the variance on the horizontal projection, laying a solid and accurate foundation for subsequent text image detection and recognition and information extraction. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 is a flow chart of an image tilt correction method provided by an embodiment of the present invention;
[0040] Figure 2 is a flowchart of step S100 in the image tilt correction method provided by an embodiment of the present invention;
[0041] Figure 3 is a flow chart of variance calculation in the image tilt correction method provided by an embodiment of the present invention;
[0042] Figure 4 is a flowchart of step S400 in the image tilt correction method provided by an embodiment of the present invention;
[0043] Figure 5 is a schematic diagram of functional modules of an image tilt correction device provided by an embodiment of the present invention;
[0044] Figure 6 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0045] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0046] Extracting specific information from complex documents is particularly important in many business activities, such as information extraction from damage assessment photos in the financial field. This is a critical and complex process due to the wide variety of business documents and templates, such as contracts, invoices, reports, or news articles. The tilt of the image will seriously affect the recognition effect of automatic recognition models such as OCR. At present, there are many deep learning-based algorithms that can achieve large-angle text image tilt correction (such as 90°, 180°), but they are powerless for small-angle tilt (within 15°).
[0047] Therefore, the present invention provides an image tilt correction method, which first obtains an image to be corrected and preprocesses the image to obtain a shadow-removed image; then, the shadow-removed image is rotated within a preset range and the variance of the image on the horizontal projection after each rotation is calculated; then, the calculated variances are compared to determine the most appropriate rotation angle; finally, the image is rotated according to the comparison result to complete the tilt correction.
[0048] The preprocessing of the image to be corrected includes RGB channel separation, dilation operation processing, median filtering processing and absolute difference processing, so as to effectively highlight the features of the image, such as edges and corners, while eliminating noise and protecting the edges from being blurred.
[0049] When calculating the variance, the present invention rotates the shadow-removing image multiple times within a preset range, calculates the variance of the image on the horizontal projection after each rotation, and uses the variance of the projection to describe the difference between valleys and peaks, thereby accurately achieving small-angle tilt correction.
[0050] The variances are then compared to determine the image rotation angle that is closest to the horizontal direction, and the comparison results are used to guide the final image rotation correction operation.
[0051] Finally, the actual image rotation operation is performed based on the comparison result to complete the tilt correction of the image. The tilt correction also includes the step of enhancing the image after the tilt correction, which helps to improve the efficiency and accuracy of subsequent OCR recognition.
[0052] The present invention can quickly and accurately rotate text images with small angles to a horizontal state by preprocessing the image and using variance calculation on the horizontal projection, laying a solid foundation for text detection and recognition and information extraction, and also significantly improving the accuracy and efficiency of specific information extraction in related business activities.
[0053] See also Figure 1 , Figure 1 The present invention is a flowchart of an image tilt correction method provided by an embodiment of the present invention. The image tilt correction method involved in the present invention can be used in image tilt correction in the financial field to quickly and accurately correct the tilt of small-angle tilted images. The image tilt correction method involved in the present invention can be executed by an electronic device that can perform operations such as receiving or sending data, and can be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, portable computers, desktop servers, etc. Figure 1 As shown, the method specifically includes the following steps S100 to S400.
[0054] S100: Acquire an image to be corrected, and preprocess the image to be corrected to obtain an image with shadows removed.
[0055] In this embodiment, when image tilt correction is implemented, if the image is taken by a mobile phone, there may be a hand shadow, which may cause the image to be incorrectly corrected. Therefore, the image without removing the shadow is likely to be incorrectly rotated after tilt correction, because the shadow area can easily be regarded as a large paragraph of text and change the horizontal projection variance of the original text image. Therefore, before tilt correction, the embodiment of the present invention can prevent the image from being incorrectly corrected by preprocessing the image to be corrected.
[0056] S200: rotating the shadow-removed image several times within a preset range, and calculating the variance of the shadow-removed image in horizontal projection after each rotation.
[0057] In this embodiment, the horizontal direction is determined by the edge projection method, that is, the projection of the image in the horizontal direction is determined. Specifically, taking the license plate as an example, due to the existence of the horizontal line at the rear of the car, when the body is not tilted, the projection value of the license plate will be very prominent; but when the body is tilted, the horizontal line at the rear of the car will be distributed on multiple lines, thereby making the projection difference worse. That is, the projection peaks and valleys of the standard license plate are distinct, while the troughs of the tilted license plate are blurred. Therefore, it can be determined that the license plate is most horizontal when the peaks and valleys are the most distinct. For text images, the horizontal projection of the gaps between rows has similar characteristics to the projection of the standard license plate, that is, the horizontal text image projection has distinct peaks and valleys, while the troughs of the tilted text image are blurred. Unlike the standard license plate image, which has only one peak and valley, the horizontal text image projection often has multiple significant peaks and valleys because there are gaps between each segment. A careful observation shows that the horizontal projection of the corrected image is sparser, the peaks and valleys are wider, and the number is greater.
[0058] Therefore, the embodiment of the present invention uses the variance of the projection to describe the difference between valleys and peaks, thereby achieving small-angle tilt correction.
[0059] S300: Compare the calculated variances to obtain a comparison result.
[0060] In this embodiment, the larger the variance, the closer the rotated image is to the horizontal direction. Therefore, the embodiment of the present invention can obtain the rotation angle of the image closest to the horizontal direction by comparing the variances, and then perform tilt correction according to the angle. The comparison result can be a sorting result of the variance, which is convenient for directly obtaining the largest variance.
[0061] S400: Based on the comparison result, rotate the image to be corrected or the image with shadow removed to complete the tilt correction of the image to be corrected.
[0062] In this embodiment, the image to be corrected can be directly rotated, or the image with shadows removed can be rotated. For example, after finding the angle, the original image is rotated and used for subsequent OCR and other processing, and the image with shadows removed is discarded. Alternatively, the image with shadows removed is rotated and, after a series of processing, the user subsequently performs OCR recognition.
[0063] In the embodiment of the present invention, the image to be corrected is first obtained, and the image to be corrected is preprocessed to obtain a shadow-removed image; then, the shadow-removed image is rotated several times within a preset range, and the variance of the shadow-removed image on the horizontal projection after each rotation is calculated; then, the calculated variances are compared to obtain a comparison result; finally, based on the comparison result, the image to be corrected or the shadow-removed image is rotated to complete the tilt correction of the image to be corrected. The present invention can quickly and accurately rotate a text image with a small angle to a horizontal position by preprocessing the image and calculating the variance on the horizontal projection, laying a solid and accurate foundation for subsequent text image detection and recognition and information extraction.
[0064] In some embodiments, see Figure 2 , the step S100 specifically includes:
[0065] S110, acquiring an image to be corrected, and performing RGB channel separation on the image to be corrected to obtain an R channel image, a G channel image, and a B channel image;
[0066] S120, performing dilation operation processing, median filtering processing, and absolute difference processing on the R channel image, the G channel image, and the B channel image in sequence;
[0067] S130 , merging the processed R channel image, G channel image and B channel image to obtain the shadow-removed image.
[0068] In this embodiment, the R, G, and B channels of the image are first separated, and then dilation operations, median filtering, and absolute difference operations are performed on them respectively, and then the RGB channels of the image are merged to generate a new image.
[0069] Furthermore, RGB channel separation refers to the process of processing the red, green and blue color channels of a color image separately. In digital image processing, RGB channel separation is widely used, and this technology plays an important role from simple image editing to complex image analysis and pattern recognition. The implementation method of RGB channel separation is relatively simple, and mainly relies on the relevant functions provided by the image processing library. Taking OpenCV as an example, the RGB channels of an image can be easily split through the cv2.sp l it function. This function accepts a color image as input and returns an array containing three single-channel images. Each single-channel image corresponds to the red, green and blue channels of the original image. It can be understood that when reading an image using OpenCV, the order of the channels is actually BGR rather than RGB.
[0070] In step S100, image preprocessing is a key step, the purpose of which is to remove shadows in the image, thereby improving the accuracy of subsequent tilt correction, which may also include:
[0071] S101. Acquire an image to be corrected through an image acquisition device. The image to be corrected may be a digital photo, a scanned document, or a digital image in other forms.
[0072] S102: Convert the acquired image into an RGB color model, and separate three color channels of red (R), green (G), and blue (B).
[0073] S103, performing a dilation operation on each separated channel image, which can use a structural element to convolve with the image to expand the bright area in the image, thereby highlighting the text and edge features in the image.
[0074] S104, performing median filtering on the expanded image to eliminate noise in the image. Median filtering is a nonlinear filtering technology that achieves the effect of smoothing the image by replacing the value of each pixel point with the median value in its neighborhood.
[0075] S105 , performing absolute difference processing on the median filtered image to enhance the changed area in the image, and highlighting the changes in the image by calculating the difference between adjacent pixel points.
[0076] S106, re-merging the processed R, G, and B channel images to obtain an image with the shadow removed, which can be specifically achieved through image processing software to ensure that the color information of the image is preserved.
[0077] The dilation operation is a morphological operation, which is mainly used to expand the highlight area or white part in the image. This operation is achieved by using a specific structural element (also called a kernel) to perform a convolution operation with the original image. In the field of digital image processing, the dilation operation is widely used, from image preprocessing to complex image analysis and pattern recognition, this technology plays an important role. In this embodiment, the principle of the dilation operation is to scan each pixel of the image with a structural element in an image with a black background (0) and a white foreground (1). If at least one pixel value in the area covered by the structural element is 1, the value of the pixel is set to 1. In this way, the highlight area in the image will be enlarged, achieving the effect of dilation. When performing the dilation operation, a template is used. The shape of the template can be different forms such as rectangle, ellipse, cross, etc. These structural elements can be generated by the cv2.getStructur i ngElement() function in OpenCV. The main parameters of the dilation operation include the shape and size of the kernel, the position of the anchor point, and the number of iterations. The size of the kernel determines the degree of dilation, and the number of iterations affects the number of repetitions of the operation, thereby enhancing the dilation effect. In this embodiment, the dilation operation can highlight the features of the image, such as edges and corners. By judiciously choosing the size and shape of the structural elements, these features can be enhanced without altering other regions.
[0078] Median filtering is a nonlinear signal processing technique that processes each pixel by setting its value to the median of the pixel values in its neighborhood. This method is very effective in eliminating isolated noise points in an image and protects the image edges from being blurred during the process. By selecting the values of a pixel and its surrounding pixels in a digital image, sorting these values and taking the value in the middle as the new value of the current pixel. For example, in a 3×3 neighborhood, after sorting the pixel values in the neighborhood, the value in the middle will replace the original pixel value, thereby achieving a smoothing effect.
[0079] Absolute difference processing is a technique for analyzing image changes or detecting moving targets by calculating the absolute difference between corresponding pixels of two images. Image absolute difference is to subtract the pixel values of corresponding positions of the two images and take the absolute value to obtain the change amplitude of each pixel. Specifically, assuming that there are two images I1 and I2 of the same scene at different time points, applying absolute difference to these two images can obtain a new image D, in which each pixel value represents the change amount of the pixel value at the corresponding position in the original image. When operating, first, you need to load the two images to be compared and make sure they are the same size. Then, use the cv2.absd iff() function in OpenCV to calculate the difference between the two images. If the images are of different sizes, you need to use cv2.res i ze() to resize them first. Image absolute difference can simply and quickly detect moving targets in image sequences, with relatively small amount of calculation, suitable for real-time processing.
[0080] In some embodiments, see Figure 3 In step S200, the calculation method of the variance of the shadow-removed image on the horizontal projection is:
[0081] S210, obtaining a horizontal projection value of each row of the shadow-removed image, wherein the shadow-removed image is an image with m rows and n columns;
[0082] S220, calculating the average value of the horizontal projection value of each row;
[0083] S230: Calculate the variance of the shadow-removed image in horizontal projection based on the average value and the horizontal projection value of each row.
[0084] In this embodiment, in order to describe the difference between peaks and valleys, the variance of the projection is used to judge the peaks and valleys. Specifically, the edge projection method described in the present invention is a technology widely used in image analysis and machine vision, and is mainly used to detect and correct the text direction in the image, perform feature extraction and image enhancement, etc. The edge projection method is particularly common when processing text images. For example, by utilizing the blank area characteristics between text lines, the direction of the text can be effectively detected by rotating the image and horizontally projecting the pixel values. When the text direction is horizontal, the number of projections with a value of 0 reaches the maximum, which helps to automatically correct the tilted document image. The edge projection method is also very effective in removing the interference of longer lines in the image. When performing edge detection, it is often necessary to delete large interference objects such as table lines and long straight lines that may affect the accumulation of pixel values. This can be achieved through connected component analysis, calculating the distance between the center of mass of each connected area and the boundary, and the area with too large a distance can be identified as a line and removed.
[0085] Therefore, for a grayscale digital image with m rows and n columns, its horizontal projection must be an m-row column matrix. The horizontal projection value of the i-th row is denoted as xi. The algorithm is as follows: rotate the text image within a certain range, and calculate the variance of the horizontal projection of the image after each rotation. The angle of each rotation can be set according to actual needs, such as 1°, 0.1 degree, etc.
[0086] In some embodiments, the calculation formula of the variance of the shadow-removed image in horizontal projection is:
[0087]
[0088] Wherein, Var is the variance of the shadow-removed image on the horizontal projection, xi is the horizontal projection value of the i-th row of the shadow-removed image, μ is the average value of the horizontal projection values of each row, and k is the k-th row of the shadow-removed image. Specifically,
[0089] In this embodiment, the variance is calculated and then compared to obtain the rotation angle of the text image to the horizontal.
[0090] In some embodiments, the preset range is -20° to 20°.
[0091] In this embodiment, the tilt correction can achieve rotation within the range of -20° to 20°. Preferably, in the actual landing scene, in order to reduce time consumption, a smaller range can be selected, such as selecting -15° to 15° in the contract recognition scene. Specifically, there are many algorithms in the prior art that can achieve large-angle text image tilt correction (such as the built-in function of Python's rapid_orientation library), but can only process integer multiples of 90°. Before small-angle tilt correction, large-angle tilt correction is usually used first. In theory, the tilt angle of the corrected image is between -45° and 45°. The present invention can also directly traverse within the range of -45° to 45°, but doing so is time-consuming. Therefore, in practical applications, if the quality of the input data in production is not particularly poor, the present invention preferably selects -15° to 15° to -20° to 20°.
[0092] After the image preprocessing is completed, the image rotation and variance calculation may be performed to determine the best rotation angle, so step S200 may also include:
[0093] S201, setting a number of rotation angles within a preset range (such as -20° to 20°), wherein these angles can be set at equal intervals or at unequal intervals as required.
[0094] S202, rotating the image with the shadow removed at each angle, and calculating the horizontal projection value of the image after each rotation.
[0095] S203. For each rotated image, calculate the horizontal projection value of each row thereof, wherein the horizontal projection value refers to the accumulation of pixel values of the image in the horizontal direction.
[0096] S204, calculating the variance of the horizontal projection value of the image after each rotation. The variance is an indicator to measure the degree of dispersion of the horizontal projection value distribution of the image. The larger the variance, the closer the image is to the horizontal at the rotation angle.
[0097] S205 , comparing the calculated variances to obtain a rotation angle with the largest variance, which is the optimal rotation angle of the image.
[0098] After determining the optimal rotation angle, image correction is performed to complete the tilt correction, so step S300 may also include:
[0099] S301, rotating the image with the shadow removed according to an optimal rotation angle, wherein the rotation operation can be implemented by image processing software.
[0100] S302: Perform enhancement processing on the rotation-corrected image. The rotation-corrected image may require further enhancement processing to improve image quality. The enhancement processing may include contrast enhancement, sharpening and other operations.
[0101] When enhancing the contrast, the contrast of the image can be adjusted to make the text and background in the image clearer. Contrast enhancement can be achieved through methods such as histogram equalization.
[0102] When sharpening an image, the image can be sharpened by using methods such as a high-pass filter to enhance the edge features of the image.
[0103] In some embodiments, see Figure 4 , the step S400 specifically includes:
[0104] S410, based on the comparison result, obtaining the rotation angle of the shadow-removed image when the variance is maximum;
[0105] S420: Based on the maximum variance being the rotation angle of the shadow-removed image, rotate the image to be corrected or the shadow-removed image to complete the tilt correction of the image to be corrected.
[0106] In this embodiment, after all the variances are obtained, a comparison result can be obtained by comparing the variances. The comparison result reflects the size relationship of the variances. Then, based on the comparison result, the rotation angle of the shadow removal image when the variance is maximum can be obtained. This angle is the rotation angle of the text image to the horizontal. Therefore, in order to achieve tilt correction, it is only necessary to rotate the image by this angle.
[0107] In order to further improve the accuracy and efficiency of image tilt correction, the tilt correction also includes:
[0108] The adaptive rotation angle setting is performed according to the image to be corrected or the image to remove the shadow. Specifically, the setting range and interval of the rotation angle can be adaptively adjusted according to the complexity of the image content to improve the accuracy of the correction.
[0109] After setting the rotation angle, multi-scale analysis is performed, specifically, the image can be analyzed at different scales to identify the main features and structures in the image, so as to more accurately determine the rotation angle.
[0110] Afterwards, for auxiliary correction, it can use machine learning algorithms to train the model to recognize the tilt features of the image and assist in determining the optimal rotation angle.
[0111] Afterwards, the correction parameters are adjusted in real time to achieve real-time feedback of the correction effect during the image correction process, and the correction parameters are dynamically adjusted according to the feedback results to improve the accuracy of the correction.
[0112] Finally, the image quality after tilt correction is quantitatively evaluated to evaluate the image quality, such as clarity, contrast, etc., after image correction, and further optimize the image based on the evaluation results.
[0113] In some embodiments, the image tilt correction method further includes:
[0114] When the tilt correction of the image to be corrected is achieved by rotating the shadow-removed image, image enhancement processing is performed on the image after the tilt correction.
[0115] In this embodiment, when the shadow removal image is used for rotation, the shadow removal image filters out some dark pixels, causing the field color to become lighter and difficult to detect and identify. Therefore, the present invention uses some image enhancement methods, such as dilation processing, to enhance the text contrast to facilitate subsequent OCR recognition.
[0116] The technical solution provided by the present invention first obtains the image to be corrected, pre-processes the image to be corrected to obtain a shadow-removed image; then, within a preset range, rotates the shadow-removed image several times, and calculates the variance of the shadow-removed image on the horizontal projection after each rotation; then compares the calculated variances to obtain a comparison result; finally, based on the comparison result, rotates the image to be corrected or the shadow-removed image to complete the tilt correction of the image to be corrected. The present invention can quickly and accurately rotate a text image with a small angle to the horizontal by pre-processing the image and calculating the variance on the horizontal projection, laying a solid and accurate foundation for subsequent text image detection and recognition and information extraction.
[0117] The image tilt correction method of the present invention can be used in various fields requiring image processing, such as:
[0118] Document scanning: During the document scanning process, the document tilt is automatically corrected to improve the quality of the scanned image;
[0119] Photo editing: In photo editing software, the function of automatic tilt correction is provided to facilitate users to edit and adjust the photos they have taken;
[0120] Medical image processing: In medical image analysis, automatically correct the image tilt to improve the accuracy of diagnosis;
[0121] Security monitoring: In the security monitoring system, the tilt of the monitoring image is automatically corrected to improve the clarity and recognition rate of monitoring;
[0122] Industrial inspection: In industrial automated inspection, the tilt of product images is automatically corrected to improve the accuracy and efficiency of inspection.
[0123] Application of deep learning technology: Use deep learning algorithms to improve the intelligence and automation level of image tilt correction.
[0124] Multimodal image processing: Combine image data from different sources and types for comprehensive analysis and processing to improve the accuracy and robustness of correction.
[0125] Real-time image processing: Improve the speed of image processing, realize real-time or near real-time image tilt correction, and meet the needs of rapid response.
[0126] Cross-platform compatibility: Optimize algorithms and software so that they can run on different operating systems and hardware platforms, increasing the breadth of applications.
[0127] User interaction optimization: Improve the user interface and interaction methods, provide a more intuitive and convenient operating experience, and lower the usage threshold.
[0128] Another embodiment of the present invention provides an image tilt correction device, see Figure 5 The image tilt correction device includes a preprocessing module 11, a variance calculation module 12, a comparison module 13 and a correction module 14.
[0129] The preprocessing module 11 is used to obtain the image to be corrected and preprocess the image to be corrected to obtain a shadow-removed image.
[0130] The variance calculation module 12 is used to rotate the shadow-removed image several times within a preset range, and calculate the variance of the shadow-removed image in horizontal projection after each rotation.
[0131] The comparison module 13 is used to compare the calculated variances to obtain a comparison result.
[0132] The correction module 14 is used to rotate the image to be corrected or the image with shadow removed based on the comparison result, so as to complete the tilt correction of the image to be corrected.
[0133] In this embodiment, firstly, the image to be corrected is obtained, and the image to be corrected is preprocessed to obtain a shadow-removed image; then, within a preset range, the shadow-removed image is rotated several times, and the variance of the shadow-removed image on the horizontal projection after each rotation is calculated; then, the calculated variances are compared to obtain a comparison result; finally, based on the comparison result, the image to be corrected or the shadow-removed image is rotated to complete the tilt correction of the image to be corrected. The present invention can quickly and accurately rotate a text image with a small angle to the horizontal by preprocessing the image and calculating the variance on the horizontal projection, laying a solid and accurate foundation for subsequent text image detection and recognition and information extraction.
[0134] It should be noted that the module referred to in the present invention refers to a series of computer program instruction segments that can perform specific functions, which is more suitable for describing the execution process of image tilt correction than a program. For the specific implementation of each module, please refer to the corresponding method embodiment above, which will not be repeated here.
[0135] In some embodiments, the preprocessing module 11 is specifically used for:
[0136] Acquire an image to be corrected, and perform RGB channel separation on the image to be corrected to obtain an R channel image, a G channel image, and a B channel image;
[0137] Performing dilation processing, median filtering processing, and absolute difference processing on the R channel image, the G channel image, and the B channel image respectively;
[0138] The processed R channel image, G channel image and B channel image are merged to obtain the shadow-removed image.
[0139] In some embodiments, the calculation method of the variance of the shadow-removed image on the horizontal projection is:
[0140] Obtaining a horizontal projection value of each row of the shadow-removed image, wherein the shadow-removed image is an image with m rows and n columns;
[0141] Calculate the average of the horizontal projection values of each row;
[0142] Based on the average value and the horizontal projection value of each row, the variance of the shadow-removed image in the horizontal projection is calculated.
[0143] In some embodiments, the calculation formula of the variance of the shadow-removed image in horizontal projection is:
[0144]
[0145] Wherein, Var is the variance of the shadow-removed image in horizontal projection, xi is the horizontal projection value of the i-th row of the shadow-removed image, μ is the average value of the horizontal projection values of each row, and k is the k-th row of the shadow-removed image.
[0146] In some embodiments, the preset range is -20° to 20°.
[0147] In some embodiments, the correction module 14 is specifically used to:
[0148] Based on the comparison result, obtaining the rotation angle of the shadow-removed image when the variance is maximum;
[0149] Based on the fact that the maximum variance is the rotation angle of the shadow-removed image, the image to be corrected or the shadow-removed image is rotated to complete the tilt correction of the image to be corrected.
[0150] In some embodiments, the image tilt correction device further includes:
[0151] When the tilt correction of the image to be corrected is achieved by rotating the shadow-removed image, image enhancement processing is performed on the image after the tilt correction.
[0152] Another embodiment of the present invention provides an electronic device, such as Figure 6 As shown, the electronic device 10 includes:
[0153] One or more processors 110 and memory 120, Figure 6A processor 110 is used as an example for description. The processor 110 and the memory 120 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.
[0154] The processor 110 is used to complete various control logics of the electronic device 10, and it can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISC Machine) or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of these components. In addition, the processor 110 can also be any traditional processor, microprocessor or state machine. The processor 110 can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP and / or any other such configuration.
[0155] The memory 120 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions corresponding to the image tilt correction method in the embodiment of the present invention. The processor 110 executes various functional applications and data processing of the electronic device 10 by running the non-volatile software programs, instructions and units stored in the memory 120, that is, the image tilt correction method in the above method embodiment is implemented.
[0156] The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store an operating platform, an application required for at least one function; the data storage area may store data created according to the use of the electronic device 10, etc. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 120 may optionally include a memory remotely arranged relative to the processor 110, and these remote memories may be connected to the electronic device 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0157] One or more units are stored in the memory 120, and when executed by one or more processors 110, perform the image tilt correction method in any of the above method embodiments, for example, perform the above described Figure 1 Method steps in:
[0158] Acquire an image to be corrected, and preprocess the image to be corrected to obtain a shadow-removed image;
[0159] Rotating the shadow-removed image several times within a preset range, and calculating the variance of the shadow-removed image in horizontal projection after each rotation;
[0160] Comparing the calculated several variances to obtain a comparison result;
[0161] Based on the comparison result, the image to be corrected or the image with shadow removed is rotated to complete the tilt correction of the image to be corrected.
[0162] In some embodiments, the processor 110 further performs the following steps:
[0163] Acquire an image to be corrected, and perform RGB channel separation on the image to be corrected to obtain an R channel image, a G channel image, and a B channel image;
[0164] Performing dilation processing, median filtering processing, and absolute difference processing on the R channel image, the G channel image, and the B channel image respectively;
[0165] The processed R channel image, G channel image and B channel image are merged to obtain the shadow-removed image.
[0166] In some embodiments, the processor 110 further performs the following steps:
[0167] Obtaining a horizontal projection value of each row of the shadow-removed image, wherein the shadow-removed image is an image with m rows and n columns;
[0168] Calculate the average of the horizontal projection values of each row;
[0169] Based on the average value and the horizontal projection value of each row, the variance of the shadow-removed image in the horizontal projection is calculated.
[0170] In some embodiments, the calculation formula of the variance of the shadow-removed image in horizontal projection is:
[0171]
[0172] Wherein, Var is the variance of the shadow-removed image in horizontal projection, xi is the horizontal projection value of the i-th row of the shadow-removed image, μ is the average value of the horizontal projection values of each row, and k is the k-th row of the shadow-removed image.
[0173] In some embodiments, the preset range is -20° to 20°.
[0174] In some embodiments, the processor 110 further performs the following steps:
[0175] Based on the comparison result, obtaining the rotation angle of the shadow-removed image when the variance is maximum;
[0176] Based on the fact that the maximum variance is the rotation angle of the shadow-removed image, the image to be corrected or the shadow-removed image is rotated to complete the tilt correction of the image to be corrected.
[0177] In some embodiments, the processor 110 further performs the following steps:
[0178] When the tilt correction of the image to be corrected is achieved by rotating the shadow-removed image, the image after the tilt correction is subjected to image enhancement processing. Those skilled in the art will appreciate that Figure 6 The embodiments of the electronic device shown in the figure do not constitute a limitation on the specific composition of the electronic device. In other embodiments, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, in some embodiments, the electronic device may include a memory, a processor, and a network interface. In such embodiments, the structure and function of the memory and the processor are similar to those in the figure. Figure 6 The embodiments shown are consistent and will not be described again here.
[0179] In another embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the automatic question-answering method disclosed in the embodiment of the present invention is implemented.
[0180] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the above-described equipment, devices and units can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. Those of ordinary skill in the art can appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.
[0181] In the several embodiments provided by the present invention, it should be understood that the disclosed equipment, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. Units with the same function may also be combined into one unit. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or units, or may be an electrical, mechanical or other form of connection.
[0182] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.
[0183] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0184] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a background server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), disk or CD-ROM and other media that can store program codes.
[0185] The image tilt correction method, device, electronic device and storage medium provided by the present invention first obtain the image to be corrected, pre-process the image to be corrected to obtain a shadow-removed image; then rotate the shadow-removed image several times within a preset range, and calculate the variance of the shadow-removed image on the horizontal projection after each rotation; then compare the calculated variances to obtain a comparison result; finally, based on the comparison result, rotate the image to be corrected or the shadow-removed image to complete the tilt correction of the image to be corrected. The present invention can quickly and accurately rotate a text image with a small angle to the horizontal by pre-processing the image and calculating the variance on the horizontal projection, laying a solid and accurate foundation for subsequent text image detection and recognition and information extraction.
[0186] Compared with the prior art, the image tilt correction effect of the present invention is mainly reflected in the following aspects:
[0187] 1) Improved accuracy: Pre-processing removes shadows and reduces the impact of non-text areas in the image on tilt correction. The edge projection method and variance calculation are used to determine the optimal rotation angle, making the correction process more accurate, especially for images with small angle tilts, which can achieve more precise correction.
[0188] 2) High degree of automation: The present invention reduces manual intervention and improves processing efficiency through automated image processing procedures. The preset rotation range and automatically calculated variance comparison results make the entire correction process free of manual judgment and suitable for batch processing of images.
[0189] 3) Wide range of applications: The present invention is not only suitable for document scanning and photo editing, but can also be used in many fields such as medical imaging, security monitoring, industrial detection, etc. Its versatility and flexibility enable it to achieve good correction effects in different scenarios.
[0190] 4) Image quality optimization: After completing the tilt correction, the present invention also provides an image enhancement processing step to further improve the clarity and contrast of the image, providing higher quality image data for subsequent image recognition and information extraction.
[0191] 5) Real-time performance: By optimizing the algorithm and improving the computational efficiency, the present invention can achieve real-time or near real-time image correction, which is particularly important for application scenarios that require fast response.
[0192] 6) User-friendliness: With the support of the graphical user interface (GUI), users can operate and set parameters more intuitively, which improves the user experience. At the same time, the automated processing flow reduces the difficulty of operation, making it easy for non-professionals to use.
[0193] 7) Stability and reliability: Through multiple rotations and variance calculations, the present invention can stably identify the optimal correction angle, reduce the risk of correction failure due to a single measurement error, and improve overall reliability.
[0194] 8) Scalability: The modular design of the present invention allows for the further integration of more advanced image processing technologies, such as deep learning, on the existing basis to achieve higher levels of image analysis and processing.
[0195] 9) Economic benefits: Automated and accurate image correction can reduce labor costs and improve production efficiency, especially in large-scale image processing tasks, with significant economic benefits.
[0196] 10) Environmental adaptability: Since the image preprocessing and correction of the present invention do not rely on specific lighting conditions or backgrounds, it can maintain a stable correction effect in different environments, thereby enhancing its adaptability in changing environments.
[0197] Therefore, the present invention shows remarkable effects in terms of accuracy, degree of automation, scope of application, image quality, real-time performance, user-friendliness, stability, scalability, economic benefits and environmental adaptability, and provides an efficient and reliable solution for the field of image processing.
[0198] The specific implementation of the present invention described above does not constitute a limitation on the protection scope of the present invention. Any other corresponding changes and modifications made based on the technical concept of the present invention should be included in the protection scope of the claims of the present invention.
Claims
1. A method for image tilt correction, characterized in that: The steps include: Acquire an image to be corrected, and preprocess the image to be corrected to obtain a shadow-removed image; Rotating the shadow-removed image several times within a preset range, and calculating the variance of the shadow-removed image in horizontal projection after each rotation; Comparing the calculated several variances to obtain a comparison result; Based on the comparison result, the image to be corrected or the image with shadow removed is rotated to complete the tilt correction of the image to be corrected.
2. The image tilt correction method according to claim 1, characterized in that: The step of obtaining the image to be corrected and preprocessing the image to be corrected to obtain the image with shadows removed includes: Acquire an image to be corrected, and perform RGB channel separation on the image to be corrected to obtain an R channel image, a G channel image, and a B channel image; Performing dilation processing, median filtering processing, and absolute difference processing on the R channel image, the G channel image, and the B channel image respectively; The processed R channel image, G channel image and B channel image are merged to obtain the shadow-removed image.
3. The image tilt correction method according to claim 1, characterized in that: The calculation method of the variance of the shadow-removed image on the horizontal projection is: Obtaining a horizontal projection value of each row of the shadow-removed image, wherein the shadow-removed image is an image with m rows and n columns; Calculate the average of the horizontal projection values of each row; Based on the average value and the horizontal projection value of each row, the variance of the shadow-removed image in the horizontal projection is calculated.
4. The image tilt correction method according to claim 3, characterized in that: The calculation formula of the variance of the shadow-removed image on the horizontal projection is: Wherein, Var is the variance of the shadow-removed image in horizontal projection, xi is the horizontal projection value of the i-th row of the shadow-removed image, μ is the average value of the horizontal projection values of each row, and k is the k-th row of the shadow-removed image.
5. The image tilt correction method according to claim 1, characterized in that: The preset range is -20° to 20°.
6. The image tilt correction method according to claim 1, characterized in that: The step of rotating the image to be corrected or the image to be shadow-removed based on the comparison result to complete the tilt correction of the image to be corrected comprises: Based on the comparison result, obtaining the rotation angle of the shadow-removed image when the variance is maximum; Based on the fact that the maximum variance is the rotation angle of the shadow-removed image, the image to be corrected or the shadow-removed image is rotated to complete the tilt correction of the image to be corrected.
7. The image tilt correction method according to claim 1, characterized in that: Also includes: When the tilt correction of the image to be corrected is achieved by rotating the shadow-removed image, image enhancement processing is performed on the image after the tilt correction.
8. An image tilt correction device, characterized in that: include: A preprocessing module is used to obtain an image to be corrected and preprocess the image to be corrected to obtain an image with shadows removed; A variance calculation module, used to rotate the shadow-removed image several times within a preset range, and calculate the variance of the shadow-removed image in horizontal projection after each rotation; A comparison module, used for comparing the calculated variances to obtain a comparison result; The correction module is used to rotate the image to be corrected or the image with shadow removed based on the comparison result to complete the tilt correction of the image to be corrected.
9. An electronic device, characterized in that: include: Processor and memory; The memory stores a computer program executable by the processor; When the processor executes the computer program, the steps of the image tilt correction method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium, characterized in that: include: Processor and memory; The memory stores a computer program executable by the processor; When the processor executes the computer program, the steps of the image tilt correction method according to any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
Method for correcting license plate slant based on rotating projection
CN105488501A
Tilt correction method for scanning document image
CN106447615A
Image-inclination correction method and device, storage medium and computer equipment
CN108090875A
Image shadow removing method and system
CN111652806A
Counting analysis method for table signatures
CN116453130A
