Image correction method, device, storage medium and electronic device
Image correction through the target high-resolution network, the difficulty of recognition caused by distortion of the handheld camera's photo image is solved, and the accuracy and efficiency of image recognition are improved, especially in image text recognition, providing technical support for automated office work.
Patent Information
- Application Number
- CN202210613088.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-31
- Publication Date
- 2025-08-01
- Estimated Expiration
- 2042-05-31
AI Technical Summary
Images taken by handheld cameras are easily deformed due to factors such as uneven light, perspective changes, paper distortion, folding and wrinkling, resulting in increased recognition difficulty.
Image correction is performed using a target high-resolution network, and distortion correction is performed by acquiring the target optical flow and front background segmentation map of the image to be corrected, including distortion correction and interpolation processing of the foreground area.
Automatic correction of distorted images is realized, the difficulty of subsequent image recognition is reduced, the accuracy of image recognition and information processing efficiency is improved, especially in terms of image text recognition capabilities, providing support for automated office work.
Smart Images
Figure CN114972098B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image processing technologies, and in particular, to an image correction method, apparatus, storage medium, and electronic device. Background Art
[0002] With the popularization of mobile devices in recent years, photographed images (e.g., document images) taken by handheld cameras and the like have become increasingly common. Different from the images captured by flatbed scanners, the photographed images based on handheld cameras are more likely to be distorted due to various factors such as uneven illumination, perspective changes, paper distortion, folding, and wrinkling, making it difficult to achieve an ideal recognition effect for the photographed images. Summary of the Invention
[0003] To overcome the problems existing in the related art, the present disclosure provides an image correction method, apparatus, storage medium, and electronic device.
[0004] To achieve the above object, in a first aspect, the present disclosure provides an image correction method, including:
[0005] Obtain an image to be corrected;
[0006] Input the image to be corrected into a target high-resolution network to obtain a target optical flow and a foreground-background segmentation map corresponding to the image to be corrected;
[0007] Perform distortion correction on the image to be corrected according to the target optical flow and the foreground-background segmentation map to obtain a target image.
[0008] Optionally, the performing distortion correction on the image to be corrected according to the target optical flow and the foreground-background segmentation map to obtain a target image includes:
[0009] Extract a foreground region from the image to be corrected according to the foreground-background segmentation map, and determine a foreground optical flow corresponding to the foreground region from the target optical flow;
[0010] Perform distortion correction on the foreground region according to the foreground optical flow to obtain a target image.
[0011] Optionally, the performing distortion correction on the foreground region according to the foreground optical flow to obtain a target image includes:
[0012] Perform distortion correction on the foreground region according to the foreground optical flow;
[0013] Perform interpolation processing on the foreground region obtained after distortion correction to obtain a target image.
[0014] Optionally, the target high-resolution network is trained in the following manner:
[0015] Obtain a sample image, where the sample image is an undistorted image;
[0016] Perform distortion processing on the sample image to obtain a distorted image;
[0017] Determine the actual optical flow between the sample image and the distorted image;
[0018] Input the distorted image into the current high-resolution network to obtain the predicted optical flow corresponding to the distorted image;
[0019] Update the model parameters of the current high-resolution network according to the actual optical flow and the predicted optical flow;
[0020] In response to meeting the training cut-off condition, determine the current high-resolution network as the target high-resolution network;
[0021] In response to not meeting the training cut-off condition, re-execute the steps from obtaining the sample image to updating the model parameters of the current high-resolution network according to the actual optical flow and the predicted optical flow.
[0022] Optionally, the updating the model parameters of the current high-resolution network according to the actual optical flow and the predicted optical flow includes:
[0023] Determine the smooth L1 norm loss of the current high-resolution network according to the actual optical flow and the predicted optical flow;
[0024] Update the model parameters of the current high-resolution network according to the smooth L1 norm loss.
[0025] Optionally, the distortion processing includes at least one of the following:
[0026] Wrinkle processing, tilt processing, and twist processing.
[0027] Optionally, the actual optical flow is dense optical flow;
[0028] The determining the actual optical flow between the sample image and the distorted image includes:
[0029] Calculate the actual optical flow between the sample image and the distorted image through a dense optical flow algorithm.
[0030] In a second aspect, the present disclosure provides an image correction device, including:
[0031] A first acquisition module, configured to acquire an image to be corrected;
[0032] A feature extraction module, configured to input the to-be-corrected image acquired by the first acquisition module into a target high-resolution network, to obtain a target optical flow and a foreground-background segmentation map corresponding to the to-be-corrected image;
[0033] A first correction module, configured to perform distortion correction on the to-be-corrected image acquired by the first acquisition module according to the target optical flow and the foreground-background segmentation map obtained by the feature extraction module, to obtain a target image.
[0034] In a third aspect, the present disclosure provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the image correction method provided in the first aspect of the present disclosure are implemented.
[0035] In a fourth aspect, the present disclosure provides an electronic device, including:
[0036] A memory, on which a computer program is stored;
[0037] A processor, configured to execute the computer program in the memory to implement the steps of the image correction method provided in the first aspect of the present disclosure.
[0038] In the above technical solution, the to-be-corrected image is input into the target high-resolution network to obtain a target optical flow and a foreground-background segmentation map corresponding to the to-be-corrected image; then, according to the target optical flow and the foreground-background segmentation map, distortion correction is performed on the to-be-corrected image to obtain a target image. In this way, automatic correction of distorted images can be achieved, the difficulty of subsequent image recognition can be reduced, the accuracy of image recognition and the efficiency of image information processing can be improved, and thus the image recognition capabilities in different scenarios such as real and network scenarios, especially the image text recognition capabilities, can be promoted, adding bricks and tiles to the innovation of automated office. In addition, the target high-resolution network adopts a repetitive multi-scale fusion method, parallelly connecting subnets from high resolution to low resolution, and using low-resolution representations with the same depth and similar levels to improve the high-resolution representation, making the high-resolution to low-resolution representations stronger semantically and the predicted heatmaps more accurate spatially, so as to accurately predict the target optical flow and the foreground-background segmentation map of the to-be-corrected image, and further improve the accuracy of subsequent distortion correction and the accuracy of subsequent image recognition.
[0039] Other features and advantages of the present disclosure will be described in detail in the subsequent specific implementation part. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] The drawings are used to provide a further understanding of the present disclosure, and constitute a part of the specification, and are used to explain the present disclosure together with the following specific implementation manners, but do not constitute a limitation to the present disclosure. In the drawings:
[0041] Figure 1It is a flowchart of an image correction method shown according to an exemplary embodiment.
[0042] Figure 2 It is a schematic structural diagram of a target high-resolution network shown according to an exemplary embodiment.
[0043] Figure 3 It is a flowchart of a training method of a target high-resolution network shown according to an exemplary embodiment.
[0044] Figure 4 It is a block diagram of an image correction device shown according to an exemplary embodiment.
[0045] Figure 5 It is a block diagram of an electronic device shown according to an exemplary embodiment.
[0046] Figure 6 It is a block diagram of an electronic device shown according to an exemplary embodiment. Detailed implementation manners
[0047] The following will describe the detailed implementation manners of the present disclosure with reference to the accompanying drawings. It should be understood that the detailed implementation manners described herein are only used to illustrate and explain the present disclosure, and are not used to limit the present disclosure.
[0048] It should be noted that all actions of obtaining signals, information or data in the present disclosure are carried out on the premise of complying with the corresponding data protection regulations and policies of the country where it is located, and with the authorization given by the owner of the corresponding device.
[0049] Figure 1 It is a flowchart of an image correction method shown according to an exemplary embodiment. As Figure 1 shown, the method may include the following S101 to S103.
[0050] In S101, an image to be corrected is obtained.
[0051] In the present disclosure, the image to be corrected may be a distorted image caused by factors such as uneven illumination, perspective change, paper distortion, folding, wrinkling, etc. Among them, the image to be corrected may be a pure text image, an image combining text and graphics, etc., and the present disclosure does not make specific limitations.
[0052] In S102, the image to be corrected is input into the target high-resolution network to obtain the target optical flow and the foreground-background segmentation map corresponding to the image to be corrected.
[0053] Specifically, the target high-resolution network may perform feature extraction on the image to be corrected to obtain the target optical flow and the foreground-background segmentation map corresponding to the image to be corrected, where the foreground-background segmentation map is a segmentation map of the foreground and background of the image to be corrected.
[0054] In the present disclosure, the above-mentioned target high-resolution network can be understood as: in the process of feature extraction of the image to be corrected, the feature map always maintains a high resolution, and a type of network model that gradually and parallelly adds a low-resolution feature map sub-network to the main network of the high-resolution feature map. That is, when using the target high-resolution network to process the image to be corrected, the resolution of the feature map corresponding to the image to be corrected will not be reduced. In other words, 'high resolution' does not clearly indicate that the resolution is higher than a specific value, but is used to indicate that the resolution of the feature map will not be reduced when processing the feature map.
[0055] For ease of understanding, it can be combined with Figure 2 for illustration. Figure 2 is a schematic structural diagram of a target high-resolution network shown according to an exemplary embodiment. The target high-resolution network may include multiple parallel branches. In the process of processing the image to be corrected by the high-resolution network, different-resolution branches will be obtained through downsampling technology. Figure 2 Each square solid line box in Figure 2 can be regarded as a feature map, and the arrow can represent the information interaction channel between the feature maps. And Figure 2 the arrow 204 in
[0056] can also represent that downsampling technology is performed to obtain a feature map with a lower resolution; similarly, the arrow 205 and the arrow 206 can also represent that downsampling technology is adopted, so that a feature map with a lower resolution can be obtained. And every time a new branch is added, it can be regarded as starting a new image processing stage. For example, Figure 2 the dashed box 201 in
[0057] In S103, according to the target optical flow and the foreground and background segmentation map, the image to be corrected is subjected to distortion correction to obtain a target image.
[0058] In the above technical solution, the image to be corrected is input into the target high-resolution network to obtain the target optical flow and the foreground and background segmentation map corresponding to the image to be corrected; then, according to the target optical flow and the foreground and background segmentation map, the image to be corrected is subjected to distortion correction to obtain the target image. In this way, automatic correction of distorted images can be achieved, the difficulty of subsequent image recognition can be reduced, the accuracy of image recognition and the efficiency of image information processing can be improved, and thus the image recognition ability in different scenarios such as the real world and the network, especially the image text recognition ability, can be promoted, adding bricks and tiles to the innovation of automated office. In addition, the target high-resolution network adopts a repetitive multi-scale fusion method, parallelly connecting subnets from high resolution to low resolution, and using low-resolution representations with the same depth and similar levels to improve the high-resolution representation, making the high-resolution to low-resolution representations stronger semantically and the predicted heatmaps more accurate spatially, so that the target optical flow and the foreground and background segmentation map of the image to be corrected can be accurately predicted, and further improving the accuracy of subsequent distortion correction and further enhancing the accuracy of subsequent image recognition.
[0059] The following details the specific implementation of performing distortion correction on the image to be corrected according to the target optical flow and the foreground and background segmentation map in S103 above to obtain the target image. Specifically, it can be achieved through the following steps (1) and (2).
[0060] (1) According to the foreground and background segmentation map, extract the foreground region from the image to be corrected, and determine the foreground optical flow corresponding to the foreground region from the target optical flow.
[0061] Specifically, the background region can be removed from the image to be corrected according to the foreground and background segmentation map to obtain the foreground region; then, the optical flow corresponding to the foreground region in the target optical flow is used as the foreground optical flow.
[0062] (2) According to the foreground optical flow, perform distortion correction on the foreground region to obtain the target image.
[0063] In one implementation, distortion correction can be performed on the foreground region according to the foreground optical flow; then, the foreground region obtained after distortion correction is determined as the target image.
[0064] Specifically, mapping processing can be performed on each pixel point in the foreground region according to the foreground optical flow, that is, mapping to a blank reference image without distortion to obtain the target image.
[0065] In another implementation, distortion correction can be performed on the foreground region according to the foreground optical flow; then, interpolation processing is performed on the foreground region obtained after distortion correction to obtain the target image.
[0066] Exemplarily, nearest neighbor interpolation processing can be performed on the foreground region obtained after distortion correction.
[0067] In this embodiment, since multiple pixel points in the foreground region may be mapped to the same position in the foreground region obtained after distortion correction, there will be a region in the foreground region obtained after distortion correction that is not mapped by the pixel points in the foreground region before mapping, that is, pixel loss. In this way, there will be small holes in the foreground region obtained after distortion correction. Therefore, the missing pixel points are filled by interpolation processing to improve the image quality and further improve the accuracy of subsequent image recognition.
[0068] The following will elaborate on the specific training method of the above-mentioned target high-resolution network. Specifically, it can be implemented through Figure 3 S301 - S307 shown in
[0069] In S301, a sample image is obtained.
[0070] In the present disclosure, the sample image is an undistorted image. In one embodiment, a printed document image can be used as the sample image.
[0071] In S302, the sample image is subjected to distortion processing to obtain a distorted image.
[0072] In the present disclosure, the distortion processing may include at least one of wrinkling processing, tilting processing, and twisting processing.
[0073] In one embodiment, any one of wrinkling processing, tilting processing, and twisting processing can be used to perform distortion processing on the sample image to obtain a distorted image.
[0074] Exemplarily, the sample image is subjected to twisting processing to obtain a distorted image.
[0075] In another embodiment, any two of wrinkling processing, tilting processing, and twisting processing can be used to perform distortion processing on the sample image to obtain a distorted image.
[0076] Exemplarily, the sample image is subjected to wrinkling processing and twisting processing to obtain a distorted image.
[0077] In yet another embodiment, wrinkling processing, tilting processing, and twisting processing can be used to perform distortion processing on the sample image to obtain a distorted image, that is, the sample image is subjected to wrinkling processing, tilting processing, and twisting processing to obtain a distorted image.
[0078] In S303, the actual optical flow between the sample image and the distorted image is determined.
[0079] In one embodiment, the actual optical flow is a dense optical flow, and accordingly, the target optical flow and the foreground optical flow are both dense optical flows. In this case, a dense optical flow algorithm can be used to calculate the actual optical flow between the sample image and the distorted image.
[0080] For example, the above-mentioned dense optical flow algorithm can be, for example, Farneback algorithm, DeepFlow algorithm, FlowNet algorithm, PWC-Net algorithm, LiteFlowNet algorithm, StruPyNet algorithm, etc., which is not specifically limited in this disclosure.
[0081] In the above S301 to S303, the distorted image is obtained by distorting the undistorted image. After that, the actual optical flow between the undistorted image and the distorted image obtained by the processing is calculated by the optical flow algorithm. In this way, the model training data such as the distorted image and the actual optical flow are obtained, which is simple and fast without the need for manual data labeling.
[0082] In S304, the distorted image is input into the current high-resolution network to obtain the predicted optical flow corresponding to the distorted image.
[0083] In S305 , the model parameters of the current high-resolution network are updated according to the actual optical flow and the predicted optical flow.
[0084] In S306 , it is determined whether a training cutoff condition is met.
[0085] In the present disclosure, the training cutoff condition may be that the number of network training times reaches a preset number threshold, or that the loss of the current high-resolution network is less than a preset loss threshold.
[0086] If the training cutoff condition is not met, a new sample image is obtained, that is, the process returns to the above S301 and continues to execute until the training cutoff condition is met; if the cutoff condition is met, the following S307 is executed.
[0087] In S307 , the current high-resolution network is determined as the target high-resolution network.
[0088] The following describes in detail the specific implementation of updating the model parameters of the current high-resolution network based on the actual optical flow and the predicted optical flow in S305. Specifically, the loss of the current high-resolution network can be determined based on the actual optical flow and the predicted optical flow. The loss of the current high-resolution network is used to measure the pixel displacement distance between the predicted optical flow and the actual optical flow. The model parameters of the current high-resolution network are then updated based on this loss.
[0089] In one embodiment, the above loss is mean absolute error loss.
[0090] Exemplarily, according to the actual optical flow and the predicted optical flow, the loss of the current high-resolution network (i.e., the mean absolute error loss) can be determined through the following equation (1):
[0091]
[0092] where loss(x, y) is the loss of the current high-resolution network; x is the predicted optical flow corresponding to the distorted image; y is the true optical flow between the sample image and the distorted image; x i is the i-th pixel point of the above-mentioned distorted image, i = 1, 2, ……, n; n is the number of pixel points included in the above-mentioned distorted image; f(x i ) is the predicted optical flow corresponding to the i-th pixel point x i of the above-mentioned distorted image; y i is the actual optical flow corresponding to the i-th pixel point x i of the above-mentioned distorted image.
[0093] In another embodiment, the above loss can be the mean square error loss.
[0094] Exemplarily, according to the actual optical flow and the predicted optical flow, the loss of the current high-resolution network (i.e., the mean square error loss) can be determined through the following equation (2):
[0095]
[0096] In yet another embodiment, the above loss can be the smooth L1 norm loss.
[0097] [[ID=3I]]Exemplarily, according to the actual optical flow and the predicted optical flow, the loss of the current high-resolution network (i.e., the smooth L1 norm loss) can be determined through the following equation (3):
[0098]
[0099] where, in the above equation (3), when the gap between the predicted optical flow and the actual optical flow corresponding to the i-th pixel point x i of the distorted image is small, the gradient (i.e., the derivative) of this gap will also become small, that is, the gradient of the current high-resolution network is small enough, so that the current high-resolution network can converge to a higher accuracy; when the gap between the predicted optical flow and the actual optical flow corresponding to the i-th pixel point x i of the distorted image is large, the upper limit value of the gradient (i.e., the derivative) of this gap is 1, that is, the gradient (i.e., the derivative) of the current high-resolution network is not too large, thus avoiding the damage of the network parameters of the current high-resolution network caused by the too large gradient of the current high-resolution network and avoiding gradient explosion.
[0100] Figure 4is a block diagram of an image correction device shown according to an exemplary embodiment. As Figure 4 shown, the device 400 includes:
[0101] A first acquisition module 401, configured to acquire an image to be corrected;
[0102] A feature extraction module 402, configured to input the image to be corrected acquired by the first acquisition module 401 into a target high-resolution network to obtain a target optical flow and a foreground / background segmentation map corresponding to the image to be corrected;
[0103] A first correction module 403, configured to perform distortion correction on the image to be corrected acquired by the first acquisition module 401 according to the target optical flow and the foreground / background segmentation map obtained by the feature extraction module 402 to obtain a target image.
[0104] In the above technical solution, the image to be corrected is input into a target high-resolution network to obtain a target optical flow and a foreground / background segmentation map corresponding to the image to be corrected; then, according to the target optical flow and the foreground / background segmentation map, distortion correction is performed on the image to be corrected to obtain a target image. In this way, automatic correction of distorted images can be achieved, the difficulty of subsequent image recognition can be reduced, the accuracy of image recognition and the efficiency of image information processing can be improved, and thus the image recognition ability in different scenarios such as the real world and the network can be promoted, especially the image text recognition ability, contributing to the innovation of automated office. In addition, the target high-resolution network adopts a repetitive multi-scale fusion method, parallelly connecting subnets from high resolution to low resolution, and using low-resolution representations with the same depth and similar levels to improve the high-resolution representation, so that the high-resolution to low-resolution representations are stronger semantically and the predicted heatmaps are more accurate spatially, thereby enabling accurate prediction of the target optical flow and the foreground / background segmentation map of the image to be corrected, and further improving the accuracy of subsequent distortion correction and the accuracy of subsequent image recognition.
[0105] Optionally, the first correction module 403 includes:
[0106] A first determination sub-module, configured to extract a foreground region from the image to be corrected according to the foreground / background segmentation map and determine a foreground optical flow corresponding to the foreground region from the target optical flow;
[0107] A first correction sub-module, configured to perform distortion correction on the foreground region according to the foreground optical flow to obtain a target image.
[0108] Optionally, the first correction sub-module includes:
[0109] A second correction sub-module, configured to perform distortion correction on the foreground region according to the foreground optical flow;
[0110] An interpolation processing sub-module is used to perform interpolation processing on the foreground region obtained after distortion correction to obtain a target image.
[0111] Optionally, the target high-resolution network is trained by a training device, where the training device includes:
[0112] A second acquisition module is used to acquire a sample image, where the sample image is an undistorted image;
[0113] A distortion processing module is used to perform distortion processing on the sample image to obtain a distorted image;
[0114] A determination module is used to determine the actual optical flow between the sample image and the distorted image;
[0115] A second correction module is used to input the distorted image into the current high-resolution network to obtain the predicted optical flow corresponding to the distorted image;
[0116] An update module is used to update the model parameters of the current high-resolution network according to the actual optical flow and the predicted optical flow;
[0117] A trigger module is used to: in response to meeting the training cut-off condition, determine the current high-resolution network as the target high-resolution network; in response to not meeting the training cut-off condition, trigger the second acquisition module to acquire a sample image.
[0118] Optionally, the update module includes:
[0119] A second determination sub-module is used to determine the smooth L1 norm loss of the current high-resolution network according to the actual optical flow and the predicted optical flow;
[0120] An update sub-module is used to update the model parameters of the current high-resolution network according to the smooth L1 norm loss.
[0121] Optionally, the distortion processing includes at least one of the following:
[0122] Wrinkle processing, tilt processing, and twist processing.
[0123] Optionally, the actual optical flow is a dense optical flow;
[0124] The determination module is used to calculate the actual optical flow between the sample image and the distorted image through a dense optical flow algorithm.
[0125] In addition, it should be noted that the above training device can be integrated into the above image correction device 400 or can be independent of the above image correction device 400, and the present disclosure does not make a specific limitation.
[0126] Regarding the device in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be elaborated herein.
[0127] The present disclosure provides a non-transitory computer-readable storage medium having stored thereon a computer program, which when executed by a processor implements the steps of the above-described image correction method provided by the present disclosure.
[0128] Figure 5 is a block diagram of an electronic device 700 shown according to an exemplary embodiment. As Figure 5 shown, the electronic device 700 may include: a processor 701, a memory 702. The electronic device 700 may further include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.
[0129] Among them, the processor 701 is used to control the overall operation of the electronic device 700 to complete all or part of the steps in the above image correction method. The memory 702 is used to store various types of data to support the operation of the electronic device 700. These data may include, for example, instructions for any application or method operating on the electronic device 700, as well as application-related data, such as contact data, received and sent messages, pictures, audio, video, and so on. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disc. The multimedia component 703 may include a screen and an audio component. The screen can be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal can be further stored in the memory 702 or sent through the communication component 705. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 704 provides an interface between the processor 701 and other interface modules, and the above other interface modules can be a keyboard, a mouse, buttons, etc. These buttons can be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IoT, eMTC, or other 5G, etc., or a combination of one or more of them is not limited herein. Therefore, the corresponding communication component 705 may include: a Wi-Fi module, a Bluetooth module, an NFC module, and so on.
[0130] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components, and is used to execute the above image correction method.
[0131] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When the program instructions are executed by a processor, the steps of the above image correction method are implemented. For example, the computer-readable storage medium may be the above-mentioned memory 702 including program instructions, and the above program instructions may be executed by the processor 701 of the electronic device 700 to complete the above image correction method.
[0132] Figure 6 FIG. 1900 is a block diagram of an electronic device 1900 shown according to an exemplary embodiment. For example, the electronic device 1900 may be provided as a server. Referring to Figure 6 , the electronic device 1900 includes a processor 1922, the number of which may be one or more, and a memory 1932 for storing computer programs executable by the processor 1922. The computer programs stored in the memory 1932 may include one or more modules each corresponding to a set of instructions. In addition, the processor 1922 may be configured to execute the computer program to execute the above image correction method.
[0133] In addition, the electronic device 1900 may further include a power supply component 1926 and a communication component 1950. The power supply component 1926 may be configured to perform power management of the electronic device 1900, and the communication component 1950 may be configured to implement communication of the electronic device 1900, for example, wired or wireless communication. In addition, the electronic device 1900 may further include an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server TM , Mac OS X TM , Unix TM , Linux TM and so on.
[0134] In another exemplary embodiment, a computer-readable storage medium including program instructions is further provided. When the program instructions are executed by a processor, the steps of the above-described image correction method are implemented. For example, the non-transitory computer-readable storage medium may be the above-described memory 1932 including program instructions, and the above program instructions may be executed by the processor 1922 of the electronic device 1900 to complete the above-described image correction method.
[0135] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program that can be executed by a programmable device, and the computer program has a code portion for executing the above-described image correction method when executed by the programmable device.
[0136] The preferred embodiments of the present disclosure have been described in detail above in conjunction with the accompanying drawings. However, the present disclosure is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.
[0137] In addition, it should be noted that, in the above specific embodiments, the various specific technical features described can be combined in any suitable manner without conflict. To avoid unnecessary repetition, the present disclosure does not separately describe various possible combination manners.
[0138] Furthermore, any combination can be made between various different embodiments of the present disclosure as long as it does not violate the idea of the present disclosure, and it should also be regarded as the content disclosed by the present disclosure.
Claims
1. An image correction method, characterized in that, Including: Obtain an image to be corrected, where the image to be corrected is a distorted image caused by at least one of uneven illumination, perspective change, and paper deformation; Input the image to be corrected into a target high-resolution network to obtain the target optical flow and foreground-background segmentation map corresponding to the image to be corrected, where the foreground-background segmentation map is a segmentation map of the foreground and background of the image to be corrected, and the target high-resolution network is a type of network model in which the feature map always maintains a high resolution during the process of feature extraction of the image to be corrected, and a low-resolution feature map subnet is gradually and parallelly added to the main network of the high-resolution feature map. When using the target high-resolution network to process the image to be corrected, the resolution of the feature map corresponding to the image to be corrected is not reduced; Perform distortion correction on the image to be corrected according to the target optical flow and the foreground-background segmentation map to obtain a target image.
2. The method according to claim 1, wherein The performing distortion correction on the image to be corrected according to the target optical flow and the foreground-background segmentation map to obtain a target image includes: Extract the foreground region from the image to be corrected according to the foreground-background segmentation map, and determine the foreground optical flow corresponding to the foreground region from the target optical flow; Perform distortion correction on the foreground region according to the foreground optical flow to obtain a target image.
3. The method according to claim 2, wherein The performing distortion correction on the foreground region according to the foreground optical flow to obtain a target image includes: Perform distortion correction on the foreground region according to the foreground optical flow; Perform interpolation processing on the foreground region obtained after distortion correction to obtain a target image.
4. The method according to any one of claims 1-3, characterized in that, The target high-resolution network is trained in the following manner: Obtain a sample image, where the sample image is an undistorted image; Perform distortion processing on the sample image to obtain a distorted image; Determine the actual optical flow between the sample image and the distorted image; Input the distorted image into the current high-resolution network to obtain the predicted optical flow corresponding to the distorted image; Update the model parameters of the current high-resolution network according to the actual optical flow and the predicted optical flow; In response to satisfying the training cut-off condition, determine the current high-resolution network as the target high-resolution network; In response to not satisfying the training cut-off condition, re-execute the steps from obtaining the sample image to updating the model parameters of the current high-resolution network according to the actual optical flow and the predicted optical flow.
5. The method according to claim 4, wherein The updating the model parameters of the current high-resolution network according to the actual optical flow and the predicted optical flow includes: Determine the smooth L1 norm loss of the current high-resolution network according to the actual optical flow and the predicted optical flow; Update the model parameters of the current high-resolution network according to the smooth L1 norm loss.
6. The method according to claim 4, wherein The distortion processing includes at least one of the following: Wrinkle processing, tilt processing, and twist processing.
7. The method according to claim 4, wherein The actual optical flow is a dense optical flow; The determining the actual optical flow between the sample image and the distorted image includes: Calculate the actual optical flow between the sample image and the distorted image through a dense optical flow algorithm.
8. An image correction device, characterized in that, Including: A first acquisition module, configured to acquire an image to be corrected, where the image to be corrected is a distorted image caused by at least one of uneven illumination, perspective change, and paper deformation; A feature extraction module, configured to input the image to be corrected acquired by the first acquisition module into a target high-resolution network to obtain a target optical flow and a foreground-background segmentation map corresponding to the image to be corrected, where the foreground-background segmentation map is a segmentation map of the foreground and background of the image to be corrected, and the target high-resolution network is a type of network model in which the feature map always maintains a high resolution during the process of feature extraction of the image to be corrected, and a low-resolution feature map sub-network is gradually and parallelly added to the main network of the high-resolution feature map. When using the target high-resolution network to process the image to be corrected, the resolution of the feature map corresponding to the image to be corrected is not reduced; A first correction module, configured to perform distortion correction on the image to be corrected acquired by the first acquisition module according to the target optical flow and the foreground-background segmentation map obtained by the feature extraction module to obtain a target image.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method according to any one of claims 1-7.
10. An electronic device, characterized in that, Comprising: A memory, on which a computer program is stored; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1-7.
Citation Information
Patent Citations
Image processing method, intelligent equipment and computer readable storage medium
CN111476737A