Image correction model training method, image correction method, device, and storage medium
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- QINGDAO INTELLIFUSION TECH CO LTD
- Filing Date
- 2022-08-30
- Publication Date
- 2026-07-21
Smart Images
Figure CN115423691B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image correction, and more particularly to a training method for an image correction model, an image correction method, a computer device, and a storage medium. Background Technology
[0002] With the continuous development of technology, facial recognition technology has been widely used in society, such as in the security field. However, since the location of the device for capturing faces is usually fixed, while the walking trajectory and body direction of pedestrians are not fixed, the captured facial images often have a certain degree of tilt or rotation. Therefore, it is necessary to perform image correction on these tilted or rotated facial images to ensure the accuracy of facial recognition. Currently, existing technologies typically label key points in facial images during image correction and then use these labeled images to train a neural network. However, this training method requires a large amount of labeled data, which is costly and prone to introducing errors, resulting in low accuracy of the final trained neural network. Summary of the Invention
[0003] This application provides a training method for an image correction model, an image correction method, a computer device, and a storage medium to reduce the training cost of the image correction model and improve the accuracy of the obtained image correction model.
[0004] In a first aspect, this application provides a method for training an image correction model, the method comprising:
[0005] Acquire training data, which includes training images and rotated images corresponding to the training images;
[0006] The rotated image is input into a preset correction network to obtain the affine transformation matrix corresponding to the rotated image;
[0007] The affine transformation matrix is used to perform an affine transformation on the rotated image, and the resulting transformation data is input into a sampling network for image sampling to obtain the corrected image corresponding to the rotated image.
[0008] Calculate the loss function value between the corrected image and the training image, and iteratively train the preset correction network and the sampling network based on the loss function value. When the training is completed, the preset correction network and the sampling network are used together as the image correction model.
[0009] Secondly, this application also provides an image correction method, the method comprising:
[0010] Obtain the image to be corrected;
[0011] The image to be corrected is input into a pre-trained image correction model to obtain a corrected image, wherein the pre-trained image correction model is trained using the image correction model training method described in the first aspect.
[0012] Thirdly, this application also provides a computer device, the computer device including a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and, when executing the computer program, implement the training method of the image correction model as described in the first aspect and / or the image correction method as described in the second aspect.
[0013] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the training method of the image correction model as described in the first aspect and / or the image correction method as described in the second aspect.
[0014] This application discloses a training method for an image correction model, an image correction method, a computer device, and a storage medium. The method uses training images and their corresponding rotated images as training data for the image correction model. This reduces the amount of labeled data during training, thereby reducing dependence on labeled data, lowering the training cost, and improving the accuracy of the trained image correction model. Furthermore, a corrected image is obtained based on an affine transformation matrix and a sampling network. Then, a preset correction network and a sampling network are iteratively trained using the corrected image and the training image. After training, the sampling network and the preset correction network are used together as the image correction model, and the sampling network is also included as part of the image correction model during training. Using the corrected image for unsupervised training of the image correction model ensures high accuracy of the trained model while reducing labeling costs. Attached Figure Description
[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1 A schematic flowchart illustrating a training method for an image correction model provided in an embodiment of this application;
[0017] Figure 2 This is a schematic diagram illustrating the steps of performing an affine transformation on a rotated image according to an embodiment of this application;
[0018] Figure 3 This is a schematic diagram illustrating the steps of image sampling provided in an embodiment of this application;
[0019] Figure 4 A schematic flowchart illustrating an image correction method provided in an embodiment of this application;
[0020] Figure 5 A schematic block diagram of a training device for an image correction model provided in an embodiment of this application;
[0021] Figure 6 This is a schematic block diagram of an image correction device provided in an embodiment of this application;
[0022] Figure 7 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation
[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0024] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.
[0025] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.
[0026] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0027] It is also understood that in the specific embodiments of this application, data related to faces is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.
[0028] Embodiments of this application provide a training method for an image correction model, an image correction method, a computer device, and a storage medium. The image correction model obtained using the training method can be used to correct acquired face images and improve the accuracy of the corrected images, thereby increasing the success rate and accuracy of face recognition in subsequent face recognition processes.
[0029] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0030] Please see Figure 1 , Figure 1 This is a schematic flowchart illustrating a training method for an image correction model provided in an embodiment of this application. This method iteratively trains the image correction model using simulated training data, which reduces training costs and improves the accuracy of the trained image correction model.
[0031] like Figure 1 As shown, the training method of the image correction model specifically includes steps S101 to S105.
[0032] Step S101: Obtain training data, which includes training images and the rotated images corresponding to the training images.
[0033] Acquire training data, which is used to train the image correction model. The training data includes the training image and its corresponding rotated image. The rotated image is obtained by rotating the training image by a preset angle.
[0034] In one embodiment, before step S101, the method may further include the following steps: acquiring a training image and rotating the training image to obtain a rotated image corresponding to the training image.
[0035] In other words, a rotated image can be obtained by rotating the training image by a certain angle beforehand. When acquiring training data, the training image, its corresponding rotated image, and the rotation angle can be directly obtained. In practice, multiple rotated images can be obtained by rotating a different training image by different angles, thereby increasing the amount of training data. Furthermore, for different training images, the rotation angles can be different or the same.
[0036] In the specific implementation process, a large number of frontal images can be selected from the business scenario as training images. These training images include images without obvious angular rotation. Then, by artificially introducing random rotation angles, a large number of rotated images for training can be generated.
[0037] When training an image correction model, if the training data consists of labeled face images and corresponding key points, a large amount of labeled data is required for network training, resulting in high costs for acquiring training data. Furthermore, the accuracy of the image correction model depends on the accuracy of the labeled data. Therefore, this application uses rotated training images to participate in the training of the image correction model. Compared with the scheme of obtaining manually labeled data including key points as training data, this method of generating training data by rotating images does not require pre-labeling of the rotated images. Instead, the rotation angle can be controlled automatically. On the one hand, this enriches the training data, reduces the dependence on manually labeled data during training, and lowers the cost of acquiring training data. On the other hand, it also reduces errors in the process of manually labeling data, thereby further improving the accuracy of the trained image correction model.
[0038] Step S102: Input the rotated image into a preset correction network to obtain the affine transformation matrix corresponding to the rotated image.
[0039] The rotated image is input into a pre-defined correction network to obtain the affine transformation matrix corresponding to the rotated image. This affine transformation matrix is also the affine transformation matrix output by the pre-defined correction network corresponding to the rotated image. This affine transformation matrix can be a 2x3 matrix, containing the information needed to correct the face image, including rotation, translation, and scaling. The affine transformation matrix can be represented as:
[0040] Θ=f loc (U)
[0041] Where U is the input image, and Θ is the affine transformation matrix. The affine transformation matrix has a 2x3 shape; therefore, the pre-defined correction network can be viewed as a six-node regressor, and Θ can be expressed as:
[0042]
[0043] In another embodiment, the preset correction network can be a pre-trained correction network trained on a small amount of labeled data. Using a small amount of labeled data to perform supervised pre-training of the correction network can improve the training efficiency of the image correction model.
[0044] In one embodiment, the method may further include the following steps: acquiring sample data, the sample data including sample images and key points corresponding to the sample images; inputting the sample images into a convolutional neural network to obtain an output affine matrix; determining a supervised affine matrix based on the key points corresponding to the sample images and preset positioning points; calculating the loss function value between the output affine matrix and the supervised affine matrix, and pre-training the convolutional neural network according to the loss function value to obtain a pre-trained correction network.
[0045] The process involves acquiring sample data, which refers to labeled sample images, including the sample images themselves and the labeled keypoints within them. In practice, the sample data can be labeled data from the training dataset or new data outside of the training dataset. The sample images are then input into a convolutional neural network (CNN) to obtain the output of the CNN, which is the affine matrix.
[0046] Then, a supervised affine matrix is determined based on the key points corresponding to the sample images and the preset positioning points. The preset positioning points are points with definite coordinate positions set in advance; each key point can correspond to a preset positioning point. In practice, by setting preset positioning points, key points at the same location in different images can be corrected to a fixed position. For example, the coordinates of the eyes and mouth in a face image are always at a fixed point.
[0047] After obtaining the supervised affine matrix, the loss values of the output affine matrix and the supervised affine matrix are calculated, and the convolutional neural network is iteratively trained based on the calculated loss function values to obtain the pre-trained correction network.
[0048] In practice, the mean absolute error (MAE) loss function can be used to iteratively train the convolutional neural network. For example, since both the output affine matrix and the supervision affine matrix contain six values, the values at corresponding positions in the output affine matrix and the supervision affine matrix can be subtracted and their absolute values calculated. Then, the absolute values can be added together and averaged to obtain the loss function value between the output affine matrix and the supervision affine matrix.
[0049] The parameters of the convolutional neural network (CNN) are continuously adjusted by using the loss function value between the output affine matrix and the supervised affine matrix. This process aims to make the output affine matrix of the CNN more closely resemble the supervised affine matrix, until the loss function value of the CNN reaches a preset value. At this point, the CNN converges and can be used as a pre-trained correction network. By utilizing the loss function value between the output affine matrix and the supervised affine matrix for supervised training of the CNN in a matrix manner, the preset correction network can be obtained relatively quickly, thereby improving the training speed of the image correction model.
[0050] In another embodiment, the method may further include the following steps: acquiring training sample data, the training sample data including a rotated sample image corresponding to a training sample image and a rotation angle corresponding to the training sample image; inputting the rotated sample image into a convolutional neural network to obtain an output affine matrix corresponding to the rotated sample image; determining a supervised affine matrix based on the rotated sample image and the rotation angle corresponding to the rotated sample image; calculating a loss function value between the output affine matrix and the supervised affine matrix, and pre-training the convolutional neural network based on the loss function value to obtain a pre-trained correction network.
[0051] Training sample data is acquired and used for pre-training the convolutional neural network. This training sample data includes rotated sample images corresponding to the training sample images and the corresponding rotation angles. The rotated sample images are obtained by rotating the training sample images by a preset angle. For example, if a training sample image is rotated 15° clockwise, the corresponding rotation angle is 15°. In practice, the training sample data can be the training data used for iterative training of the preset correction and sampling networks, thereby reducing the number of samples required during the training of the image correction model and thus lowering the training cost.
[0052] The rotated sample image is input into the convolutional neural network to obtain the output affine matrix corresponding to the rotated sample image. Since the rotated sample image is obtained by rotating the training sample image by a certain angle, that is, the rotation angle of the rotated sample image is known. Because rotating the image is equivalent to multiplying the image matrix by an affine matrix, and this affine matrix is related to the rotation angle, the supervised affine matrix corresponding to the rotated image can be directly calculated based on the rotated sample image and its corresponding rotation angle.
[0053] After obtaining the supervised affine matrix, the loss values of the output affine matrix and the supervised affine matrix are calculated, and the convolutional neural network is iteratively trained based on the calculated loss function values to obtain the pre-trained correction network.
[0054] In practice, the mean absolute error (MAE) loss function can be used to iteratively train the convolutional neural network. By continuously adjusting the parameters of the convolutional neural network using the loss function value between the output affine matrix and the supervised affine matrix, the output affine matrix of the convolutional neural network can be made closer to the supervised affine matrix, until the loss value of the convolutional neural network's loss function reaches a preset value. At this point, the convolutional neural network converges, and this converged convolutional neural network can be used as a pre-trained correction network.
[0055] In practice, since the final image correction model is used to correct facial images, it is often a prerequisite for face recognition. Therefore, the image correction model should have as few parameters as possible and run quickly to improve the recognition speed of face recognition. For example, a CNN network with a MobileNet network structure can be selected as a convolutional neural network.
[0056] In one embodiment, before the step of inputting the rotated image into a preset correction network to obtain the affine transformation matrix corresponding to the rotated image, the method may further include the following step: performing image preprocessing on the rotated image, wherein the image preprocessing includes size adjustment and / or image enhancement.
[0057] Image preprocessing is performed on the rotated image. Image preprocessing includes resizing, which adjusts the size of the rotated image to a fixed size. Image preprocessing may also include image enhancement, which is used to make the image clearer.
[0058] In the specific implementation process, image preprocessing can be performed on the rotated image or on the training image. After image preprocessing of the training image, the rotated image obtained is the preprocessed image, so image preprocessing of the rotated image is not required.
[0059] Step S103: Perform an affine transformation on the rotated image based on the affine transformation matrix, and input the obtained transformation data into a sampling network for image sampling to obtain the corrected image corresponding to the rotated image.
[0060] An affine transformation is performed on the rotated image using an affine transformation matrix to obtain transformed data. This transformed data includes the mapped coordinates of each pixel in the rotated image after the affine transformation. The transformed data is then input into a sampling network for image sampling, resulting in a corrected image corresponding to the rotated image. Image sampling can be differential image sampling, which involves interpolating and rounding the pixel position value obtained through the affine transformation to obtain the corresponding actual sampling coordinates. Finally, pixels are sampled in the rotated image based on these actual sampling coordinates. In other words, for each pixel position in the corrected image, a corresponding pixel value is found in the rotated image and used to fill the position.
[0061] In one embodiment, please refer to Figure 2 This is a schematic diagram illustrating the steps of performing an affine transformation on a rotated image according to an embodiment of this application. Figure 2 As shown, the step of performing an affine transformation on the rotated image based on the affine transformation matrix may include: step S1031, obtaining the pixel coordinates of each pixel in the rotated image; step S1032, mapping the pixel coordinates of each pixel based on the affine transformation matrix to obtain the mapped coordinates of each pixel.
[0062] First, obtain the pixel coordinates of a single pixel in the rotated image, which can be denoted as: After the affine transformation matrix Θ, the mapped coordinates of the pixel become The specific process of affine transformation is as follows:
[0063]
[0064] By mapping the coordinates of all pixels in the rotated image in the manner described above, the mapped coordinates of each pixel can be obtained.
[0065] In one embodiment, the step of inputting the obtained transformation data into a sampling network for image sampling to obtain the corrected image corresponding to the rotated image may include:
[0066] When the mapped coordinates of the pixel are not integers, the mapped coordinates of the pixel are input into the sampling network for image sampling to obtain the corrected image corresponding to the rotated image; when the mapped coordinates of the pixel are integers, the pixel corresponding to the mapped coordinates is obtained in the rotated image according to the mapped coordinates and pixel filling is performed to obtain the corrected image.
[0067] Normally, the mapped coordinates of pixels calculated by the affine transformation matrix are floating-point values rather than integer values. Therefore, in order to obtain the corrected image, these non-integer mapped coordinates need to be interpolated and rounded before pixel filling can be performed.
[0068] In the specific implementation process, after obtaining the mapped coordinates of each pixel, it is determined whether the mapped coordinates of the pixel are integers. If the mapped coordinates of the pixel are integers, it means that the mapped coordinates of the pixel correspond to some spatial position in the rotated image. In this case, the pixel corresponding to the sampling coordinates can be directly obtained in the rotated image based on the mapped coordinates of the pixel for pixel filling. However, if the mapped coordinates of the pixel are not integers, it means that the mapped coordinates cannot be directly mapped to the spatial position in the rotated image. In this case, the mapped coordinates of the pixel need to be input into the sampling network for image sampling. After all pixels have been filled, the corrected image corresponding to the rotated image is obtained.
[0069] In one embodiment, please refer to Figure 3 This is a schematic diagram illustrating the image sampling steps provided in an embodiment of this application. Figure 3 As shown, the step of inputting the mapped coordinates of the pixels into a sampling network for image sampling to obtain the corrected image corresponding to the rotated image may include:
[0070] Step S1033: Interpolate and round the mapped coordinates of the pixel to obtain the sampling coordinates corresponding to the pixel; Step S1034: Obtain the pixel corresponding to the sampling coordinate in the rotated image according to the sampling coordinate and fill the pixel to obtain the corrected image.
[0071] Interpolation is performed on the mapped coordinates of pixels. In practice, various methods can be used for interpolation, such as bilinear interpolation and nearest neighbor interpolation. Bilinear interpolation involves finding the four nearest integer points to the desired coordinates and summing them according to distance weights, with closer points having higher weights. The interpolation process can be summarized as follows:
[0072]
[0073] Numerical parameters, V represents the value of the rotated image at (n,m,c). i c Indicates the corrected image in The value of .
[0074] After obtaining the sampling coordinates, the corresponding pixel points in the rotated image are sampled according to the sampling coordinates. Then, the obtained pixel points are filled with pixels until all pixels in the rotated image are traversed. Finally, the image obtained after pixel filling is used as the corrected image.
[0075] Step S104: Calculate the loss function value between the corrected image and the training image, and iteratively train the preset correction network and the sampling network according to the loss function value. When the training is completed, the preset correction network and the sampling network are used together as the image correction model.
[0076] After obtaining the corrected image, a loss function is used to calculate the loss between the corrected image and the training image. In practice, the loss function can be the mean squared error (MSE) loss function. The pre-defined correction network and sampling network are then iteratively trained based on the calculated loss function value.
[0077] In the specific implementation process, since the sampling network is differentiable and satisfies the backpropagation condition, it can be trained end-to-end with the preset correction network. That is to say, the sampling network is used as a network layer in the image correction model to participate in the training of the preset correction network.
[0078] In the process of iteratively training the preset correction network and sampling network, the preset correction network and sampling network can be trained together, or the preset correction network can be iteratively trained using a sampling network with fixed parameters.
[0079] When training both the pre-defined correction network and the sampling network, a loss function is calculated between the corrected image and the training image. Based on this loss function value, the sampling parameters in the sampling network and the weights of the network parameters in the pre-defined correction network are adjusted to ensure that the corrected image output by the image correction model more closely approximates the training image, thereby improving the accuracy of image correction. In practice, the weights of the network parameters in the pre-defined correction network can be adjusted first. After the weights of the network parameters in the pre-defined correction network are appropriately adjusted, the sampling parameters in the sampling network are then adjusted accordingly.
[0080] When the parameters of the sampling network are fixed, the loss function value between the corrected image and the training image is calculated based on the loss function. At this point, the weight values of the network parameters in the preset correction network can be adjusted according to the calculated loss function value, making the affine transformation matrix output by the preset correction network more accurate. Furthermore, after training, the trained preset correction network and the sampling network are used together as an image correction model to participate in image correction. In specific implementation, methods such as stochastic gradient descent, gradient descent, Newton's method, quasi-Newton methods, and conjugate gradient methods can be used to adjust the weight values of the parameters in the preset correction network.
[0081] When calculating the loss function value between the corrected image and the training image using the mean squared error loss function, the distance between each pixel in the corrected image and the training image can be calculated, the square of the distance can be obtained, and then the sums of the squares can be averaged to obtain the loss function value between the corrected image and the training image. When the loss function value between the corrected image and the training image reaches a preset value or reaches a minimum value, the training of the preset correction network can be considered complete.
[0082] By sampling the output of the network and using the corrected image and training image to iteratively train the preset correction network and sampling network, the accuracy of the affine transformation matrix output by the preset correction network can be improved, thereby comprehensively improving the correctness and accuracy of the corrected image output by the entire image correction model.
[0083] The image correction model training method provided in the above embodiments uses training images and their corresponding rotated images as training data for the image correction model. This reduces the amount of labeled data during training, thereby reducing dependence on labeled data, lowering the training cost of the image correction model, and improving the accuracy of the trained model. Furthermore, a corrected image is obtained based on an affine transformation matrix and a sampling network. Then, a preset correction network and sampling network are iteratively trained using the corrected image and training images. After training, the sampling network and the preset correction network are used together as the image correction model, and the sampling network is also included as part of the model during training. Using the corrected image for unsupervised training of the image correction model ensures high accuracy of the trained model while reducing labeling costs.
[0084] Please see Figure 4 , Figure 4 This is a schematic diagram illustrating the steps of an image correction method provided in an embodiment of this application. For example... Figure 4 As shown, the image correction method includes steps S201 and S202.
[0085] Step S201: Obtain the image to be corrected.
[0086] Before image correction, the acquired image can be preprocessed to obtain the image to be corrected. Image preprocessing includes modifying the image size to adjust it to a preset size to ensure the accuracy of the corrected image obtained after image correction.
[0087] Step S202: Input the image to be corrected into a pre-trained image correction model to obtain a corrected image, wherein the pre-trained image correction model is trained using the above-described image correction model training method.
[0088] By inputting the image to be corrected into the image correction model trained using the aforementioned image correction model training method, the corrected image corresponding to the image to be corrected can be obtained.
[0089] In the specific implementation process, after the image to be corrected is input into the pre-trained image correction model, it passes through the correction network in the pre-trained image correction model and outputs the affine transformation matrix corresponding to the image to be corrected. Then, the affine transformation matrix is used to perform an affine transformation on the image to be corrected to obtain the transformation data corresponding to the image to be corrected. The transformation data includes the mapped coordinates of each pixel in the image to be corrected after the affine transformation. Then, the transformation data corresponding to the image to be corrected is input into the sampling network for image sampling, and the corrected image generated after image sampling is used as the output of the pre-trained image correction model, thereby obtaining the corrected image corresponding to the image to be corrected.
[0090] The image correction method provided in the above embodiments uses a pre-trained image correction model to correct the image to be corrected, which can perform image correction with high accuracy, thereby improving the success rate and accuracy of recognition when performing face recognition or other actions in the future.
[0091] Please see Figure 5 , Figure 5 This is a schematic block diagram of a training device for an image correction model provided in an embodiment of this application. Figure 5 As shown, the training device for the image correction model includes: a data acquisition module 301, a matrix generation module 302, an image generation model 303, and an iterative training module 304. Among them,
[0092] The data acquisition module 301 is used to acquire training data, which includes training images and rotated images corresponding to the training images.
[0093] The matrix generation module 302 is used to input the rotated image into a preset correction network to obtain the affine transformation matrix corresponding to the rotated image.
[0094] Image generation model 303 is used to perform an affine transformation on the rotated image based on the affine transformation matrix, and input the obtained transformation data into a sampling network for image sampling to obtain a corrected image corresponding to the rotated image.
[0095] The iterative training module 304 is used to calculate the loss function value between the corrected image and the training image, and to iteratively train the preset correction network and the sampling network according to the loss function value. When the training is completed, the preset correction network and the sampling network are used together as the image correction model.
[0096] Please see Figure 6 , Figure 6 This is a schematic block diagram of an image correction device provided in an embodiment of this application. Figure 6 As shown, the image correction device includes an image acquisition module 401 and an image correction module 402.
[0097] in,
[0098] Image acquisition module 401 is used to acquire the image to be corrected.
[0099] The image correction module 402 is used to input the image to be corrected into a pre-trained image correction model to obtain a corrected image, wherein the pre-trained image correction model is trained using the image correction model training method described above.
[0100] Please see Figure 7 , Figure 7 This is a schematic block diagram illustrating the structure of a computer device according to an embodiment of this application. The computer device may be a server or a terminal.
[0101] like Figure 7 As shown, the computer device includes a processor, memory, and network interface connected via a system bus, wherein the memory may include non-volatile storage media and internal memory.
[0102] Non-volatile storage media can store operating systems and computer programs. These computer programs include program instructions that, when executed, cause the processor to perform a training method for any image correction model.
[0103] The processor provides computing and control capabilities, supporting the operation of the entire computer device.
[0104] Internal memory provides an environment for the execution of computer programs in non-volatile storage media. When the computer program is executed by the processor, it enables the processor to execute any training method for an image correction model.
[0105] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0106] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.
[0107] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps:
[0108] Acquire training data, which includes training images and rotated images corresponding to the training images;
[0109] The rotated image is input into a preset correction network to obtain the affine transformation matrix corresponding to the rotated image;
[0110] The affine transformation matrix is used to perform an affine transformation on the rotated image, and the resulting transformation data is input into a sampling network for image sampling to obtain the corrected image corresponding to the rotated image.
[0111] Calculate the loss function value between the corrected image and the training image, and iteratively train the preset correction network and the sampling network based on the loss function value. When the training is completed, the preset correction network and the sampling network are used together as the image correction model.
[0112] In one embodiment, the preset correction network is a pre-trained correction network; the processor is further configured to implement:
[0113] Acquire sample data, which includes sample images and key points corresponding to the sample images;
[0114] The sample image is input into a convolutional neural network to obtain the output affine matrix;
[0115] The supervisory affine matrix is determined based on the key points and preset positioning points corresponding to the sample images;
[0116] Calculate the loss function value between the output affine matrix and the supervised affine matrix, and pre-train the convolutional neural network based on the loss function value to obtain a pre-trained correction network.
[0117] In one embodiment, the preset correction network is a pre-trained correction network; the processor is further configured to implement:
[0118] Acquire training sample data, which includes rotated sample images corresponding to training sample images and rotation angles corresponding to the training sample images;
[0119] The rotated sample image is input into a convolutional neural network to obtain the output affine matrix corresponding to the rotated sample image;
[0120] The supervised affine matrix is determined based on the rotated sample image and the rotation angle corresponding to the rotated sample image;
[0121] Calculate the loss function value between the output affine matrix and the supervised affine matrix, and pre-train the convolutional neural network based on the loss function value to obtain a pre-trained correction network.
[0122] In one embodiment, when implementing the affine transformation of the rotated image based on the affine transformation matrix, the processor is configured to:
[0123] Obtain the pixel coordinates of each pixel in the rotated image;
[0124] The pixel coordinates of each pixel are mapped based on the affine transformation matrix to obtain the mapped coordinates of each pixel.
[0125] In one embodiment, when the processor inputs the obtained transformed data into a sampling network for image sampling to obtain the corrected image corresponding to the rotated image, it is configured to:
[0126] When the mapped coordinates of the pixel are not integers, the mapped coordinates of the pixel are input into the sampling network for image sampling to obtain the corrected image corresponding to the rotated image;
[0127] When the mapped coordinates of the pixel are integers, the pixel corresponding to the mapped coordinates is obtained in the rotated image and pixel filling is performed to obtain the corrected image.
[0128] In one embodiment, when the processor implements the step of inputting the mapped coordinates of the pixel points into a sampling network for image sampling to obtain the corrected image corresponding to the rotated image, it is configured to:
[0129] The mapping coordinates of the pixel are interpolated and rounded to obtain the sampling coordinates corresponding to the pixel.
[0130] Based on the sampling coordinates, obtain the pixels corresponding to the sampling coordinates in the rotated image, fill in the pixels, and obtain the corrected image.
[0131] In one embodiment, before inputting the rotated image into a preset correction network to obtain the affine transformation matrix corresponding to the rotated image, the processor performs the following:
[0132] The rotated image is preprocessed, including resizing and / or image enhancement.
[0133] In another embodiment, the processor is configured to run a computer program stored in memory to perform the following steps:
[0134] Obtain the image to be corrected;
[0135] The image to be corrected is input into a pre-trained image correction model to obtain a corrected image. The pre-trained image correction model is trained using the image correction model training method described above.
[0136] The embodiments of this application also provide a computer-readable storage medium storing a computer program, the computer program including program instructions, and the processor executing the program instructions to implement the training method and / or image correction method of any of the image correction models provided in the embodiments of this application.
[0137] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device.
[0138] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A training method for an image correction model, characterized in that, The method includes: Acquire training data, which includes a training image and a rotated image corresponding to the training image; the rotated image is obtained by rotating the training image by a preset angle, and the training data also includes the rotation angle corresponding to the training image, which is used to indicate the preset angle of the rotated image relative to the training image; The rotated image is input into a preset correction network to obtain the affine transformation matrix corresponding to the rotated image; The affine transformation matrix is used to perform an affine transformation on the rotated image, and the resulting transformation data is input into a sampling network for image sampling to obtain the corrected image corresponding to the rotated image. Calculate the loss function value between the corrected image and the training image, and iteratively train the preset correction network and the sampling network based on the loss function value. When the training is completed, the preset correction network and the sampling network are used together as the image correction model. The preset correction network is a pre-trained correction network; before inputting the rotated image into the preset correction network, the method further includes: Acquire training sample data, which includes rotated sample images corresponding to training sample images and rotation angles corresponding to the training sample images; The rotated sample image is input into a convolutional neural network to obtain the output affine matrix corresponding to the rotated sample image; The supervised affine matrix is determined based on the rotated sample image and the rotation angle corresponding to the rotated sample image; Calculate the loss function value between the output affine matrix and the supervised affine matrix, and pre-train the convolutional neural network based on the loss function value to obtain a pre-trained correction network.
2. The training method for the image correction model according to claim 1, characterized in that, Before inputting the rotated image into a preset correction network, the method further includes: Acquire sample data, which includes sample images and key points corresponding to the sample images; The sample image is input into a convolutional neural network to obtain the output affine matrix; The supervisory affine matrix is determined based on the key points and preset positioning points corresponding to the sample images; Calculate the loss function value between the output affine matrix and the supervised affine matrix, and pre-train the convolutional neural network based on the loss function value to obtain a pre-trained correction network.
3. The training method for the image correction model according to claim 1, characterized in that, The affine transformation of the rotated image based on the affine transformation matrix includes: Obtain the pixel coordinates of each pixel in the rotated image; The pixel coordinates of each pixel are mapped based on the affine transformation matrix to obtain the mapped coordinates of each pixel.
4. The training method for the image correction model according to claim 3, characterized in that, The step of inputting the obtained transformed data into a sampling network for image sampling to obtain the corrected image corresponding to the rotated image includes: When the mapped coordinates of the pixel are not integers, the mapped coordinates of the pixel are input into the sampling network for image sampling to obtain the corrected image corresponding to the rotated image; When the mapped coordinates of the pixel are integers, the pixel corresponding to the mapped coordinates is obtained in the rotated image and pixel filling is performed to obtain the corrected image.
5. The training method for the image correction model according to claim 4, characterized in that, The step of inputting the mapped coordinates of the pixels into a sampling network for image sampling to obtain the corrected image corresponding to the rotated image includes: The mapping coordinates of the pixel are interpolated and rounded to obtain the sampling coordinates corresponding to the pixel. Based on the sampling coordinates, obtain the pixels corresponding to the sampling coordinates in the rotated image, fill in the pixels, and obtain the corrected image.
6. The training method for the image correction model according to any one of claims 1-5, characterized in that, Before inputting the rotated image into a preset correction network to obtain the affine transformation matrix corresponding to the rotated image, the method includes: The rotated image is preprocessed, including resizing and / or image enhancement.
7. An image correction method, characterized in that, The method includes: Obtain the image to be corrected; The image to be corrected is input into a pre-trained image correction model to obtain a corrected image, wherein the pre-trained image correction model is trained using the training method for image correction models as described in any one of claims 1 to 6.
8. A computer device, characterized in that, The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, in executing the computer program, implement the training method of the image correction model as described in any one of claims 1 to 6 and / or the image correction method as described in claim 7.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the training method of the image correction model as described in any one of claims 1 to 6 and / or the image correction method as described in claim 7.