Image correction method, operation correction method, and related device, equipment and medium

Abstract

The application discloses an image correction method, a homework correction method and related devices, equipment and media, wherein the image correction method comprises: extracting image features of a to-be-corrected image; modeling a dependency relationship between pixels in the to-be-corrected image based on the image features to obtain global features of the to-be-corrected image; performing local feature enhancement based on the global features to obtain enhanced features; performing prediction based on the enhanced features to obtain a deformation field; and correcting the to-be-corrected image based on the deformation field to obtain a corrected image. The above scheme can improve the balance of image correction and the applicability to different deformations.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image correction method, a job correction method, and related devices, equipment, and media. Background Technology

[0002] With the continuous advancement of digital transformation, the visualization of paper documents has been widely used in many fields. However, due to factors such as improper shooting angles or uneven lighting, paper documents often suffer from distortions such as deformation and curvature in image data, which in turn affects subsequent tasks such as image analysis.

[0003] Existing image correction methods are mainly based on geometric modeling, neural networks, etc., but the former is mainly applicable to regular deformations and has poor adaptability to nonlinear deformations, while the latter has uneven correction effects. In view of this, how to improve the uniformity of image correction and its applicability to different deformations has become an urgent problem to be solved. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide an image correction method, a job correction method, and related devices, equipment, and media that can improve the uniformity of image correction and its applicability to different deformations.

[0005] To address the aforementioned technical problems, the first aspect of this application provides an image correction method, comprising: extracting image features of an image to be corrected; modeling the dependencies between pixels in the image to be corrected based on the image features to obtain global features of the image to be corrected; performing local feature enhancement based on the global features to obtain enhanced features; performing prediction based on the enhanced features to obtain a deformation field; and correcting the image to be corrected based on the deformation field to obtain a corrected image.

[0006] To address the aforementioned technical problems, a second aspect of this application provides a method for grading assignments, comprising: acquiring an assignment image as an image to be corrected; performing correction processing on the image to be corrected to obtain a corrected image; wherein the corrected image is obtained by the image correction method described in the first aspect; performing recognition based on the corrected image to obtain assignment data; and grading based on the assignment data to obtain a grading result.

[0007] To address the aforementioned technical problems, a third aspect of this application provides an image correction device, comprising: a feature extraction module, a global modeling module, a local enhancement module, a deformation prediction module, and a correction processing module. The feature extraction module is used to extract image features of the image to be corrected; the global modeling module is used to model the dependencies between pixels in the image to be corrected based on the image features to obtain global features of the image to be corrected; the local enhancement module is used to enhance local features based on the global features to obtain enhanced features; the deformation prediction module is used to predict based on the enhanced features to obtain a deformation field; and the correction processing module is used to correct the image to be corrected based on the deformation field to obtain a corrected image.

[0008] To address the aforementioned technical problems, a fourth aspect of this application provides a job correction device, comprising: an image acquisition module, an image correction module, an image recognition module, and a data processing module. The image acquisition module is used to acquire a job image as an image to be corrected. The image correction module is used to perform correction processing on the image to be corrected to obtain a corrected image. The corrected image is obtained by the image correction device described in the third aspect above. The image recognition module is used to perform recognition based on the corrected image to obtain job data. The data processing module is used to perform correction based on the job data to obtain a correction result.

[0009] To address the aforementioned technical problems, this application provides an electronic device that includes at least a memory and a processor coupled to each other. The memory stores at least program instructions, and the processor executes the program instructions to implement the image correction method in the first aspect or the job correction method in the second aspect.

[0010] To address the aforementioned technical problems, a sixth aspect of this application provides a computer-readable storage medium storing program instructions executable by a processor. These program instructions are used to implement the image correction method of the first aspect or the job correction method of the second aspect.

[0011] The above scheme extracts image features from the image to be corrected, models the dependencies between pixels in the image based on these features to obtain global features, and then enhances local features based on these global features to obtain enhanced features. Based on these enhanced features, a deformation field is predicted, and finally, the image to be corrected is corrected based on this deformation field to obtain the corrected image. Therefore, on the one hand, by sequentially extracting global features and enhancing local features, it is possible to further enhance local details while capturing the global deformation pattern of the image, which helps improve the balance of image correction. On the other hand, because global features are extracted to capture the global deformation pattern during image correction, compared with traditional geometry-based methods, it can handle not only linear deformations but is also more conducive to handling nonlinear deformations, thus improving applicability to different deformations. Therefore, it can improve the balance of image correction and its applicability to different deformations. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating an embodiment of the image correction method of this application;

[0013] Figure 2a This is a schematic diagram illustrating an embodiment of feature extraction of the image to be corrected according to this application;

[0014] Figure 2b This is a schematic diagram illustrating an embodiment of global feature extraction in this application;

[0015] Figure 2c This is a schematic diagram of a process for enhancing local features according to an embodiment of this application;

[0016] Figure 2d This is a schematic diagram illustrating the effect of an embodiment of image correction based on deformation field according to this application;

[0017] Figure 2e This is a schematic diagram of a process of an embodiment of the image correction method of this application;

[0018] Figure 2f This is a schematic diagram illustrating the effect of an embodiment of image correction according to this application;

[0019] Figure 2g This is a schematic diagram illustrating the effect of another embodiment of image correction in this application;

[0020] Figure 2h This is a schematic diagram illustrating the effect of yet another embodiment of image correction in this application;

[0021] Figure 2i This is a schematic diagram illustrating the effect of yet another embodiment of image correction in this application;

[0022] Figure 2j This is a schematic diagram illustrating the effect of yet another embodiment of image correction in this application;

[0023] Figure 2k This is a schematic diagram illustrating the effect of yet another embodiment of image correction in this application;

[0024] Figure 3 This is a flowchart illustrating an embodiment of the homework correction method of this application;

[0025] Figure 4 This is a schematic diagram of the frame of an embodiment of the image correction device of this application;

[0026] Figure 5 This is a schematic diagram of the framework of an embodiment of the work correction device of this application;

[0027] Figure 6 This is a schematic diagram of the framework of an embodiment of the electronic device of this application;

[0028] Figure 7 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0029] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0030] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0031] In this paper, the terms "system" and "network" are often used interchangeably. The term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the slash " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this paper indicates two or more objects.

[0032] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the image correction method of this application. Specifically, it may include the following steps:

[0033] Step S11: Extract image features of the image to be corrected.

[0034] In one implementation scenario, the image to be corrected can be a photograph of a paper document. Furthermore, the specific content of the image to be corrected can vary depending on the application scenario. For example, in an educational application scenario, the image to be corrected could be a photograph of a test paper, exercise book, or other paper document; or, in a medical application scenario, it could be a photograph of a medical record, examination report, or other paper document; or, in an industrial application scenario, it could be a photograph of an instruction manual, examination report, or other paper document. Of course, the above examples are merely a few possible cases of the image to be corrected in practical applications, and the specific content of the image to be corrected is not limited here, nor will it be listed in detail.

[0035] In one implementation scenario, when extracting features from the image to be corrected, feature extraction models such as convolutional neural networks can be used to extract features from the image to be corrected, thereby obtaining the image features of the image to be corrected. It should be noted that when the feature extraction model is built based on a convolutional neural network, there is no limit to the number of convolutional layers in the feature extraction model.

[0036] In another implementation scenario, please refer to Figure 2a , Figure 2a This is a schematic diagram illustrating an embodiment of feature extraction of the image to be corrected according to this application. Figure 2a As shown, unlike the aforementioned implementation methods, to minimize the computational complexity of subsequent global feature modeling and other processes, when extracting features from the image to be corrected, preliminary dimensionality reduction can be performed on the image to be corrected to obtain initial features. Then, downsampling is performed on these initial features to obtain features to be processed. Next, feature extraction is performed on these features using a cross-stage local network to obtain a first feature. Further processing of the first feature using a fusion attention mechanism yields a second feature, which can include channel attention and spatial attention. This allows for feature extraction of the second feature using a cross-stage local network to obtain a third feature. Finally, spatial pyramid pooling is performed on the third feature to obtain the image features. This approach, by sequentially performing preliminary dimensionality reduction, downsampling, feature extraction based on a cross-stage local network, fusion attention mechanism, feature extraction based on a cross-stage local network, and spatial pyramid pooling, helps to reduce the computational complexity of subsequent global feature modeling and other processes.

[0037] In a specific implementation scenario, for ease of description, the image to be corrected can be denoted as H*W*C, where H represents the height, W represents the width, and C represents the number of channels. For example, for an RGB image, the number of channels C can be 3. Furthermore, the resolution H*W of the image to be corrected can be set according to actual application needs, such as 496*496, etc. The resolution of the image to be corrected is not limited here, nor will it be listed in detail. It should be noted that the paper document in the image to be corrected may have structural distortions caused by shooting angle, paper bending, or other deformation factors. In this case, the corrected image obtained by the image correction process in this embodiment can remove the aforementioned structural distortions. Of course, the image to be corrected may also not have structural distortions. In this case, the corrected image obtained by the image correction process in this embodiment can be essentially the image to be corrected itself.

[0038] In a specific implementation scenario, such as Figure 2a As shown, initial dimensionality reduction can be achieved through convolution operations. For example, a 5x5 convolution kernel can be used to perform feature transformation on the image to be corrected, thereby achieving initial dimensionality reduction while preserving key information.

[0039] In a specific implementation scenario, such as Figure 2a As shown, after obtaining the initial features, it is possible to base the analysis on a 2x downsampling factor (i.e., ... Figure 2a In step s=2), the initial features are downsampled to obtain the features to be processed. Of course, the above example is only one possible example of the downsampling factor. Other possible cases are not limited here. The downsampling factor can also be set to 4, 8, etc., but we will not list all the downsampling factors here.

[0040] In a specific implementation scenario, such as Figure 2a As shown, after obtaining the features to be processed, the cross-stage partial network can be used to extract features from these features to obtain the first feature. It should be noted that the cross-stage partial network fuses feature branches at different levels, improving gradient fluidity while reducing computational redundancy and enhancing feature representation capabilities. Furthermore, the specific process of feature extraction using the cross-stage partial network can be found in the technical details of the cross-stage partial (CSP) network, and will not be elaborated upon here.

[0041] In a specific implementation scenario, such as Figure 2aAs shown, after obtaining the first feature, it can be further processed based on the fusion attention mechanism to obtain the second feature. It should be noted that combining channel attention and spatial attention enables the network to focus more on key regions of the image, improving its ability to perceive complex deformed regions. Furthermore, the specific process of the fusion attention mechanism can be found in the technical details of spatial attention and channel attention, which will not be elaborated upon here.

[0042] In a specific implementation scenario, such as Figure 2a As shown, after processing the first feature based on the fusion attention mechanism to obtain the second feature, and before performing feature extraction on the second feature based on the cross-stage local network, it is possible to first check whether the total number of downsampling operations does not exceed a preset number. It should be noted that the preset number can be set according to the actual application needs. For example, if it is necessary to fully perceive the deformed region, the preset number can be set appropriately larger; or, if it is necessary to appropriately reduce the computational load of the feature extraction stage, the preset number can be set appropriately smaller. Figure 2a As shown, for ease of description, the preset number of iterations can be denoted as N. Based on this, in response to the total number of downsampling iterations not exceeding the preset number N, a second feature can be selected as the new initial feature. For example, the latest second feature can be selected as the new initial feature. Further, for the new initial feature, the aforementioned step of downsampling based on the initial feature to obtain the feature to be processed can be repeated until the total number of downsampling iterations equals the preset number. At this point, feature extraction based on the cross-stage partial (CSP) network can be performed on the latest second feature to obtain the third feature and subsequent steps, thereby obtaining the image features of the image to be corrected. It should be noted that the specific process of feature extraction using the cross-stage partial (CSP) network can be found in the technical details of the cross-stage partial (CSP) network, and will not be elaborated here. The above method, after processing the first feature based on the fusion attention mechanism to obtain the second feature, and before extracting the second feature based on the cross-stage local network, checks whether the total number of downsampling operations is not higher than a preset number. If it is not higher than the preset number, the second feature is selected as the new initial feature, and the aforementioned downsampling steps based on the initial feature are returned to iterate for the new initial feature until the total number of downsampling operations equals the preset number. Therefore, the perception of deformed structures can be significantly improved through multiple iterations.

[0043] In a specific implementation scenario, please refer to the relevant documents. Figure 2aAfter obtaining the third feature, spatial pyramid pooling is performed based on the third feature to obtain the image features of the image to be corrected. It should be noted that by using pooling operations at different scales, the model's ability to understand global image information can be enhanced, ensuring that the model can effectively cope with deformations in complex scenes as much as possible. In addition, for the specific process of spatial pyramid pooling, please refer to the technical details of Spatial Pyramid Pooling (SPP), which will not be elaborated here.

[0044] Step S12: Model the dependencies between pixels in the image to be corrected based on image features to obtain the global features of the image to be corrected.

[0045] In one implementation scenario, as a possible example, to model the dependencies between pixels in the image to be corrected, sub-features at various locations in the image features can be processed based on an attention mechanism to model the dependencies between these locations and obtain global features. It should be noted that since each location in the image features corresponds to a different pixel region in the image to be corrected, the dependencies between pixels in the image to be corrected can be modeled accordingly. For ease of understanding, the image features can be denoted as H'*W'*C', and the sub-features at each location can be denoted as 1*1*C', meaning there are a total of H'*W' sub-features at different locations. Based on this, the H'*W' sub-features at different locations in the image features can be processed using attention mechanisms such as self-attention to obtain global features. Of course, the specific process of processing sub-features at various locations in the image features based on the attention mechanism can be found in the technical details of attention mechanisms such as self-attention, and will not be elaborated here.

[0046] In another implementation scenario, please refer to Figure 2b , Figure 2b This is a schematic diagram illustrating an embodiment of global feature extraction in this application. For example... Figure 2b As shown, unlike the aforementioned implementation, as another possible example, to model the dependencies between pixels in the image to be corrected, positional encoding can be performed based on image features to obtain a fourth feature. This fourth feature is then processed using a multi-head self-attention mechanism to obtain a fifth feature. A nonlinear transformation is then performed on the fifth feature to obtain a sixth feature. Finally, the fourth and sixth features can be fused to obtain the global feature. This approach, by sequentially executing positional encoding, multi-head self-attention, nonlinear transformation, and feature fusion, can effectively model the dependencies between pixels in the image to be corrected, facilitating a better understanding of global image deformation. It is particularly suitable for processing large-scale distortions or perspective distortions in images.

[0047] In a specific implementation scenario, such as Figure 2bAs shown, before positional encoding, channel transformation of image features can be performed. Understandably, since subsequent processing steps such as multi-head self-attention mechanisms typically require a fixed number of channels, the number of channels in the image features can be adjusted using methods such as 1x1 convolutional layers to suit the subsequent computational needs. For example, if the required number of channels in subsequent multi-head self-attention mechanism steps is K (e.g., 16, 32, 64), the number of channels in the image features can be adjusted using 1x1 convolutions to also adjust it to K.

[0048] In a specific implementation scenario, taking the image feature representation as H'*W'*C' as an example, the sub-features at each position can be denoted as 1*1*C', meaning there are a total of H'*W' sub-features at different positions. Based on this, positional encoding can be performed on each sub-feature to ensure that subsequent self-attention and other processing steps can understand the spatial relationships between sub-features. It should be noted that the specific methods of positional encoding can be found in the technical details of Position Encoding (PE), which will not be elaborated upon here.

[0049] In a specific implementation scenario, after obtaining the fourth feature through location encoding, the fourth feature can be processed based on a multi-head self-attention mechanism to obtain the fifth feature. Specifically, the fourth feature can be divided according to the number of attention heads, resulting in sub-features to be processed by each attention head. Based on this, during the processing of each sub-feature by the attention heads, feature projection can be performed on the sub-feature using query projection parameters, key projection parameters, and value projection parameters respectively, resulting in query features, key features, and value features. Then, based on the query features and key features, attention scores are obtained to establish long-distance dependencies between different regions of the image. The value features are then weighted using the attention scores to obtain the output features of the attention heads. Finally, the output features of each attention head can be fused to obtain the fifth feature of the multi-head self-attention mechanism. Of course, the above description is merely an exemplary description of the multi-head self-attention mechanism; for specific details, please refer to the technical details of the multi-head self-attention mechanism, which will not be elaborated upon here.

[0050] In a specific implementation scenario, after obtaining the fifth feature, a nonlinear transformation can be performed on the fifth feature to obtain the sixth feature. For example, a feedforward network can be used to perform a nonlinear transformation on the fifth feature to obtain the sixth feature. Of course, the above example is merely one possible illustration of nonlinear transformation; other possible implementations are not limited here, nor will they be listed in detail.

[0051] In a specific implementation scenario, after obtaining the sixth feature, the fourth and sixth features can be fused to obtain the global feature. For example, the fourth and sixth features can be concatenated to obtain the global feature. Of course, the above example is merely one possible implementation of fusing the fourth and sixth features; other possible implementations are not limited here, nor will they be listed in detail.

[0052] Step 13: Enhance local features based on global features to obtain enhanced features.

[0053] In one implementation scenario, a feature pyramid network can be used to process global features to enhance local features, thereby obtaining enhanced features. It's important to note that the feature pyramid network fuses high-level semantic features with low-level detail features through a top-down path, allowing shallow feature maps to acquire stronger semantic information while deeper feature maps retain more local details. For the specific process of local feature enhancement, please refer to the technical details of feature pyramids; it will not be elaborated upon here.

[0054] In another implementation scenario, please refer to [the relevant documentation]. Figure 2c , Figure 2c This is a schematic diagram illustrating a process of local feature enhancement according to an embodiment of this application. For example... Figure 2c As shown, unlike the aforementioned implementation, as another possible example, global features can be convolved separately using several dilated convolutions to obtain the output features of each dilated convolution, with each dilated convolution having a different dilation coefficient. Based on this, the output features of each dilated convolution can be fused to obtain enhanced features. This method, by performing convolution processing on global features separately using dilated convolutions with different dilation coefficients and then fusing the processed features, can more accurately recover image details, especially in areas of local deformation.

[0055] In a specific implementation scenario, the set of dilation coefficients for several dilated convolutions can encompass at least two dilation coefficients from 1, 2, 3, 4, and 5 to restore local details in an image, effectively expanding the receptive field and capturing local information from different scales, making it particularly suitable for handling local deformations. For example, there can be a total of 5 dilated convolutions, and the dilation coefficients for each dilated convolution can be 1, 2, 3, 4, and 5 respectively. Of course, the above example is merely one possible illustration of dilated convolutions and their dilation coefficients in practical applications; other possible scenarios are not limited here, nor will they be listed in detail.

[0056] In a specific implementation scenario, please refer to the relevant documents. Figure 2cAfter obtaining the output features of each dilated convolution, the output features of each dilated convolution can be concatenated and then subjected to feature dimensionality reduction through a convolutional layer such as 5*5 to obtain enhanced features.

[0057] Step S14: Make predictions based on the enhanced features to obtain the deformation field.

[0058] Specifically, after obtaining the enhanced features, predictions can be made based on them to obtain the deformation field. For example, the enhanced features can be decoded using a decoder to obtain the deformation field. It should be noted that each element in the deformation field describes the displacement of pixels in the image (e.g., offsets in the X and Y directions). For instance, still taking the image to be corrected as H*W*C, as mentioned earlier, the size of the feature tensors such as image features, global features, and local features can be represented as H'*W'*C'. In this case, the output size of the deformation field can be represented as 2*H'*W', meaning that each feature position can correspond to an offset of size 2 (offsets in the X and Y directions, respectively). Furthermore, the decoder can include, but is not limited to, network structures such as Transformer; the network structure of the decoder is not limited here.

[0059] Step S15: Correct the image to be corrected based on the deformation field to obtain the corrected image.

[0060] In one implementation scenario, the deformed field can be upsampled to the resolution of the image to be corrected, and then the upsampled deformed field can be used to correct the image to obtain the corrected image. Please refer to [reference needed]. Figure 2d , Figure 2d This is a schematic diagram illustrating the effect of an embodiment of image correction based on deformation field according to this application. Figure 2d As shown, Figure 2d The three images represent the image to be corrected, control points based on the deformation field (shown in red), and the corrected image, respectively. Taking a resolution of 496*496 for the image to be corrected as an example, if the resolution of the deformation field is 31*31, then each 16*16 pixel region in the image to be corrected can correspond to a pixel (control point) in the deformation field. Therefore, upsampling can be performed based on the deformation field; for example, in this case, it can be upsampled by a factor of 16 to obtain a 496*496 deformation field. This means that each pixel in the image to be corrected can correspond to an offset of size 2 (offsets in the X and Y directions, respectively). The image to be corrected can then be corrected based on the upsampled deformation field to obtain the corrected image, thus restoring the image's geometry.

[0061] In one implementation scenario, please refer to the following: Figure 2e , Figure 2e This is a schematic diagram illustrating the process of an embodiment of the image correction method of this application. Figure 2eAs shown, the image to be corrected is first processed through image feature extraction to obtain image features. These image features are then modeled using global features to obtain global features. The global features are then enhanced using local features to obtain enhanced features. These enhanced features are further predicted using a deformation field to obtain a deformation field. Finally, the deformation field is applied to the image to be corrected to obtain the corrected image. Please refer to the following documentation. Figure 2f , Figure 2f This is a schematic diagram illustrating the effect of an embodiment of image correction according to this application. For example... Figure 2f As shown, Figure 2f The left image is the image to be corrected, and the right image is the corrected image after image correction according to the embodiments of this disclosure. For images to be corrected that have perspective distortion, after image correction according to the embodiments of this disclosure, perspective distortion can be corrected, and the overall global and local details of the paper document in the image to be corrected can be highlighted. Please refer to the following: Figure 2g , Figure 2g This is a schematic diagram illustrating the effect of another embodiment of image correction in this application. For example... Figure 2g As shown, Figure 2g The left image is the image to be corrected, and the right image is the corrected image after image correction according to the embodiments of this disclosure. For images to be corrected that exhibit curling deformation when turning pages in a book, image correction according to the embodiments of this disclosure can correct the curling deformation and highlight both the overall picture and local details of the paper document in the image to be corrected. Please refer to... Figure 2h , Figure 2h This is a schematic diagram illustrating the effect of yet another embodiment of image correction in this application. For example... Figure 2h As shown, Figure 2h The left image is the image to be corrected, and the right image is the corrected image after image correction according to the embodiments of this disclosure. For the image to be corrected that exhibits curling deformation when viewed from the front, after image correction according to the embodiments of this disclosure, the curling deformation can be corrected, and the overall global and local details of the paper document in the image to be corrected can be highlighted. Please refer to... Figure 2i , Figure 2i This is a schematic diagram illustrating the effect of yet another embodiment of image correction in this application. For example... Figure 2i As shown, Figure 2i The left image is the image to be corrected, and the right image is the corrected image after image correction according to the embodiments of this disclosure. For the image to be corrected that exhibits curling deformation from a top-down view, after image correction according to the embodiments of this disclosure, the curling deformation can be corrected, and the overall global and local details of the paper document in the image to be corrected can be highlighted. Please refer to... Figure 2j , Figure 2j This is a schematic diagram illustrating the effect of yet another embodiment of image correction in this application. For example... Figure 2j As shown, Figure 2j The left image is the image to be corrected, and the right image is the corrected image after image correction according to the embodiments of this disclosure. For the image to be corrected that is curled under a folded view, after image correction according to the embodiments of this disclosure, on the one hand, the curling deformation can be corrected, and on the other hand, the overall global and local details of the paper document in the image to be corrected can be highlighted. Please refer to... Figure 2k , Figure 2k This is a schematic diagram illustrating the effect of yet another embodiment of image correction in this application. For example... Figure 2k As shown, Figure 2k The left image is the image to be corrected, and the right image is the corrected image after image correction according to the embodiments of this disclosure. For images to be corrected that are curled due to crumpling, after image correction according to the embodiments of this disclosure, on the one hand, the curling deformation can be corrected, and on the other hand, the overall global and local details of the paper document in the image to be corrected can be highlighted. The above examples demonstrate that the embodiments of this disclosure can effectively correct various complex deformations. Of course, Figures 2f to 2g The examples shown are merely a few possible examples of the image to be corrected and the corrected image in practical applications. Other possible situations are not limited here, nor will they be listed one by one.

[0062] The above scheme extracts image features from the image to be corrected, models the dependencies between pixels in the image based on these features to obtain global features, and then enhances local features based on these global features to obtain enhanced features. Based on these enhanced features, a deformation field is predicted, and finally, the image to be corrected is corrected based on this deformation field to obtain the corrected image. Therefore, on the one hand, by sequentially extracting global features and enhancing local features, it is possible to further enhance local details while capturing the global deformation pattern of the image, which helps improve the balance of image correction. On the other hand, because global features are extracted to capture the global deformation pattern during image correction, compared with traditional geometry-based methods, it can handle not only linear deformations but is also more conducive to handling nonlinear deformations, thus improving applicability to different deformations. Therefore, it can improve the balance of image correction and its applicability to different deformations.

[0063] Please see Figure 3 , Figure 3 This is a flowchart illustrating an embodiment of the homework correction method of this application. Specifically, it may include the following steps:

[0064] Step S31: Obtain the work image as the image to be corrected.

[0065] In one implementation scenario, the images of assignments can be obtained by taking pictures of paper documents (such as test papers, exercise books, etc.) with electronic devices such as smartphones and learning machines, and then the electronic devices can directly use them as images to be corrected for subsequent processing.

[0066] In another implementation scenario, homework images can also be obtained by taking pictures of paper documents (such as test papers, exercise books, etc.) with electronic devices such as smartphones and learning machines, and then uploaded to the homework grading system for further processing.

[0067] Step S32: Perform correction processing on the image to be corrected to obtain the corrected image.

[0068] In this embodiment of the disclosure, the corrected image is obtained by the process steps in the above-described image correction method embodiments. For details, please refer to the aforementioned image correction method embodiments, which will not be repeated here.

[0069] Step S33: Recognize based on the corrected image to obtain job data.

[0070] Specifically, recognition technologies such as OCR (Optical Character Recognition) can be used to recognize the corrected image to obtain the job data. It should be noted that since the job image has already undergone correction processing as the image to be corrected in the aforementioned steps, the corrected image has corrected as much as possible any possible deformed structures in the original image to be corrected and highlighted the overall and local details of the document. Therefore, the accuracy of image recognition can be improved as much as possible.

[0071] Step S34: Grade the work based on the work data to obtain the grading results.

[0072] Specifically, the standard answers to the assignment data can be obtained in advance, and then semantic understanding can be performed using pre-trained language models such as BERT to provide grading results based on the assignment data and its standard answers. Alternatively, a large language model can be used to grade the assignment data. For example, prompts can be constructed based on the assignment data, and these prompts can instruct the large language model to grade the assignment data. The output of the large language model in response to the prompts can then be obtained as the grading result of the assignment data. Of course, the above examples are only a few possible implementations for grading assignment data in practical applications. The grading methods for assignment data are not limited here, nor will they be listed in detail.

[0073] The above scheme acquires a work image as the image to be corrected, performs correction processing on the image to be corrected to obtain a corrected image, and obtains the corrected image through the process steps in the above image correction method embodiment. The corrected image is then recognized with the image to obtain work data, and then graded based on the work data to obtain the graded result. Since the corrected image is obtained through the process steps in the above image correction method embodiment, it can improve the uniformity of image correction and its applicability to different deformations. Based on this, image recognition can improve the accuracy of work data, and further, work grading based on this can also improve the accuracy of work grading.

[0074] Please see Figure 4 , Figure 4 This is a schematic diagram of the framework of an embodiment of the image correction device of this application. The image correction device 40 includes: a feature extraction module 41, a global modeling module 42, a local enhancement module 43, a deformation prediction module 44, and a correction processing module 45. The feature extraction module 41 is used to extract image features of the image to be corrected; the global modeling module 42 is used to model the dependency relationship between pixels in the image to be corrected based on the image features to obtain the global features of the image to be corrected; the local enhancement module 43 is used to enhance local features based on the global features to obtain enhanced features; the deformation prediction module 44 is used to predict based on the enhanced features to obtain a deformation field; and the correction processing module 45 is used to correct the image to be corrected based on the deformation field to obtain a corrected image.

[0075] The above scheme extracts image features from the image to be corrected, models the dependencies between pixels in the image based on these features to obtain global features, and then enhances local features based on these global features to obtain enhanced features. Based on these enhanced features, a deformation field is predicted, and finally, the image to be corrected is corrected based on this deformation field to obtain the corrected image. Therefore, on the one hand, by sequentially extracting global features and enhancing local features, it is possible to further enhance local details while capturing the global deformation pattern of the image, which helps improve the balance of image correction. On the other hand, because global features are extracted to capture the global deformation pattern during image correction, compared with traditional geometry-based methods, it can handle not only linear deformations but is also more conducive to handling nonlinear deformations, thus improving applicability to different deformations. Therefore, it can improve the balance of image correction and its applicability to different deformations.

[0076] In some disclosed embodiments, the feature extraction module 41 includes a preliminary dimensionality reduction submodule for performing preliminary dimensionality reduction based on the image to be corrected to obtain initial features of the image to be corrected; the feature extraction module 41 includes a downsampling submodule for downsampling based on the initial features to obtain features to be processed; the feature extraction module 41 includes a first extraction submodule for extracting features from the features to be processed based on a cross-stage local network to obtain a first feature; the feature extraction module 41 includes a fusion attention submodule for processing the first feature based on a fusion attention mechanism to obtain a second feature; wherein the fusion attention mechanism includes channel attention and spatial attention; the feature extraction module 41 includes a second extraction submodule for extracting features from the second feature based on a cross-stage local network to obtain a third feature; the feature extraction module 41 includes a pyramid pooling submodule for performing spatial pyramid pooling based on the third feature to obtain image features.

[0077] In some disclosed embodiments, the feature extraction module 41 includes a count detection submodule, used to detect whether the total number of downsampling executions is not higher than a preset number; the feature extraction module 41 includes a loop iteration submodule, used to select a second feature as a new initial feature in response to the total number of downsampling executions not being higher than the preset number, and return to the step of downsampling based on the initial feature to obtain the feature to be processed, until the total number of downsampling executions is not higher than the preset number.

[0078] In some disclosed embodiments, the global modeling module 42 includes a position encoding submodule for performing position encoding based on image features to obtain a fourth feature; the global modeling module 42 includes a multi-head self-attention submodule for processing the fourth feature based on a multi-head self-attention mechanism to obtain a fifth feature; the global modeling module 42 includes a nonlinear transformation submodule for performing a nonlinear transformation based on the fifth feature to obtain a sixth feature; and the global modeling module 42 includes a residual connection submodule for fusing the fourth and sixth features to obtain a global feature.

[0079] In some disclosed embodiments, the local enhancement module 43 includes a dilated convolution submodule, which is used to perform convolution processing on the global features based on several dilated convolutions to obtain the output features of the several dilated convolutions respectively; wherein the several dilated convolutions have different dilation coefficients; the local enhancement module 43 includes a feature fusion submodule, which is used to fuse the output features of the several dilated convolutions respectively to obtain enhanced features.

[0080] In some disclosed embodiments, the set of hole coefficients for each of the plurality of dilated convolutions covers at least two of the hole coefficients in 1, 2, 3, 4, and 5.

[0081] In some disclosed embodiments, the correction processing module 45 includes an upsampling submodule for upsampling the image to be corrected based on the deformation field to the resolution of the image to be corrected; the correction processing module 45 includes a correction submodule for correcting the image to be corrected based on the upsampled deformation field to obtain a corrected image.

[0082] Please see Figure 5 , Figure 5 This is a schematic diagram of the framework of an embodiment of the work grading device of this application. The work grading device 50 includes: an image acquisition module 51, an image correction module 52, an image recognition module 53, and a data processing module 54. The image acquisition module 51 is used to acquire a work image as an image to be corrected; the image correction module 52 is used to perform correction processing based on the image to be corrected to obtain a corrected image; wherein, the corrected image is obtained by the above-mentioned image correction device; the image recognition module 53 is used to perform recognition based on the corrected image to obtain work data; the data processing module 54 is used to grade based on the work data to obtain a grading result.

[0083] In the above scheme, the job correction device 50 acquires a job image as an image to be corrected, performs correction processing on the image to be corrected to obtain a corrected image, and the corrected image is obtained by the above-mentioned image correction device, thereby recognizing the corrected image to obtain job data, and then correcting the job data to obtain a correction result. Since the corrected image is obtained by the process steps in the above-mentioned image correction method embodiment, the uniformity of image correction and its applicability to different deformations can be improved. Based on this, image recognition can improve the accuracy of job data, and further job correction based on this can also improve the accuracy of job correction.

[0084] Please see Figure 6 , Figure 6 This is a schematic diagram of a framework of an embodiment of the electronic device of this application. The electronic device 60 includes at least a memory 61 and a processor 62 coupled to each other. The memory 61 stores at least program instructions, and the processor 62 is used to execute the program instructions to implement the steps in any of the above-described image correction method embodiments, or to implement the steps in any of the above-described job correction method embodiments. For details, please refer to the foregoing disclosed embodiments, which will not be repeated here. As a possible example, the electronic device 60 may include, but is not limited to, devices such as mobile phones, tablet computers, learning machines, smart screens, and servers. The specific type of the electronic device 60 is not limited here.

[0085] Specifically, processor 62 controls itself and memory 61 to implement the steps in any of the above-described image correction method embodiments or the steps in any of the above-described job grading method embodiments. Processor 62 can also be referred to as a CPU (Central Processing Unit). Processor 62 may be an integrated circuit chip with signal processing capabilities. Processor 62 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 62 can be implemented using integrated circuit chips.

[0086] The above scheme involves the electronic device 60 extracting image features from the image to be corrected, modeling the dependencies between pixels in the image based on these features to obtain global features, and then enhancing local features based on these global features to obtain enhanced features. Based on these enhanced features, a deformation field is predicted, and finally, the image to be corrected is corrected based on the deformation field to obtain the corrected image. Therefore, by sequentially extracting global features and enhancing local features, local details can be further enhanced while capturing the global deformation pattern of the image, which helps improve the balance of image correction. Furthermore, because global features are extracted to capture the global deformation pattern during image correction, compared to traditional geometry-based methods, it can handle not only linear deformations but is also more conducive to handling nonlinear deformations, thus improving applicability to different deformations. Therefore, it can improve the balance of image correction and its applicability to different deformations. Furthermore, the work image is acquired as the image to be corrected, and correction processing is performed on the image to be corrected to obtain the corrected image. The corrected image is obtained by the process steps in the above-described image correction method embodiment. The corrected image is then recognized to obtain work data, and the work data is then graded to obtain the grading result. Since the corrected image is obtained by the process steps in the above-described image correction method embodiment, the uniformity of image correction and its applicability to different deformations can be improved. Based on this, image recognition can improve the accuracy of work data, and based on this, work grading can also improve the accuracy of work grading.

[0087] Please see Figure 7 , Figure 7This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application. The computer-readable storage medium 70 stores program instructions 71 that can be executed by a processor. The program instructions 71 are used to implement the steps in any of the above-described image correction method embodiments or the steps in any of the above-described job correction method embodiments.

[0088] The above scheme involves a computer-readable storage medium 70 extracting image features from the image to be corrected, modeling the dependencies between pixels in the image based on these features to obtain global features, and then enhancing local features based on these global features to obtain enhanced features. Based on these enhanced features, a deformation field is predicted, and finally, the image to be corrected is corrected based on the deformation field to obtain the corrected image. Therefore, by sequentially extracting global features and enhancing local features, local details can be further enhanced while capturing the global deformation pattern of the image, which helps improve the balance of image correction. Furthermore, because global features are extracted to capture the global deformation pattern during image correction, compared to traditional geometry-based methods, it can handle not only linear deformations but is also more conducive to handling nonlinear deformations, thus improving applicability to different deformations. Therefore, it can improve the balance of image correction and its applicability to different deformations. Furthermore, the work image is acquired as the image to be corrected, and correction processing is performed on the image to be corrected to obtain the corrected image. The corrected image is obtained by the process steps in the above-described image correction method embodiment. The corrected image is then recognized to obtain work data, and the work data is then graded to obtain the grading result. Since the corrected image is obtained by the process steps in the above-described image correction method embodiment, the uniformity of image correction and its applicability to different deformations can be improved. Based on this, image recognition can improve the accuracy of work data, and based on this, work grading can also improve the accuracy of work grading.

[0089] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0090] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0091] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0092] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0093] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0094] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0095] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

Claims

1. An image correction method, characterized in that, include: Extract image features from the image to be corrected; Based on the image features, the dependencies between pixels in the image to be corrected are modeled to obtain the global features of the image to be corrected; Based on the global features, local feature enhancement is performed to obtain enhanced features; Based on the enhanced features, a prediction is made to obtain the deformation field; The image to be corrected is corrected based on the deformation field to obtain a corrected image; wherein, the extraction of image features from the image to be corrected includes: Preliminary dimensionality reduction is performed on the image to be corrected to obtain the initial features of the image to be corrected; Based on the initial features, downsampling is performed to obtain the features to be processed; The first feature is obtained by extracting features from the features to be processed based on a cross-stage local network. The first feature is processed based on a fusion attention mechanism to obtain the second feature; wherein, the fusion attention mechanism includes channel attention and spatial attention; The total number of times the downsampling has been performed is not higher than a preset number; wherein the preset number is positively correlated with the sufficiency of the deformation region to be sensed. In response to the fact that the total number of downsampling operations is not higher than the preset number, the second feature is selected as the new initial feature, and the step of downsampling based on the initial feature to obtain the feature to be processed is returned for the new initial feature, until the total number of downsampling operations is equal to the preset number, then the process jumps to extract the latest second feature based on the cross-stage local network to obtain the third feature, and spatial pyramid pooling is performed based on the third feature to obtain the image feature.

2. The method according to claim 1, characterized in that, The step of modeling the dependencies between pixels in the image to be corrected based on the image features to obtain the global features of the image to be corrected includes: Based on the image features, position encoding is performed to obtain the fourth feature; The fifth feature is obtained by processing the fourth feature based on the multi-head self-attention mechanism; Based on the fifth feature, a nonlinear transformation is performed to obtain the sixth feature; The global feature is obtained by fusing the fourth feature and the sixth feature.

3. The method according to claim 1, characterized in that, The process of enhancing local features based on the global features to obtain enhanced features includes: The global features are processed by several dilated convolutions to obtain the output features of each dilated convolution; wherein the several dilated convolutions have different dilation coefficients. The enhanced features are obtained by fusing the output features of the several dilated convolutions.

4. The method according to claim 3, characterized in that, The set of dilation coefficients for each of the plurality of dilated convolutions covers at least two of the dilation coefficients in 1, 2, 3, 4, and 5.

5. The method according to claim 1, characterized in that, The step of correcting the image to be corrected based on the deformation field to obtain a corrected image includes: Upsample to the resolution of the image to be corrected based on the deformation field; The image to be corrected is corrected based on the deformation field after upsampling to obtain the corrected image.

6. A method for grading homework, characterized in that, include: Obtain the image of the work as the image to be corrected; The image to be corrected is subjected to correction processing to obtain a corrected image; wherein the corrected image is obtained by the image correction method according to any one of claims 1 to 5; Based on the corrected image, the operation data is obtained; The work data is graded to obtain the grading results.

7. An image correction device, characterized in that, include: The feature extraction module is used to extract image features from the image to be corrected. A global modeling module is used to model the dependencies between pixels in the image to be corrected based on the image features, so as to obtain the global features of the image to be corrected. The local enhancement module is used to enhance local features based on the global features to obtain enhanced features; The deformation prediction module is used to predict the deformation field based on the enhanced features; The correction processing module is used to correct the image to be corrected based on the deformation field to obtain a corrected image; wherein, the extraction of image features from the image to be corrected includes: Preliminary dimensionality reduction is performed on the image to be corrected to obtain the initial features of the image to be corrected; Based on the initial features, downsampling is performed to obtain the features to be processed; The first feature is obtained by extracting features from the features to be processed based on a cross-stage local network. The first feature is processed based on a fusion attention mechanism to obtain the second feature; wherein, the fusion attention mechanism includes channel attention and spatial attention; The total number of times the downsampling has been performed is not higher than a preset number; wherein the preset number is positively correlated with the sufficiency of the deformation region to be sensed. In response to the fact that the total number of downsampling operations is not higher than the preset number, the second feature is selected as the new initial feature, and the step of downsampling based on the initial feature to obtain the feature to be processed is returned for the new initial feature, until the total number of downsampling operations is equal to the preset number, then the process jumps to extract the latest second feature based on the cross-stage local network to obtain the third feature, and spatial pyramid pooling is performed based on the third feature to obtain the image feature.

8. A homework correction device, characterized in that, include: The image acquisition module is used to acquire the work image as the image to be corrected. An image correction module is used to perform correction processing on the image to be corrected to obtain a corrected image; wherein the corrected image is obtained by the image correction device according to claim 7; The image recognition module is used to perform recognition based on the corrected image to obtain job data; The data processing module is used to revise the work data and obtain the revision results.

9. An electronic device, characterized in that, It includes at least a memory and a processor, wherein the memory stores at least program instructions, and the processor is used to execute the program instructions to implement the image correction method according to any one of claims 1 to 5, or the job correction method according to claim 6.

10. A computer-readable storage medium, characterized in that, The device stores program instructions that can be executed by a processor, the program instructions being used to implement the image correction method according to any one of claims 1 to 5, or the job correction method according to claim 6.

Images