Test paper image processing method, device, storage medium and electronic equipment

Abstract

The present disclosure relates to a test paper image processing method, device, storage medium and electronic device. The method comprises: obtaining an original test paper image corresponding to a test paper image to be processed; the test paper image to be processed is an image after a user answers on the test paper corresponding to the original test paper image; obtaining optical flow information between the original test paper image and the test paper image to be processed; aligning the original test paper image and the test paper image to be processed according to the optical flow information; obtaining answer information corresponding to the test paper image to be processed according to the aligned original test paper image and the test paper image to be processed. In this way, the user's answer information can be obtained more accurately according to the aligned image, which improves the accuracy of answer information acquisition. At the same time, since there is no need to perform handwritten text recognition, the answer information can also be accurately obtained for scenes where both the questions and the answers are handwritten, which increases the application scenarios of test paper image processing and improves the accuracy of answer information acquisition.

Description

Technical Field

[0001] The present disclosure relates to the field of computers, and in particular, to a test paper image processing method, device, storage medium and electronic equipment. Background Art

[0002] With the development of computer and Internet technology, more and more people use electronic devices to mark students' test papers. For example, a user can convert the paper version of the test paper information into a test paper image by taking a photo or scanning the test paper, and upload it to an electronic device (such as a server or terminal device) to mark the test paper image. The test paper image uploaded by the user may include question information and answer information. For example, the question information may be the printout on the original test paper, and the answer information may be the handwriting of the user's answer based on the original test paper. In order to improve the efficiency of electronic marking, it is necessary to obtain the answer information in the test paper image uploaded by the user. In the related art, the acquisition of answer information is not accurate enough. Summary of the invention

[0003] The purpose of the present disclosure is to provide a test paper image processing method, device, storage medium and electronic device to partially solve the above-mentioned problems existing in the related art.

[0004] In order to achieve the above-mentioned object, the first aspect of the present disclosure provides a test paper image processing method, the method comprising:

[0005] Acquire an original test paper image corresponding to the test paper image to be processed; the test paper image to be processed is an image after the user answers the test paper corresponding to the original test paper image;

[0006] Obtaining optical flow information between the original test paper image and the test paper image to be processed;

[0007] Performing image alignment on the original test paper image and the test paper image to be processed according to the optical flow information;

[0008] According to the aligned original test paper image and the test paper image to be processed, answer information corresponding to the test paper image to be processed is obtained.

[0009] Optionally, the acquiring of optical flow information between the original test paper image and the test paper image to be processed includes:

[0010] Determine a target offset vector for each pixel point of the original test paper image and the test paper image to be processed;

[0011] Acquire the optical flow information according to the target offset vector;

[0012] The aligning the original test paper image and the test paper image to be processed according to the optical flow information comprises:

[0013] According to the target offset vector, the position of each pixel point of the original test paper image in the image is adjusted to align the original test paper image with the test paper image to be processed.

[0014] Optionally, determining a target offset vector of each pixel point of the original test paper image and the test paper image to be processed comprises:

[0015] According to a preset energy optimization function, a target offset vector of each pixel point of the original test paper image and the test paper image to be processed is calculated; the preset energy optimization function includes optical flow continuity constraint parameters that characterize the original test paper image and the test paper image to be processed.

[0016] Optionally, the preset energy optimization function also includes a brightness constraint parameter, a smoothness constraint parameter and a feature matching constraint parameter.

[0017] Optionally, acquiring answer information corresponding to the test paper image to be processed according to the aligned original test paper image and the test paper image to be processed includes:

[0018] Performing image binarization processing on the test paper image to be processed to obtain a binary image to be processed;

[0019] Performing image binarization processing on the original test paper image to obtain an original binary image;

[0020] Determine the answering area according to the binary image to be processed and the original binary image;

[0021] The answer information is determined according to the test paper image to be processed and the answer area.

[0022] Optionally, obtaining the original test paper image corresponding to the test paper image to be processed includes:

[0023] Obtaining an image identifier in the test paper image to be processed; the image identifier includes the test paper identifier corresponding to the test paper image to be processed and the page number of the test paper image to be processed;

[0024] According to the image identifier, an original test paper image of the original test paper corresponding to the test paper image to be processed is obtained from a preset test paper library.

[0025] Optionally, the test paper image to be processed can be obtained in the following manner:

[0026] Acquire, by means of a camera, a pending image after the user has answered questions on the test paper corresponding to the original test paper image;

[0027] Determine a plurality of preset key points of the pending image by a key point detection algorithm;

[0028] The test paper image to be processed is cropped and corrected according to the preset key points to obtain the test paper image to be processed.

[0029] In a second aspect, the present disclosure provides a test paper image processing device, the device comprising:

[0030] An image acquisition module is used to acquire an original test paper image corresponding to the test paper image to be processed; the test paper image to be processed is an image after the user answers the test paper corresponding to the original test paper image;

[0031] An information acquisition module, used for acquiring optical flow information between the original test paper image and the test paper image to be processed;

[0032] An image alignment module, used for aligning the original test paper image and the test paper image to be processed according to the optical flow information;

[0033] The image processing module is used to obtain the answer information corresponding to the test paper image to be processed based on the aligned original test paper image and the test paper image to be processed.

[0034] Optionally, the information acquisition module is used to determine a target offset vector of each pixel point of the original test paper image and the test paper image to be processed; and acquire the optical flow information according to the target offset vector;

[0035] The image alignment module is used to adjust the position of each pixel point of the original test paper image in the image according to the target offset vector, so as to align the original test paper image with the test paper image to be processed.

[0036] Optionally, the information acquisition module is used to calculate the target offset vector of each pixel point of the original test paper image and the test paper image to be processed according to a preset energy optimization function; the preset energy optimization function includes optical flow continuity constraint parameters that characterize the original test paper image and the test paper image to be processed.

[0037] Optionally, the preset energy optimization function also includes a brightness constraint parameter, a smoothness constraint parameter and a feature matching constraint parameter.

[0038] Optionally, the image processing module is used to perform image binarization processing on the test paper image to be processed to obtain a binary image to be processed; perform image binarization processing on the original test paper image to obtain an original binary image; determine the answer area based on the binary image to be processed and the original binary image; determine the answer information based on the test paper image to be processed and the answer area.

[0039] Optionally, the image acquisition module is used to obtain the image identification in the test paper image to be processed; the image identification includes the test paper identification corresponding to the test paper image to be processed and the page number of the test paper image to be processed; according to the image identification, the original test paper image of the original test paper corresponding to the test paper image to be processed is obtained from a preset test paper library.

[0040] Optionally, the image acquisition module is used to acquire, through a camera device, a pending image after the user answers the questions on the test paper corresponding to the original test paper image; determine multiple preset key points of the pending image through a key point detection algorithm; and perform cropping and correction on the pending test paper image according to the preset key points to obtain the pending test paper image.

[0041] In a third aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in the first aspect of the present disclosure.

[0042] In a fourth aspect, the present disclosure provides an electronic device, comprising: a memory on which a computer program is stored; and a processor for executing the computer program in the memory to implement the steps of the method described in the first aspect of the present disclosure.

[0043] By adopting the above technical solution, the original test paper image corresponding to the test paper image to be processed is obtained; the test paper image to be processed is the image after the user answers the test paper corresponding to the original test paper image; the optical flow information between the original test paper image and the test paper image to be processed is obtained; the original test paper image and the test paper image to be processed are aligned according to the optical flow information; the answer information corresponding to the test paper image to be processed is obtained according to the aligned original test paper image and the test paper image to be processed. In this way, the user's answer information can be obtained more accurately according to the aligned image, which improves the accuracy of answer information acquisition. At the same time, since there is no need to perform handwritten text recognition, the answer information can also be accurately obtained for scenes where both the questions and the answers are handwritten, which increases the application scenarios of test paper image processing and improves the accuracy of answer information acquisition.

[0044] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0046] Figure 1 It is a flow chart of a test paper image processing method provided by an embodiment of the present disclosure.

[0047] Figure 2It is a schematic diagram of an original binary image provided by an embodiment of the present disclosure.

[0048] Figure 3 It is a schematic diagram of a binary image to be processed provided by an embodiment of the present disclosure.

[0049] Figure 4 It is a schematic diagram of a new original binary image provided by an embodiment of the present disclosure after being aligned with a binary image to be processed.

[0050] Figure 5 It is a schematic diagram of a test paper image processing device provided in an embodiment of the present disclosure.

[0051] Figure 6 It is a block diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0052] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0053] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0054] In the present disclosure, words such as "first" and "second" are only used for the purpose of distinguishing descriptions and should not be understood as indicating or implying relative importance or order; terms such as "S101", "S102", "S201", "S202" and the like are used to distinguish steps and should not be understood as performing method steps in a specific order or sequential sequence; when the following description refers to drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements.

[0055] First, the application scenario of the present disclosure is explained. The present disclosure can be applied to the field of electronic teaching, especially in the scenario where an electronic device is used to analyze or mark an examination paper image obtained by taking a photo or scanning. In order to improve the efficiency of electronic marking, it is necessary to obtain the answer information in the examination paper image uploaded by the user, and analyze and mark the answer information. In the related art, the answer information can be obtained by handwriting recognition. For example, the handwritten text and printed text in the examination paper image uploaded by the user are identified, and the printed text is used as the original examination paper title, and the handwritten text is used as the user's answer information. However, this method cannot include all handwritten content, for example, it cannot recognize hand-drawn symbols, geometric patterns, etc. In addition, the questions of some examination papers can also be in the form of handwriting, and this method cannot be used in the scenario where the questions are handwritten.

[0056] In order to solve the above problems, the present disclosure provides a test paper image processing method, device, storage medium and electronic device, which align the original test paper image and the test paper image to be processed according to the optical flow information, and then obtain the answer information corresponding to the test paper image to be processed according to the aligned original test paper image and the test paper image to be processed. In this way, the user's answer information can be obtained more accurately, and the accuracy of answer information acquisition is improved. At the same time, since there is no need to perform handwritten text recognition, the answer information can also be accurately obtained for scenarios where both the questions and the answers are handwritten, which increases the application scenarios of test paper image processing and improves the accuracy of answer information acquisition.

[0057] The specific embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0058] Figure 1 A test paper image processing method provided by an embodiment of the present disclosure can be applied to electronic devices, which may include terminal devices such as smart phones, smart wearable devices, smart speakers, smart tablets, PDAs (Personal Digital Assistants), CPEs (Customer Premise Equipment), personal computers, etc., and may also include servers such as local servers or cloud servers. Figure 1 As shown, the method includes:

[0059] S101, obtaining an original test paper image corresponding to the test paper image to be processed.

[0060] The test paper image to be processed is an image after the user answers the test paper corresponding to the original test paper image.

[0061] In this step, there are many ways to obtain the original test paper image.

[0062] In some embodiments, the original test paper before the user answers can be photographed or scanned to obtain the test paper image before answering; after the user answers the original test paper, the test paper after answering can be photographed or scanned to obtain the test paper image to be processed. In this way, the test paper image before answering can be used as the original test paper image corresponding to the test paper image to be processed.

[0063] In other embodiments, the original test paper may include an image identifier. For example, the image identifier may be located at a preset position of the original test paper, and the preset position may be one or more of the positions above (e.g., header), below (e.g., footer), left or right of the test paper; the image identifier may include a test paper identifier and a page number, and the test paper identifier is used to represent the unique serial number of the original test paper, and the unique serial number may include one or more of letters, numbers and dates. For example, multiple test papers in a preset test paper library may be numbered in a preset order, so that each test paper is assigned a unique serial number, and the unique serial number is used as the test paper identifier of the original test paper. In this way, the original test paper image corresponding to the test paper image to be processed can be obtained by the following steps:

[0064] First, obtain the image identifier in the test paper image to be processed.

[0065] For example, the test paper image to be processed can be input into a preset image recognition model to obtain an image identifier, and the preset image recognition model may include an OCR (Optical Character Recognition) model, and the image identifier can be obtained by performing text recognition at a preset position (such as a header, a footer, or an edge position in any direction of the top, bottom, left, or right) of the test paper image to be processed through the OCR model. The image identifier may include a test paper identifier corresponding to the test paper image to be processed and a page number of the test paper image to be processed.

[0066] Secondly, according to the image identifier, the original test paper image of the original test paper corresponding to the test paper image to be processed is obtained from a preset test paper library.

[0067] For example, the preset test paper library may include a correspondence between an image identifier and an original test paper image, through which the corresponding original test paper image can be retrieved according to the image identifier, thereby obtaining a set of image pairs of the test paper image to be processed and the original test paper image.

[0068] S102: Obtaining optical flow information between the original test paper image and the test paper image to be processed.

[0069] Among them, the optical flow information can be used to represent the offset information of the target object in the two images. The target object can be the questions, text, formulas, graphics and other information in the original test paper image. For example, the offset information of the target question in the original test paper image and the same target question in the test paper image to be processed can be determined.

[0070] In some embodiments, the optical flow information can be obtained by a sparse optical flow method. For example, multiple first target points on the original test paper image can be determined, the first position information of the first target point in the original test paper image and the second position information of the first target point in the test paper image to be processed can be determined, the offset of the first target point can be calculated based on the first position information and the second position information, and the optical flow information can be determined based on the offsets of the multiple first target points.

[0071] In other embodiments, the optical flow information can also be obtained by a dense optical flow method. The dense optical flow method can perform pixel-level image registration to obtain pixel-level offset information. For example, the position information of each pixel point on the original test paper image and the offset of the corresponding pixel point of the test paper image to be processed can be calculated, and the optical flow information can be determined based on the offset of multiple pixels. In this way, more accurate optical flow information can be obtained.

[0072] S103: aligning the original test paper image and the test paper image to be processed according to the optical flow information.

[0073] For example, the original test paper image can be remapped according to the optical flow information, for example, by adjusting the position of each pixel to stretch or scale the original test paper image to obtain the original test paper image aligned with the test paper image to be processed.

[0074] S104. Obtain answer information corresponding to the image of the test paper to be processed according to the aligned original test paper image and the image of the test paper to be processed.

[0075] For example, the aligned original test paper image can be compared with the test paper image to be processed, and the test paper image to be processed and the aligned original test paper image can be subtracted to obtain the answer area, that is, the area that is not blank in the test paper image to be processed but blank in the original test paper image is used as the answer area. The text, formula, pattern and other information in the answer area of ​​the test paper image to be processed can be used as the answer information.

[0076] In addition, the intersection of the test paper image to be processed and the aligned original test paper image can be used as the original area (also called the non-answer area, printing area), and the image area on the test paper image to be processed other than the original area can be used as the answer area. The answer area can be stored in the form of a mask, and the mask is used to represent the coordinates of the answer area.

[0077] By adopting the above method, the original test paper image corresponding to the test paper image to be processed is obtained; the test paper image to be processed is the image after the user answers the test paper corresponding to the original test paper image; the optical flow information between the original test paper image and the test paper image to be processed is obtained; the original test paper image and the test paper image to be processed are aligned according to the optical flow information; the answer information corresponding to the test paper image to be processed is obtained according to the aligned original test paper image and the test paper image to be processed. In this way, the user's answer information can be obtained more accurately according to the aligned image, and the accuracy of answer information acquisition is improved. At the same time, since there is no need to perform handwritten text recognition, the answer information can also be accurately obtained for the scene where both the question and the answer are handwritten, which increases the application scenarios of test paper image processing and improves the accuracy of answer information acquisition.

[0078] In another embodiment of the present disclosure, the method of obtaining the optical flow information in the above step S102 may include: determining the target offset vector of each pixel point of the original test paper image and the test paper image to be processed; and obtaining the optical flow information according to the target offset vector. For example, the target offset vectors of multiple pixels may be used as the optical flow information.

[0079] In another embodiment of the present disclosure, the target offset vector may be obtained by the following steps:

[0080] For example, the original test paper image and the test paper image to be processed can be input into a pre-trained optical flow network, and the target offset vector of each pixel point can be calculated by the optical flow network, and the target offset vector can be used as optical flow information.

[0081] For example, the target offset vector can be calculated using the following formula (1):

[0082] F=G(I src ,I tgt ) (1);

[0083] Where F represents the target offset vector, I tgt Represents the test paper image to be processed, I src Represents the original test paper image, and G represents the dense optical flow calculation.

[0084] In this way, after obtaining the target offset vector, the position of each pixel point of the original test paper image in the image can be adjusted according to the target offset vector to align the original test paper image with the test paper image to be processed.

[0085] For example, each pixel of the original test paper image may be adjusted according to the target offset vector to obtain a new original test paper image that is aligned with the test paper image to be processed.

[0086] For example, the original test paper image can be adjusted by the following formula (2):

[0087] I aligned =H(I src ,F) (2);

[0088] Where F represents the target offset vector, I src represents the original test paper image, H represents the image pixel position adjustment processing, I aligned Represents the new original test paper image after being aligned with the test paper image to be processed.

[0089] For example, Figures 2 to 4 The above I src ,I tgt and I aligned Schematic diagram of , where: Figure 2 is a schematic diagram of an original test paper image provided by an embodiment of the present disclosure (for example, I src ), Figure 3 is a schematic diagram of a test paper image to be processed provided by an embodiment of the present disclosure (for example, I tgt ), Figure 4 The original test paper image is processed according to the method provided in any of the above embodiments, and the new original test paper image (for example, I aligned ).

[0090] In another embodiment of the present disclosure, the target offset vector of each pixel point of the to-be-processed test paper image and the original test paper image may be calculated according to a preset energy optimization function.

[0091] The preset energy optimization function may include one or more parameters, for example, may include optical flow continuity constraint parameters that characterize the test paper image to be processed and the original test paper image.

[0092] For example, the optical flow continuity constraint parameter can be calculated by the following formula (3):

[0093]

[0094] Among them, E C represents the continuity constraint parameter of the optical flow, x represents the x-axis coordinate of the specified pixel in the original test paper image, y represents the y-axis coordinate of the specified pixel in the original test paper image, u represents the offset of the specified pixel in the x-axis direction, that is, the difference between the x-axis coordinate of the specified pixel in the test paper image to be processed and the x-axis coordinate in the original test paper image, v represents the offset of the specified pixel in the y-axis direction, that is, the difference between the y-axis coordinate of the specified pixel in the test paper image to be processed and the y-axis coordinate in the original test paper image, It means to find the second-order derivative (for example represents the second-order derivative of u), It means to find the first-order derivative (for example Indicates x 2 Find the first-order derivative), Express The vector performs a modulo operation, where ζ represents a preset constant (eg, 0.001, 0.01, or 0.1), which can be used to prevent the target value (eg, the denominator or square root) from being zero.

[0095] It should be noted that in the related art, the energy optimization function corresponding to the optical flow algorithm is used to calculate the optical flow information of general scenes. In general scenes, since the displacement of objects can be regarded as random, that is, the movement direction and size of adjacent pixels in the image may not be continuous. For example, in a street view video, the movement directions of adjacent vehicles are different, so a discontinuous optical flow is obtained. For example, the optical flow between the moving object and the background is discontinuous, so the optical flow algorithm in the related art does not consider the continuity of the optical flow. For images taken from paper test papers or other paper documents, if the paper documents are not torn, their optical flow information is continuous. The optical flow continuity constraint parameter can be used to ensure the optical flow continuity of the test paper image to be processed and the original test paper image. By including the optical flow continuity constraint parameter in the preset energy optimization function, the continuity constraint on the test paper image can be enhanced, which is more targeted for the test paper image processing scene and can improve the accuracy of the optical flow information calculation.

[0096] Furthermore, the preset energy optimization function may also include a brightness constraint parameter, a smoothness constraint parameter and a feature matching constraint parameter.

[0097] For example, the preset energy optimization function may include the following formula (4):

[0098] E(u,v)=∫(E D +αE S +βE M +λE C )dxdy (4);

[0099] Where E(u,v) represents the target offset vector, u represents the offset of the specified pixel point in the x-axis direction, v represents the offset of the specified pixel point in the y-axis direction, and E D represents the brightness constraint parameter, E S represents the smoothness constraint parameter, E M represents the feature matching constraint parameter, E Crepresents the above optical flow continuity constraint parameter, x represents the x-axis coordinate of the specified pixel in the original test paper image, dx represents the differential of x, y represents the y-axis coordinate of the specified pixel in the original test paper image, dy represents the differential of y, α represents the first weight hyperparameter, β represents the second weight hyperparameter, and λ represents the third weight hyperparameter. The above weight hyperparameters α, β, and λ can be obtained after training based on sample data.

[0100] The brightness constraint parameter may include the following formula (5):

[0101]

[0102] Among them, E D represents the brightness constraint parameter, c represents the number of channels of the original test paper image. In RBG mode, the number of channels can be 3, u represents the offset of the specified pixel in the x-axis direction, that is, the difference between the x-axis coordinate in the test paper image to be processed and the x-axis coordinate in the original test paper image corresponding to the specified pixel, v represents the offset of the specified pixel in the y-axis direction, that is, the difference between the y-axis coordinate in the test paper image to be processed and the y-axis coordinate in the original test paper image corresponding to the specified pixel, ζ represents a preset constant (for example, 0.001, 0.01 or 0.1), which can be used to avoid the target value being zero, δ represents the fourth weight hyperparameter, and ε represents the fifth weight hyperparameter. The above weight hyperparameters δ and ε can also be obtained after training based on sample data. represents the first brightness constraint sub-parameter, Represents the second brightness constraint sub-parameter (also called gradient constraint parameter).

[0103] The first brightness constraint sub-parameter may include the following formula (6):

[0104]

[0105] in, represents the first brightness constraint sub-parameter, I i represents the pixel value (e.g., brightness value) of the specified pixel point on the i-th channel, x represents the x-axis coordinate of the specified pixel point in the original test paper image, y represents the y-axis coordinate of the specified pixel point in the original test paper image, t may represent a preset time interval. In this embodiment, t may be a preset fixed value (e.g., 1 second), It means to find the first-order derivative (for example represents the first-order derivative with respect to x), and ζ also represents a preset constant (for example, 0.001, 0.01 or 0.1).

[0106] The second brightness constraint sub-parameter may include the following formula (7):

[0107]

[0108] in, represents the second brightness constraint sub-parameter (also called gradient constraint parameter), Indicates the average pixel value (e.g., average brightness value) of the pixel with the same x-axis coordinate as the specified pixel on the i-th channel. Indicates the average pixel value (e.g., average brightness value) of the pixel with the same y-axis coordinate as the specified pixel on the i-th channel. It means to find the first-order derivative (for example represents the first-order derivative with respect to x), and ζ also represents a preset constant (for example, 0.001, 0.01 or 0.1).

[0109] The above smoothness constraint parameters may include the following formula (8):

[0110]

[0111] Among them, E S represents the smoothness constraint parameter, u represents the offset of the specified pixel point in the x-axis direction, and v represents the offset of the specified pixel point in the y-axis direction. Represents the first-order derivative of the offset in the x-axis direction, Represents the first derivative of the offset in the y-axis direction.

[0112] The above feature matching constraint parameters may include the following formula (9):

[0113]

[0114] Among them, E M represents the feature matching constraint parameter, D can characterize the test paper variable whether the specified pixel point can be matched. When the specified pixel point in the test paper image to be processed has a pixel point that matches the original test paper image, D can be set to 1. Otherwise, when the specified pixel point in the test paper image to be processed does not have a pixel point that matches the original test paper image, D can be set to 0. w represents the offset to be determined obtained by the current iteration calculation, w′ represents the target offset vector obtained by the previous round of iteration calculation, ζ also represents a preset constant (for example, 0.001, 0.01 or 0.1), and φ(x) represents the operation on x through the following formula (10):

[0115]

[0116] in, represents the value obtained by multiplying the minimum eigenvalue of the autocorrelation matrix of x by 10, σ Mrepresents a preset value, for example, it can be 50, and x can be ), Δ(x) represents the operation on x through the following formula (11):

[0117]

[0118] Wherein, Δ(x) represents the operation of x through formula (11), c represents the number of channels of the original test paper image, which can be 3 in RBG mode, and x can also be ), Indicates solving the first-order derivative, w′(x) indicates the value of x obtained in the previous iteration of the current iteration, Represents the pixel value (e.g. brightness value) of the original test paper image at the specified pixel point in the i-th channel. Represents the pixel value (e.g. brightness value) of the original test paper image at the specified pixel point in the i-th channel.

[0119] In this way, the above-mentioned preset energy optimization function can be trained through sample data to determine the weight hyperparameters therein; then, based on the trained preset energy optimization function, the target offset vector of each pixel point of the to-be-processed test paper image and the original test paper image can be calculated.

[0120] It should be noted that the method of obtaining optical flow information by using a preset energy optimization function may also refer to the method in the related art, and the present disclosure does not limit this.

[0121] In another embodiment of the present disclosure, the above step S104 may include the following steps:

[0122] Firstly, the aligned original test paper image and the test paper image to be processed are both subjected to image binarization processing to obtain an original binary image corresponding to the original test paper image and a binary image to be processed corresponding to the test paper image to be processed.

[0123] In this step, the aligned original test paper image is subjected to image binarization processing to obtain an original binary image; and the to-be-processed test paper image is subjected to image binarization processing to obtain a to-be-processed binary image.

[0124] For example, through image binarization processing, the grayscale value of the pixel point on the image to be processed (which may be the aligned original test paper image or the test paper image to be processed) can be set to a first preset grayscale value or a second preset grayscale value, wherein the first preset grayscale value may be a grayscale maximum value representing black, such as 255; and the second preset grayscale value may be a grayscale minimum value representing white. For example, the grayscale of pixels greater than or equal to the preset critical grayscale value is set to the first preset grayscale value, and the grayscale of pixels less than the preset critical grayscale value is set to the second preset grayscale value, thereby realizing image binarization processing. In this way, after the binarization processing, the binary image to be processed and the original binary image both present an obvious visual effect of only black and white, so as to accurately and efficiently compare the two images and determine the answer area.

[0125] Furthermore, if the test paper image to be processed is uploaded by taking a photo, in order to more accurately calculate the optical flow information, the test paper image to be processed obtained by taking the photo can be preprocessed, and the preprocessing may include shadow erasing, noise elimination, rectangle correction, etc.

[0126] Then, the answering area is determined according to the binary image to be processed and the original binary image.

[0127] For example, the area with a difference between the binary image to be processed and the original binary image can be used as the answer area; for example, the answer area is obtained by subtracting the binary image to be processed from the original binary image, that is, the area that is not blank in the test paper image to be processed but blank in the original test paper image is used as the answer area. The answer area can be stored in the form of a mask, and the mask is used to represent the coordinates of the answer area.

[0128] In addition, the intersection of the binary image to be processed and the original binary image can be used as the original area (also called the non-answer area, printing area), and the image area on the test paper image to be processed except the original area can be used as the answer area.

[0129] Finally, the answer information is determined according to the test paper image to be processed and the answer area.

[0130] For example, information such as text, formulas, patterns, etc. in the answer area of ​​the test paper image to be processed can be used as the answer information.

[0131] In another embodiment of the present disclosure, the above-mentioned test paper image to be processed can be obtained in the following manner:

[0132] First, a pending image after the user answers the test paper corresponding to the original test paper image is obtained through a camera device.

[0133] Secondly, multiple preset key points of the image to be determined are determined through a key point detection algorithm.

[0134] Finally, the test paper image to be processed is cropped and corrected according to the preset key points to obtain the test paper image to be processed.

[0135] For example, the preset key point can be the rectangular vertices of the image to be determined. A quadrilateral area can be determined based on the rectangular vertices, the image part outside the quadrilateral area is cut off, and then the remaining quadrilateral is subjected to rectangular correction to obtain a standard rectangular image, which is used as the test paper image to be processed.

[0136] In this way, by cropping and correcting the captured image to be processed, the acquired test paper image to be processed can be made into a more standardized rectangular image, and the surrounding non-test paper area can be removed, so as to more accurately and efficiently analyze the test paper image to be processed and accurately obtain the answer information in the image.

[0137] Figure 5 is a schematic diagram of the structure of a test paper image processing device provided by an embodiment of the present disclosure, such as Figure 5 As shown, the device comprises:

[0138] The image acquisition module 501 is used to acquire the original test paper image corresponding to the test paper image to be processed; the test paper image to be processed is the image after the user answers the test paper corresponding to the original test paper image;

[0139] The information acquisition module 502 is used to acquire the optical flow information between the original test paper image and the test paper image to be processed;

[0140] An image alignment module 503 is used to align the original test paper image and the test paper image to be processed according to the optical flow information;

[0141] The image processing module 504 is used to obtain the answer area corresponding to the test paper image to be processed according to the aligned original test paper image and the test paper image to be processed.

[0142] Optionally, the information acquisition module 502 is used to determine a target offset vector of each pixel point of the original test paper image and the test paper image to be processed; and acquire the optical flow information according to the target offset vector;

[0143] The image alignment module 503 is used to adjust the position of each pixel of the original test paper image in the image according to the target offset vector, so as to align the original test paper image with the test paper image to be processed.

[0144] Optionally, the information acquisition module 502 is used to calculate the target offset vector of each pixel point of the original test paper image and the test paper image to be processed according to a preset energy optimization function; the preset energy optimization function includes optical flow continuity constraint parameters that characterize the original test paper image and the test paper image to be processed.

[0145] Optionally, the preset energy optimization function also includes a brightness constraint parameter, a smoothness constraint parameter and a feature matching constraint parameter.

[0146] Optionally, the image processing module 504 is used to perform image binarization processing on the test paper image to be processed to obtain a binary image to be processed; perform image binarization processing on the original test paper image to obtain an original binary image; determine the answer area based on the binary image to be processed and the original binary image; determine the answer information based on the test paper image to be processed and the answer area.

[0147] Optionally, the image acquisition module 501 is used to obtain the image identification in the test paper image to be processed; the image identification includes the test paper identification corresponding to the test paper image to be processed and the page number of the test paper image to be processed; according to the image identification, the original test paper image of the original test paper corresponding to the test paper image to be processed is obtained from the preset test paper library.

[0148] Optionally, the image acquisition module 501 is used to acquire, through a camera device, a pending image after the user answers the questions on the test paper corresponding to the original test paper image; determine multiple preset key points of the pending image through a key point detection algorithm; and perform cropping and correction on the pending test paper image according to the preset key points to obtain the pending test paper image.

[0149] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0150] Figure 6 FIG. 9 is a block diagram of an electronic device 900 according to an exemplary embodiment. Figure 6 As shown, the electronic device 900 may include: a processor 901 and a memory 902. The electronic device 900 may also include one or more of a multimedia component 903, an input / output (I / O) interface 904, and a communication component 905.

[0151] The processor 901 is used to control the overall operation of the electronic device 900 to complete all or part of the steps in the above-mentioned test paper image processing method. The memory 902 is used to store various types of data to support the operation of the electronic device 900, and these data may include, for example, instructions for any application or method used to operate on the electronic device 900, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 902 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 903 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 902 or sent through the communication component 905. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 904 provides an interface between the processor 901 and other interface modules, and the other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 905 is used for wired or wireless communication between the electronic device 900 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, 5G, NB-IOT, eMTC, or other 6G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 905 may include: Wi-Fi module, Bluetooth module, NFC module, etc.

[0152] In an exemplary embodiment, the electronic device 900 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned test paper image processing method.

[0153] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned test paper image processing method are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 902 including program instructions, and the above-mentioned program instructions can be executed by the processor 901 of the electronic device 900 to complete the above-mentioned test paper image processing method. By way of example, the computer-readable storage medium can be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0154] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program that can be executed by a programmable device. The computer program has a code portion for executing the above-mentioned test paper image processing method when executed by the programmable device.

[0155] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0156] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0157] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A test paper image processing method, characterized in that: The method comprises: Acquire an original test paper image corresponding to the test paper image to be processed; the test paper image to be processed is an image after the user answers the test paper corresponding to the original test paper image; Obtaining optical flow information between the original test paper image and the test paper image to be processed; The obtaining of optical flow information between the original test paper image and the test paper image to be processed comprises: Determine a target offset vector for each pixel point of the original test paper image and the test paper image to be processed; and obtain the optical flow information according to the target offset vector; Determining the target offset vector of each pixel point of the original test paper image and the test paper image to be processed comprises: According to the preset energy optimization function, the target offset vector of each pixel point of the to-be-processed test paper image and the original test paper image is calculated; the preset energy optimization function includes an optical flow continuity constraint parameter, a brightness constraint parameter, a smoothness constraint parameter and a feature matching constraint parameter; the preset energy optimization function is as follows: ； Where E(u, v) represents the target offset vector, u represents the offset of the specified pixel point in the x-axis direction, v represents the offset of the specified pixel point in the y-axis direction, and E D represents the brightness constraint parameter, E S represents the smoothness constraint parameter, E M represents the feature matching constraint parameter, E C represents the optical flow continuity constraint parameter, x represents the x-axis coordinate of the specified pixel in the original test paper image, dx represents the differential of x, y represents the y-axis coordinate of the specified pixel in the original test paper image, dy represents the differential of y, α represents the first weight hyperparameter, β represents the second weight hyperparameter, and λ represents the third weight hyperparameter; Performing image alignment on the original test paper image and the test paper image to be processed according to the optical flow information; The aligning the original test paper image and the test paper image to be processed according to the optical flow information comprises: According to the target offset vector, the position of each pixel point of the original test paper image in the image is adjusted to align the original test paper image with the test paper image to be processed; According to the aligned original test paper image and the test paper image to be processed, answer information corresponding to the test paper image to be processed is obtained.

2. The method according to claim 1, characterized in that The step of obtaining answer information corresponding to the test paper image to be processed according to the aligned original test paper image and the test paper image to be processed comprises: Performing image binarization processing on the test paper image to be processed to obtain a binary image to be processed; Performing image binarization processing on the original test paper image to obtain an original binary image; Determine an answering area according to the binary image to be processed and the original binary image; The answer information is determined according to the test paper image to be processed and the answer area.

3. The method according to claim 1, characterized in that The step of obtaining the original test paper image corresponding to the test paper image to be processed comprises: Obtaining an image identifier in the test paper image to be processed; the image identifier includes the test paper identifier corresponding to the test paper image to be processed and the page number of the test paper image to be processed; According to the image identifier, an original test paper image of the original test paper corresponding to the test paper image to be processed is obtained from a preset test paper library.

4. The method according to any one of claims 1 to 3, characterized in that The test paper image to be processed is obtained by: Acquire, by means of a camera, a pending image after the user has answered questions on the test paper corresponding to the original test paper image; Determine a plurality of preset key points of the pending image by a key point detection algorithm; The test paper image to be processed is cropped and corrected according to the preset key points to obtain the test paper image to be processed.

5. A test paper image processing device, characterized in that: The device comprises: An image acquisition module is used to acquire an original test paper image corresponding to the test paper image to be processed; the test paper image to be processed is an image after the user answers the test paper corresponding to the original test paper image; An information acquisition module is used to acquire optical flow information between the original test paper image and the test paper image to be processed; the acquisition of optical flow information between the original test paper image and the test paper image to be processed includes: determining a target offset vector for each pixel point of the original test paper image and the test paper image to be processed; acquiring the optical flow information according to the target offset vector; the determination of the target offset vector for each pixel point of the original test paper image and the test paper image to be processed includes: calculating the target offset vector for each pixel point of the test paper image to be processed and the original test paper image according to a preset energy optimization function; the preset energy optimization function optical flow continuity constraint parameters, brightness constraint parameters, smoothness constraint parameters and feature matching constraint parameters; the preset energy optimization function is as follows: ； Where E(u, v) represents the target offset vector, u represents the offset of the specified pixel point in the x-axis direction, v represents the offset of the specified pixel point in the y-axis direction, and E D represents the brightness constraint parameter, E S represents the smoothness constraint parameter, E M represents the feature matching constraint parameter, E C represents the optical flow continuity constraint parameter, x represents the x-axis coordinate of the specified pixel in the original test paper image, dx represents the differential of x, y represents the y-axis coordinate of the specified pixel in the original test paper image, dy represents the differential of y, α represents the first weight hyperparameter, β represents the second weight hyperparameter, and λ represents the third weight hyperparameter; An image alignment module is used to align the original test paper image with the test paper image to be processed according to the optical flow information; the image alignment of the original test paper image with the test paper image to be processed according to the optical flow information comprises: adjusting the position of each pixel point of the original test paper image in the image according to the target offset vector, so as to align the original test paper image with the test paper image to be processed; The image processing module is used to obtain the answer information corresponding to the test paper image to be processed based on the aligned original test paper image and the test paper image to be processed.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

7. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 4.

Images