Data processing method, device, electronic device and computer readable storage medium

By obtaining and processing job layout pictures, using text segmentation and convolutional neural network recognition technology, we can automatically determine and correct questions and answers in the job, solving the problem of low efficiency in the existing technology, and achieving efficient and accurate automation Correction.

CN112183253BActive Publication Date: 2024-11-15BEIJING DA MI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010969800.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-09-15
Publication Date
2024-11-15
Estimated Expiration
2040-09-15

AI Technical Summary

Technical Problem

During the existing teaching process, the homework correction efficiency is low, making it difficult to efficiently organize the data on students' answers, resulting in the degree of automation and efficiency not meeting expectations.

Method used

By obtaining and processing job layout pictures, using text segmentation networks and convolutional neural networks to identify printed and handwritten text, determine the coordinates and text information of the question area, automatically match the questions and answers, and realize automatic correction.

Benefits of technology

It improves the degree of automation and efficiency of job correction, reduces the time and errors of manual correction, and ensures the accuracy and consistency of correction results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112183253B_ABST
    Figure CN112183253B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention discloses a data processing method, device, electronic device and computer-readable storage medium, wherein the method processes a layout image, determines the stem text and answer text of each question according to each answer sub-image and the corresponding question area coordinates, determines the question identifier according to the text information corresponding to the stem text, and then determines the correct answer, and realizes automatic correction by comparing the answer information corresponding to the answer text with the correct answer information. Thus, automatic correction of homework is realized, and the degree of automation and efficiency of homework correction are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data processing method, device, electronic device and computer-readable storage medium. Background Art

[0002] Correcting homework is an important part of the teaching process, and the efficiency of correcting homework is directly related to the advancement of the teaching process.

[0003] In the current teaching process, after students complete their homework, they usually need to be manually graded by teachers. Manual grading is inefficient and it is difficult to organize students' answer data efficiently and completely. Summary of the invention

[0004] In view of this, embodiments of the present invention provide a data processing method, device, electronic device, and computer-readable storage medium to improve the automation and efficiency of homework grading.

[0005] In a first aspect, an embodiment of the present invention provides a data processing method, the method comprising:

[0006] Acquire a layout image, wherein the layout image displays a plurality of answer sub-areas, wherein the answer sub-areas include a first text and a second text in printed form, and a separation mark is set between adjacent answer sub-areas;

[0007] Determine a plurality of answer sub-pictures and corresponding question area coordinates according to the layout image, wherein the plurality of answer sub-pictures correspond to different answer sub-areas respectively;

[0008] Determine first text information and second text information in the layout image, the first text information includes first text coordinates and a first text sequence, and the second text information includes second text coordinates and a second text sequence;

[0009] Determine the first text and the second text of each of the questions according to the coordinates of the question area, the first text information and the second text information;

[0010] Determining corresponding third text information according to the first text of each topic; and

[0011] The fourth text information of each topic is determined according to the third text information.

[0012] Furthermore, the separation mark is a closed dotted question frame displayed outside the plurality of question answering sub-areas;

[0013] Determining multiple answer sub-pictures and corresponding question area coordinates according to the layout picture includes:

[0014] Adjusting the size of the layout image to a predetermined size to obtain a first layout image;

[0015] Binarizing the first layout image to obtain a second layout image;

[0016] Performing an image closing operation on the second layout image to obtain a third layout image;

[0017] Performing connected domain analysis and screening on the third page image to obtain a title area frame; and

[0018] Adjust the size of the question area frame to the original size, and determine the coordinates of each of the answer sub-pictures and the corresponding question area;

[0019] The question area coordinates are used to represent the position information of each answer sub-area on the layout image.

[0020] Furthermore, the determining the first text information and the second text information in the layout image includes:

[0021] Inputting the layout image into a text segmentation network to obtain a plurality of feature vectors; and

[0022] Performing text detection and text recognition on the multiple feature vectors to obtain a first text sequence and a second text sequence as well as a first text coordinate and a second text coordinate;

[0023] Among them, the first text sequence is the printed text content in the layout image, and the first text coordinates are the position information of the first text sequence on the layout image; the second text sequence is the handwritten content in the layout image, and the second text coordinates are the position information of the second text sequence on the layout image.

[0024] Furthermore, the backbone network structure of the text segmentation network is a residual network model, and the text recognition network adopts a convolutional neural network model.

[0025] Furthermore, the first text and the second text of each question are determined according to the coordinates of the question area, the first text information and the second text information, specifically:

[0026] Analyzing and screening the first text information and the second text information according to the topic area coordinates, the first text coordinates and the second text coordinates of each of the topics; and

[0027] The first text sequence and the second text sequence that meet the preset conditions are used as the first text and the second text of each topic.

[0028] Furthermore, the third text information corresponding to the first text of each question is determined specifically as follows:

[0029] Searching the question bank content according to the first text of each question to determine the question identifier with the highest matching degree; and

[0030] Obtaining standard answer information corresponding to the question identifier, and using the standard answer information as the third text information;

[0031] The third text information is the correct answer information of each question.

[0032] Furthermore, searching the question bank content according to the first text of each question to determine the question identifier with the highest matching degree is specifically:

[0033] The questions in the question bank are searched by fuzzy matching, and the question identifier with the highest boundary distance similarity is used as the question identifier with the highest matching degree.

[0034] Furthermore, the fourth text information of each topic determined according to the third text information is specifically:

[0035] The second text of each question is compared with the third text information to determine the correction information of each question.

[0036] In a second aspect, an embodiment of the present invention provides a data processing device, the device comprising:

[0037] An acquisition unit is used to acquire a layout image, wherein the layout image displays a plurality of answer sub-areas, wherein the answer sub-areas include a first text and a second text, and a separation mark is set between adjacent answer sub-areas;

[0038] A first determining unit is used to determine a plurality of answer sub-pictures and corresponding question area coordinates according to the layout image, wherein the plurality of answer sub-pictures correspond to different answer sub-areas respectively;

[0039] A second determining unit, used to determine first text information and second text information in the layout image, wherein the first text information includes first text coordinates and a first text sequence, and the second text information includes second text coordinates and a second text sequence;

[0040] A third determining unit, configured to determine the first text and the second text of each of the questions according to the coordinates of the question area, the first text information and the second text information;

[0041] A matching unit, configured to determine corresponding third text information according to the first text of each question; and

[0042] The correction unit is used to determine the fourth text information of each question according to the third text information.

[0043] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as described above.

[0044] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium on which computer program instructions are stored, wherein the computer program instructions implement the method described above when executed by a processor.

[0045] The data processing method of the embodiment of the present invention obtains a layout image, determines multiple answer sub-images and corresponding question area coordinates according to the layout image, determines the first text information and the second text information in the layout image, determines the stem text and the answer text of each question according to the question area coordinates, the first text information and the second text information, determines the corresponding correct answer information according to the stem text of each question, and determines the correction information of each question according to the correct answer information. In this way, the determination and automatic correction of the stem text and the answer text in the homework layout image are realized, and the automation degree and efficiency of homework correction are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0047] Figure 1 is a flow chart of a data processing method according to an embodiment of the present invention;

[0048] Figure 2 is a flow chart of determining the coordinates of the answer sub-picture and the question area according to an embodiment of the present invention;

[0049] Figure 3 is a flow chart of determining first text information and second text information according to an embodiment of the present invention;

[0050] Figure 4 is a flow chart of obtaining multiple feature vectors according to an embodiment of the present invention;

[0051] Figure 5 is a flow chart of determining question stem text and answer text according to an embodiment of the present invention;

[0052] Figure 6 is a flow chart of determining correct answer information according to an embodiment of the present invention;

[0053] Figure 7 is a data flow diagram of the data processing method according to an embodiment of the present invention;

[0054] Figure 8is a schematic diagram of a data processing device according to an embodiment of the present invention;

[0055] Fig. 9 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The present invention is described below based on embodiments, but the present invention is not limited to these embodiments. In the detailed description of the present invention below, some specific details are described in detail. It is possible for a person skilled in the art to fully understand the present invention without the description of these details. In order to avoid confusing the essence of the present invention, known methods, processes, flows, components and circuits are not described in detail.

[0057] In addition, persons of ordinary skill in the art will appreciate that the drawings provided herein are for illustration purposes and are not necessarily drawn to scale.

[0058] Unless the context clearly requires otherwise, words such as “include”, “including” and the like in the specification should be interpreted as including rather than being exclusive or exhaustive; that is, as meaning “including but not limited to”.

[0059] In the description of the present invention, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0060] Correcting homework is an important part of the teaching process, and the efficiency of correcting homework is directly related to the advancement of the teaching process.

[0061] In the current teaching process, after students complete their homework, they usually need to be manually graded by teachers. Manual grading is inefficient and it is difficult to efficiently and completely sort out the students' answer data, making it impossible for students to obtain the results of homework grading in a timely manner. Therefore, the degree of automation and efficiency of grading homework needs to be improved.

[0062] The technical solution of the embodiment of the present invention provides a data processing method, which can improve the automation and efficiency of homework correction.

[0063] In this embodiment, the correction of fill-in-the-blank questions in objective questions is used as an example for explanation. However, it is easy for those skilled in the art to understand that the method of the embodiment of the present invention is also applicable to the correction of questions with fixed question stem text and corresponding answer text in different scenarios.

[0064] Figure 1 is a flow chart of a data processing method according to an embodiment of the present invention. Figure 1 As shown, the data processing method of the embodiment of the present invention includes the following steps:

[0065] In step S100, a layout image is obtained.

[0066] In this embodiment, the student writes the homework on a special answer sheet, which displays multiple answer sub-areas. The content in the answer sub-area of ​​each question includes a printed question stem and a handwritten answer.

[0067] In an optional implementation, the text on the answer sheet is scanned and the data is read by a dedicated scanner device or scanning software on a mobile terminal device to obtain a layout image.

[0068] In this embodiment, the layout image is a picture corresponding to a plurality of homeworks to be corrected. Each layout image displays a plurality of answer sub-areas, each answer sub-area including a printed question stem text and a handwritten answer text.

[0069] Preferably, in this embodiment, a separation mark is provided between adjacent answer sub-areas. Since the question stem text and answer text of each question are determined during the grading of objective questions, the different answer sub-areas are distinguished by separation marks, so that the answer sub-areas corresponding to each question are arranged relatively concentratedly, which is convenient for scanning and reading data and grading of subsequent homework.

[0070] Specifically, in order to facilitate the setting of the separation mark, in this embodiment, the separation mark between adjacent answer sub-areas is set as a closed dotted question frame, and the dotted question frame is displayed at the outer position of each answer sub-area. Thus, different answer sub-areas are separated by the dotted question frame, which is convenient for correcting each question.

[0071] In order to facilitate the homework grading process and the statistics of homework grading results, preferably, in this embodiment, the upper left corner of the answer sub-area also displays the question number, which corresponds to different question contents.

[0072] In step S110, a plurality of answer sub-pictures and corresponding question area coordinates are determined according to the layout picture.

[0073] In this embodiment, multiple answer sub-pictures correspond to different answer sub-areas, and the question area coordinates are used to represent the position information of each answer sub-area on the layout image. By determining multiple different answer sub-pictures and corresponding answer area coordinates, each answer sub-area can be separated, thereby facilitating the correction of each question, which is conducive to improving the efficiency of homework correction.

[0074] Figure 2 FIG. 1 is a flow chart of determining the coordinates of the answer sub-picture and the question area according to an embodiment of the present invention. Figure 2As shown, in an optional implementation, the embodiment of the present invention determines multiple answer sub-pictures and corresponding question area coordinates according to the layout picture, including the following steps:

[0075] In step S200, the size of the layout image is adjusted to a predetermined size to obtain a first layout image.

[0076] In this embodiment, the size of the long side of the layout image is adjusted to a set length, and the size of the adjacent sides of the long side of the layout image is adjusted according to the aspect ratio of the answer sheet. Thus, by adjusting the size of the layout image to a predetermined size, it is convenient to adjust the images corresponding to multiple homeworks to be corrected to a uniform size, which is conducive to improving the efficiency of homework correction.

[0077] Preferably, the long side size of this embodiment is adjusted to 1024.

[0078] In step S210, the first layout image is binarized to obtain a second layout image.

[0079] Image binarization refers to the process of setting the grayscale value of the pixels on the image to 0 or 255 so that the first layout image appears as a black and white image.

[0080] In this embodiment, the first layout image is processed by image binarization to obtain a second layout image in black and white.

[0081] In step S220, an image closing operation is performed on the second layout picture to obtain a third layout picture.

[0082] In this embodiment, the image closing operation is to convert the dotted line question frame that closes the periphery of each answer sub-area into a solid line frame, so that the separation of each answer sub-area is more obvious.

[0083] In step S230, the connected domain analysis and screening are performed on the third layout image to obtain the title area frame.

[0084] A connected domain refers to an image region consisting of foreground pixels with the same pixel value and adjacent positions.

[0085] In this embodiment, by analyzing the connected domains, each connected domain in the third layout image is obtained and marked, and then the marked multiple connected domains are screened according to the preset screening conditions, invalid connected domains are filtered, and the connected domains that meet the screening conditions are used as the title area frame on the third layout image.

[0086] In step S240, the size of the question area frame is adjusted to the original size, and the coordinates of each answer sub-image and the corresponding question area are determined.

[0087] In this embodiment, after obtaining the question area frame, the question area frame size needs to be restored and adjusted to the original size, and then the answer sub-images corresponding to each answer sub-area and the question area coordinates corresponding to each answer sub-image are determined. The first question area coordinates are used to represent the position information of each question on the layout image.

[0088] In step S120, the first text information and the second text information in the layout image are determined.

[0089] In this embodiment, the first text information includes the first text coordinates and the first text sequence, and the second text information includes the second text coordinates and the second text sequence, wherein the first text sequence is the printed text content in the layout image. The first text coordinates are the position information of the first text sequence on the layout image. The second text sequence is the handwritten content in the layout image. The second text coordinates are the position information of the second text sequence on the layout image.

[0090] Figure 3 FIG. 4 is a flow chart of determining first text information and second text information according to an embodiment of the present invention. Figure 3 As shown, in an optional implementation, the process of determining the first text information and the second text information in the layout image in this embodiment includes the following steps:

[0091] In step S300, the layout image is input into a text segmentation network to obtain a plurality of feature vectors.

[0092] In an optional implementation, the backbone network structure of the text segmentation network in this embodiment is resnet18+FPN.

[0093] The resnet18 network is a residual network. resnet is the abbreviation of residual network. The resnet18 network includes an input layer (image), an independent convolution layer (conv1), a maximum pooling layer (maxpool), 4 convolution residual modules (conv2_x, conv3_x, conv4_x and conv5_x), an average pooling layer (avgpool) and a soft maximum output layer. Among them, the independent convolution layer (conv1) uses 64 convolution kernels with a size of 7*7 and a stride of 2. The maximum pooling layer (maxpool) selects a convolution kernel with a pooling window of 3*3 and a stride of 2. There are 3 conv2_x configurations, 4 conv3_x configurations, 6 conv4_x configurations, and 2 conv5_x configurations in the convolution residual module. Each convolution residual module consists of 2-3 convolution layers and cascades across different convolution layers. Since resnet18 is configured with multiple cascades, it can effectively extract multi-scale features in layout images and generate multiple pyramid feature maps. At the same time, compared with other cascade residual networks, resnet18 has a lighter structure, which is conducive to improving data processing speed, and thus to improving the automation and efficiency of homework correction.

[0094] FPN is a network structure that uses the layers from bottom to top inside the conventional CNN model to express the features of different dimensions of the same size image, which can effectively generate multi-dimensional feature expressions of the image under a single image. In this embodiment, the FPN network performs feature fusion on multiple pyramid feature maps output by the resnet18 network, and calculates the enhanced feature vector through a 1*1 convolution layer.

[0095] Figure 4 FIG. 1 is a flowchart of obtaining multiple feature vectors according to an embodiment of the present invention. Figure 4 As shown, the working principle of the segmentation network is explained by taking the input parameter of the layout image as w*h*3 as an example.

[0096] First, input the layout image with input parameters of w*h*3 into the resnet18 network, and extract the features in the layout image through the resnet18 network. Specifically, extract the outputs of the convolutional layers Conv2, Conv3, Conv4, and Conv5 with strides of 4, 8, 16, and 32 respectively as high- and low-level features, and obtain multiple feature maps, which form a pyramid feature map from high to low. Then use multiple feature maps as the input of the FPN network, perform feature fusion on the multiple feature maps through multiple cascaded FPN modules, obtain a feature fusion map, and input the obtained feature fusion map into the 1*1 convolutional layer to obtain a feature vector with parameters of w*h*12.

[0097] In this embodiment, the feature vector of w*h*12 includes a text region feature vector (text_region) of w*h*2, a convolution kernel feature vector (kernel) of w*h*2, and a similarity feature vector (similarity_vectors) of w*h*8. Among them, the text region feature vector includes a 1-dimensional printed text region feature vector and a 1-dimensional handwritten text region feature vector. The convolution kernel feature vector includes a 1-dimensional printed convolution kernel feature vector and a 1-dimensional handwritten convolution kernel feature vector. The similarity feature vector includes a 4-dimensional printed similarity vector and a 4-dimensional handwritten similarity vector.

[0098] In step S310, text detection and text recognition are performed on the multiple feature vectors to obtain a first text sequence and a second text sequence as well as a first text coordinate and a second text coordinate.

[0099] In this embodiment, the first text coordinates are pixel-level coordinates of the printed text sequence in the layout image; the second text coordinates are pixel-level coordinates of the handwritten text sequence in the layout image.

[0100] It should be noted that, in this embodiment, the first text sequence includes multiple first text subsequences, and the multiple first text subsequences correspond to the printed texts arranged separately on the layout image. The first text coordinates include the first text sub-coordinates corresponding to the multiple first text subsequences, and each first text sub-coordinate is the pixel coordinate of each first text subsequence on the layout image. Specifically, since the first text subsequence is usually displayed as a rectangular area on the layout, the pixel coordinates can be represented by the position coordinates corresponding to the upper left corner and the lower right corner of the rectangular area on the layout image.

[0101] The second text sequence includes a plurality of second text subsequences, and the plurality of text subsequences correspond to handwritten texts arranged separately on the layout image. The second text coordinates include second text coordinates corresponding to the plurality of second text subsequences, and each second text subcoordinate is a pixel coordinate of each second text subsequence on the layout image. Specifically, since the second text subsequence is usually displayed as a rectangular area on the layout, the pixel coordinates can be represented by the position coordinates corresponding to the upper left corner and the lower right corner of the rectangular area on the layout image.

[0102] Preferably, in this embodiment, the idea of ​​clustering is used to perform text detection on multiple feature vectors through similarity vectors. Specifically, a cluster center is determined, and the value output by the similarity feature vector is used as the value of the point to be clustered, so that the distance from the surrounding points to the cluster center is minimized. Thus, the printed text and handwritten text and the corresponding position information of the printed text and handwritten text on the layout image are obtained.

[0103] Preferably, the text recognition network of this embodiment adopts a CRNN+CTC network. The text content on the layout image is recognized by the CRNN+CTC network, and a first text sequence of printed text and a second text sequence of handwritten text are obtained.

[0104] It should be noted that, in order to improve the accuracy of the calculation, the total loss function (Loss) in step S120 is defined as follows:

[0105] L=L tex +a×L ker +b×(L agg +L dis )

[0106] Among them, L tex and L ker They are the segmentation loss functions corresponding to the text region feature vector (text_region) and the convolution kernel feature vector (kernel), and the segmentation loss function in this embodiment adopts dice soft loss.

[0107] L agg It is a loss function to measure the feature vector of the text area and the corresponding convolution kernel feature vector of the same text, which is used to ensure that the distance between the convolution kernel of the same text and other pixels is less than the set parameter δ agg .

[0108] L dis is the loss function of the convolution kernel feature vectors of different texts, which is used to ensure that the distance between any two convolution kernel feature vectors is greater than the set parameter δ agg .

[0109] a and b are set to 0.5 and 0.25, respectively, to represent the value of balancing each loss function.

[0110] In this embodiment, the specific expressions of each loss function are as follows:

[0111]

[0112] Among them, g tex [i][j] is the g of the i-th type of text at pixel j tex The value of p tex [i][j] is the predicted value of the i-th type of text at pixel j.

[0113]

[0114] Among them, g tex [i][j] is the g of the i-th type of text at pixel j tex The value of p ker[i][j] is the predicted value p of the i-th type of text at pixel j ker .

[0115]

[0116] Where N is the number of texts in the image, T i represents the i-th text, K i is the convolution kernel corresponding to the text, D(p,K i ) is the text T i Pixels p and K within i The distance between.

[0117] D(p,K i )=max(||F(p)-G(K i )||-δ agg ,0) 2

[0118] Among them, F(p) is the similarity vector at p, G(K i ) is K i The similarity vector of δ agg is a constant. In this embodiment, agg is 0.5.

[0119] D(p,K i )=max(||F(p)-G(K i )||-δ agg ,0) 2

[0120]

[0121] Among them, D(K i ,K j )=max(δ dis -||G(K i )-G(K j )||,0) 2 , δ dis is a constant. In this embodiment, dis is 3.

[0122] In step S130, the question stem text and answer text of each question are determined according to the coordinates of each question area, the first text information and the second text information.

[0123] Figure 5 1 is a flow chart of determining the question stem text and the answer text according to an embodiment of the present invention. Figure 5 As shown, the present embodiment specifically includes the following steps to determine the stem text and answer text of each question according to the coordinates of each question area, the first text information and the second text information:

[0124] In step S500, the first text information and the second text information are analyzed and screened according to the topic area coordinates, the first text coordinates and the second text coordinates of each topic.

[0125] In step S510, the first text sequence and the second text sequence that meet the preset conditions are used as the question stem text and the answer text of each question.

[0126] In an optional implementation, since the first text sequence and the second text sequence represent all the printed text and handwritten text on the layout image, respectively, and the stem text and the answer of each question are the printed text and the handwritten text in each answer sub-area, respectively. Therefore, the first text information and the second text information are analyzed according to the coordinates of the question area of ​​each question, and the first text sub-coordinates and the second text sub-coordinates and the corresponding first text sub-sequences and second text sub-sequences of the pixel coordinates in each answer sub-area are screened out, and the screened first text sub-sequences and second text sub-sequences are arranged from left to right and from top to bottom according to the pixel coordinates, respectively, to form new first text sequences and second text sequences, and the new first text sequences and second text sequences are respectively used as the stem text and the answer text of the corresponding questions. Thus, the stem text and the answer text of each question are determined by the method of text coordinate matching and screening, and the accuracy of the determination of the stem text and the answer text is improved, and the accuracy of homework correction is guaranteed while improving the degree of automation of homework correction.

[0127] It should be noted that when a question includes multiple sub-questions and corresponding multiple answer texts, the answer texts are marked from left to right and from top to bottom according to the pixel coordinates of each answer text on the page image, and the marked answer texts correspond to different sub-questions in the question. Preferably, the answer text marks can be numbered in sequence.

[0128] In another optional implementation, after determining each answer sub-image, this embodiment uses the same method as above to input each answer sub-image into the resnet18+FPN network in turn, extract and fuse the text in each answer sub-image, and then obtain the stem text and answer text corresponding to each answer sub-area through text detection and text recognition network. Thus, different methods are used to obtain the stem text and answer text of each question according to the actual scenario, which is more flexible and conducive to speeding up the efficiency of obtaining the stem text and answer text of each question, and further improving the efficiency of homework correction.

[0129] In step S140, the corresponding correct answer information is determined according to the question stem text of each question.

[0130] Figure 6is a flow chart of determining correct answer information according to an embodiment of the present invention. Figure 6 As shown, in this embodiment, determining the corresponding correct answer information according to the question stem text of each question specifically includes the following steps:

[0131] In step S600, the question bank content is searched according to the question stem text of each question to determine the question identifier with the highest matching degree.

[0132] The question bank of this embodiment stores question stem content and corresponding answer content, and the question stem content and the answer content are arranged separately.

[0133] Preferably, when executing step S600, the present embodiment treats all the stem texts of each question as a whole, and searches all the stem contents of the corresponding assignment in the question bank by fuzzy matching according to the stem texts of each question, and uses the boundary distance similarity as the judgment index, and uses the question identifier with the highest boundary distance similarity as the question identifier with the highest matching degree. In the present embodiment, the question serial number is used as the question identifier.

[0134] In step S610, standard answer information corresponding to the question identifier is obtained, and the standard answer information is used as correct answer information.

[0135] In this embodiment, the question content and standard answer information corresponding to each question in the question bank are determined by automatic search, which is conducive to improving the automation and efficiency of homework grading.

[0136] In step S150, the correction information of each question is determined according to the correct answer information.

[0137] Preferably, in this embodiment, the answer text of each question is compared with the correct answer information to determine the correction information of each question.

[0138] It should be noted that in this embodiment, it is assumed that all questions have answers. When a question includes multiple sub-questions, the numbered answer texts are compared with the correct answer information in the question bank in the order in which the sub-questions are arranged to determine whether the answer text on the layout image is correct, and then determine the correction information for each question.

[0139] Figure 7 : is a data flow diagram of the data processing method of an embodiment of the present invention. Figure 7 As shown, A is the layout image corresponding to the homework to be corrected. Layout image A displays answer sub-areas A1 and A2 corresponding to Question 01 and Question 02. Answer sub-areas A1 and A2 respectively display closed dotted boxes and the corresponding question numbers, question stem contents and answer contents within the dotted boxes.

[0140] The process of automatic marking includes the following steps:

[0141] In step S700, preprocess the layout picture A.

[0142] In step S710, determine the answer sub - pictures B1 and B2 corresponding to the answer sub - regions A1 and A2 and the first question - region coordinates and the second - region coordinates corresponding to each answer sub - picture according to the preprocessed layout picture.

[0143] In step S720, extract features, perform feature fusion, text detection, and text recognition on the features in the layout picture to obtain the first text information and the second text information in the layout picture.

[0144] In this embodiment, the first text information is all the printed texts C1 in the layout picture and the first text coordinates corresponding to the printed texts, and the second text information is all the handwritten texts C2 in the layout picture and the second text coordinates corresponding to the handwritten texts.

[0145] Specifically, the printed text C1 obtained in the layout picture in this embodiment is "01| Fill - in - the - blank questions (1) 4.35 is read as (), eight point zero four is written as (); 02| Calculation questions 1.2 + 2.3 = ().". The handwritten text C2 obtained in the layout picture is the handwritten "four point three five 8.04 3.5".

[0146] In step S730, analyze the first text information and the second text information according to the first question - region coordinates and the second question - region coordinates, and screen out the first text coordinates and the second text coordinates whose pixel coordinates are respectively within the first question - region coordinates and the second question - region coordinates, and their corresponding printed texts D1 and D2 and answer texts E1 and E2, and use them as the stem texts and answer texts corresponding to question 01 and question 02 respectively.

[0147] Specifically, the stem text D1 corresponding to question 01 is "01| Fill - in - the - blank questions (1) 4.35 is read as (), eight point zero four is written as ().". The answer text E1 is the handwritten "four point three five 8.04".

[0148] The stem text D2 corresponding to question 02 is "02| Calculation questions 1.2 + 2.3 = ().". The answer text E2 is the handwritten "3.5".

[0149] In step S740, search the question - bank content according to the stem text of question 01 and the stem text of question 02 respectively, determine the question numbers 1 and 2 with the highest matching degree in the question bank, obtain the standard - answer information corresponding to question numbers 1 and 2, and use the standard - answer information corresponding to questions 1 and 2 in the question bank as the correct - answer information for question 01 and question 02 in the layout picture.

[0150] In step S750, the answer texts of Question 01 and Question 02 are compared with the correct answer information of Question 01 and Question 02 respectively, the grading information of Question 01 and Question 02 is determined, and the homework grading is ended.

[0151] The technical solution of the embodiment of the present invention obtains a layout image, determines multiple answer sub-images and corresponding question area coordinates according to the layout image, determines the first text information and the second text information in the layout image, determines the stem text and the answer text of each question according to the question area coordinates, the first text information and the second text information, determines the corresponding correct answer information according to the stem text of each question, and determines the correction information of each question according to the correct answer information. In this way, the determination and automatic correction of the stem text and the answer text in the homework layout image are realized, and the automation and efficiency of homework correction are improved.

[0152] Figure 8 Schematic diagram of a data processing device according to an embodiment of the present invention. Figure 8 As shown, the data processing device 8 of the embodiment of the present invention includes an acquisition unit 80 , a first determination unit 81 , a second determination unit 82 , a third determination unit 83 , a matching unit 84 and a correction unit 85 .

[0153] The acquisition unit 80 is used to acquire the layout image.

[0154] In this embodiment, the layout image shows a plurality of answer sub-areas, each of which includes printed question stem text and handwritten answer text, and separator marks are provided between adjacent answer sub-areas.

[0155] In an optional implementation, the separation marks between adjacent answer sub-areas are set as closed dashed question frames, and the dashed question frames are displayed at the outer position of each answer sub-area.

[0156] The first determining unit 81 is used to determine a plurality of answer sub-pictures and corresponding question area coordinates according to the layout picture.

[0157] In this embodiment, a plurality of answer sub-images correspond to different answer sub-areas, respectively, and the question area coordinates are used to represent the position information of each answer sub-area on the layout image.

[0158] Preferably, the first determining unit 81 includes a first layout subunit 810 , a second layout subunit 811 , a third layout subunit 812 , an analyzing subunit 813 and a fourth layout subunit 814 .

[0159] The first layout subunit 810 is used to adjust the size of the layout image to a predetermined size to obtain a first layout image.

[0160] The second layout subunit 811 is used to perform image binarization on the first layout picture to obtain a second layout picture.

[0161] The third layout subunit 812 is used to perform an image closing operation on the second layout picture to obtain a third layout picture.

[0162] The analysis subunit 813 is used to perform connected domain analysis and screening on the third page image to obtain the title area frame.

[0163] The fourth layout sub-unit 814 is used to adjust the question area frame size to the original size, and determine the coordinates of each answer sub-image and the corresponding question area.

[0164] The second determining unit 82 is used to determine the first text information and the second text information in the layout image.

[0165] In this embodiment, the first text information includes the first text coordinates and the first text sequence, and the second text information includes the second text coordinates and the second text sequence, wherein the first text sequence is the printed text content in the layout image. The first text coordinates are the position information of the first text sequence on the layout image. The second text sequence is the handwritten content in the layout image. The second text coordinates are the position information of the second text sequence on the layout image.

[0166] In an optional implementation, the second determination unit 82 includes a segmentation subunit 820 and a detection and identification subunit 821 .

[0167] The segmentation subunit 820 is used to input the layout image into the text segmentation network to obtain multiple feature vectors.

[0168] In an optional implementation, the backbone network structure of the text segmentation network in this embodiment is resnet18+FPN.

[0169] The detection and recognition subunit 821 is used to perform text detection and text recognition on multiple feature vectors, and obtain a first text sequence and a second text sequence as well as a first text coordinate and a second text coordinate.

[0170] In this embodiment, the first text coordinates are pixel-level coordinates of the printed text sequence in the layout image; the second text coordinates are pixel-level coordinates of the handwritten text sequence in the layout image.

[0171] Preferably, the text recognition network of this embodiment adopts a CRNN+CTC network. The text content on the layout image is recognized by the CRNN+CTC network, and a first text sequence of printed text and a second text sequence of handwritten text are obtained.

[0172] The third determining unit 83 is used to determine the question stem text and answer text of each question according to the coordinates of each question area, the first text information and the second text information.

[0173] Preferably, the third determining unit 83 of this embodiment includes a screening subunit 830 and a determining subunit 831 .

[0174] The screening subunit 830 is used to analyze and screen the first text information and the second text information according to the topic area coordinates, the first text coordinates and the second text coordinates of each topic.

[0175] The determination subunit 831 is used to use the first text sequence and the second text sequence that meet the preset conditions as the question stem text and the answer text of each question.

[0176] The matching unit 84 is used to determine the corresponding correct answer information according to the question stem text of each question.

[0177] In an optional implementation, the matching unit 84 of this embodiment includes a searching subunit 840 and an obtaining subunit 841 .

[0178] The search subunit 840 is used to search the question bank content according to the question stem text of each question to determine the question identifier with the highest matching degree.

[0179] Preferably, in this embodiment, the questions in the question bank are searched by fuzzy matching, and the question number with the highest boundary distance similarity is used as the question identifier with the highest matching degree.

[0180] The obtaining subunit 841 is used to obtain the standard answer information corresponding to the question identifier, and use the standard answer information as the correct answer information.

[0181] The correction unit 85 is used to determine the correction information of each question according to the correct answer information.

[0182] Preferably, in this embodiment, the answer text of each question is compared with the correct answer information to determine the correction information of each question.

[0183] Fig. 9 Schematic diagram of an electronic device according to an embodiment of the present invention. Fig. 9 As shown, Fig. 9The electronic device shown is a general address query device, which includes a general computer hardware structure, which at least includes a processor 91 and a memory 92. The processor 91 and the memory 92 are connected via a bus 93. The memory 92 is suitable for storing instructions or programs executable by the processor 91. The processor 91 can be an independent microprocessor or a collection of one or more microprocessors. Thus, the processor 91 executes the instructions stored in the memory 92, thereby executing the method flow of the embodiment of the present invention as described above to realize the processing of data and the control of other devices. The bus 93 connects the above-mentioned multiple components together, and at the same time connects the above-mentioned components to the display controller 94 and the display device and the input / output (I / O) device 95. The input / output (I / O) device 95 can be a mouse, a keyboard, a modem, a network interface, a touch input device, a somatosensory input device, a printer, and other devices known in the art. Typically, the input / output device 95 is connected to the system through an input / output (I / O) controller 99.

[0184] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices (equipment) or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may adopt a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0185] The present application is described with reference to flowcharts of methods, apparatuses (devices) and computer program products according to embodiments of the present application. It should be understood that each process in the flowchart can be implemented by computer program instructions.

[0186] These computer program instructions may be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device that implements the process Figure 1 A function specified in a process or multiple processes.

[0187] These computer program instructions may also be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce the instructions for implementing the process Figure 1 A device that specifies functions in a process or multiple processes.

[0188] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program, wherein the computer-readable program is used for a computer to execute part or all of the above method embodiments.

[0189] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by specifying relevant hardware through a program, and the program is stored in a storage medium, including several instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0190] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data processing method, It is characterized in that The method comprises: Obtaining a layout image, wherein the layout image displays a plurality of answer sub-areas, wherein the answer sub-areas include a first text and a second text, and a separator mark is set between adjacent answer sub-areas; Determine a plurality of answer sub-pictures and corresponding question area coordinates according to the layout image, wherein the plurality of answer sub-pictures correspond to different answer sub-areas respectively; determine first text information and second text information in the layout image, wherein the first text information includes first text coordinates and a first text sequence, and the second text information includes second text coordinates and a second text sequence; Determine the first text and the second text of each question according to the coordinates of the question area, the first text information and the second text information; Determining corresponding third text information according to the first text of each topic; and Determining fourth text information of each topic according to the third text information; Wherein, the determining the first text information and the second text information in the layout image includes: Inputting the layout image into a text segmentation network to obtain a plurality of feature vectors; and Performing text detection and text recognition on the multiple feature vectors to obtain a first text sequence and a second text sequence as well as a first text coordinate and a second text coordinate; The idea of ​​clustering is adopted to perform text detection on the multiple feature vectors through similarity vectors; the first text sequence is the printed text content in the layout image, and the first text coordinates are the position information of the first text sequence on the layout image; the second text sequence is the handwritten content in the layout image, and the second text coordinates are the position information of the second text sequence on the layout image; Among them, the backbone network structure of the text segmentation network is a residual network model plus an FPN model, and the FPN model performs feature fusion on multiple pyramid feature maps output by the residual network model to obtain the multiple feature vectors, and the multiple feature vectors include text area feature vectors, convolution kernel feature vectors and similarity feature vectors, wherein the text area feature vector includes a 1-dimensional printed text area feature vector and a 1-dimensional handwritten text area feature vector; the convolution kernel feature vector includes a 1-dimensional printed convolution kernel feature vector and a 1-dimensional handwritten convolution kernel feature vector; the similarity feature vector includes a 4-dimensional printed similarity vector and a 4-dimensional handwritten similarity vector; the third text information is the correct answer information of each question; and the fourth text information is the correction information of each question.

2. The data processing method according to claim 1, It is characterized in that The separation mark is a closed dotted question frame displayed outside the plurality of question answering sub-areas; Determining multiple answer sub-pictures and corresponding question area coordinates according to the layout picture includes: Adjusting the size of the layout image to a predetermined size to obtain a first layout image; Binarizing the first layout image to obtain a second layout image; Performing an image closing operation on the second layout image to obtain a third layout image; Performing connected domain analysis and screening on the third page image to obtain a title area frame; and Adjust the size of the question area frame to the original size, and determine the coordinates of each of the answer sub-pictures and the corresponding question area; The question area coordinates are used to represent the position information of each answer sub-area on the layout image.

3. The data processing method according to claim 1, It is characterized in that The text recognition adopts a convolutional neural network model.

4. The data processing method according to claim 1, It is characterized in that The determining of the first text and the second text of each question according to the coordinates of the question area, the first text information and the second text information is specifically: Analyzing and screening the first text information and the second text information according to the topic area coordinates, the first text coordinates and the second text coordinates of each of the topics; and The first text sequence and the second text sequence that meet the preset conditions are used as the first text and the second text of each topic.

5. The data processing method according to claim 1, It is characterized in that The third text information corresponding to the first text of each question is specifically determined as follows: Searching the question bank content according to the first text of each question to determine the question identifier with the highest matching degree; and Obtain standard answer information corresponding to the question identifier, and use the standard answer information as the third text information.

6. The data processing method according to claim 5, It is characterized in that The searching of the question bank content according to the first text of each question to determine the question identifier with the highest matching degree is specifically: The questions in the question bank are searched by fuzzy matching, and the question identifier with the highest boundary distance similarity is used as the question identifier with the highest matching degree.

7. The data processing method according to claim 5, It is characterized in that The fourth text information for determining each topic according to the third text information is specifically: The second text of each question is compared with the third text information to determine the fourth text information of each question.

8. A data processing device, It is characterized in that The device comprises: An acquisition unit is used to acquire a layout image, wherein the layout image displays a plurality of answer sub-areas, wherein the answer sub-areas include a first text and a second text, and a separation mark is set between adjacent answer sub-areas; A first determining unit is used to determine a plurality of answer sub-pictures and corresponding question area coordinates according to the layout image, wherein the plurality of answer sub-pictures correspond to different answer sub-areas respectively; A second determining unit, used to determine first text information and second text information in the layout image, wherein the first text information includes first text coordinates and a first text sequence, and the second text information includes second text coordinates and a second text sequence; A third determining unit, configured to determine the first text and the second text of each question according to the coordinates of the question area, the first text information and the second text information; A matching unit, configured to determine corresponding third text information according to the first text of each question; and A correction unit, used for determining fourth text information of each question according to the third text information; The second determination unit includes a segmentation subunit and a detection and recognition subunit. The segmentation subunit is used to input the layout image into the text segmentation network to obtain multiple feature vectors; the detection subunit is used to perform text detection and text recognition on the multiple feature vectors to obtain the first text sequence and the second text sequence as well as the first text coordinate and the second text coordinate; The detection subunit uses the clustering concept to perform text detection on the multiple feature vectors through the similarity vector; the first text sequence is the printed text content in the layout image, and the first text coordinates are the position information of the first text sequence on the layout image; the second text sequence is the handwritten content in the layout image, and the second text coordinates are the position information of the second text sequence on the layout image; Among them, the backbone network structure of the text segmentation network is a residual network model plus an FPN model, and the FPN model performs feature fusion on multiple pyramid feature maps output by the residual network model to obtain the multiple feature vectors, and the multiple feature vectors include text area feature vectors, convolution kernel feature vectors and similarity feature vectors, wherein the text area feature vector includes a 1-dimensional printed text area feature vector and a 1-dimensional handwritten text area feature vector; the convolution kernel feature vector includes a 1-dimensional printed convolution kernel feature vector and a 1-dimensional handwritten convolution kernel feature vector; the similarity feature vector includes a 4-dimensional printed similarity vector and a 4-dimensional handwritten similarity vector; the third text information is the correct answer information of each question; and the fourth text information is the correction information of each question.

9. An electronic device comprising a memory and a processor, It is characterized in that The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer readable storage medium having computer program instructions stored thereon, It is characterized in that The computer program instructions implement the method according to any one of claims 1 to 7 when executed by a processor.