Homework correction method and device based on image recognition

Abstract

The invention belongs to the field of image processing, and particularly provides a homework correction method and device based on image recognition, and the method comprises the steps: obtaining a student homework image from a student end, carrying out the preprocessing of the student homework image, obtaining a plurality of target region images, sequentially taking the plurality of target region images as the input of an image text recognition model, and carrying out the recognition of the image text. Performing text detection and recognition on the plurality of target area images by using an image text recognition model, outputting recognition information of each target area image, and taking the recognition information as answer content of the target area image; the answer content of the target area image is compared with the preset answer content retrieved by using the target area image identification information, the difference part in the comparison result is marked, the answer content with the marking information is output at the teacher end, the marking of the difference part helps a teacher to quickly position the error of the student, and the student experience is improved. And therefore, the workload of the teacher can be reduced, and the correction efficiency can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a homework marking method and device based on image recognition. Background Art

[0002] In the current homework grading method, teachers generally need to manually grade the homework submitted by students. This manual grading of homework will bring a relatively large workload to teachers and bring a burden to their teaching work; especially when there are many students in the class, teachers need to take out a lot of extra time to manually grade homework, which is very inefficient.

[0003] In related technologies, with the rapid development of artificial intelligence, some researchers have also adopted image recognition technology to identify the content of students' homework so that teachers can use computers to correct the homework; this method is essentially to perform text recognition on the homework images uploaded by students, generate an online homework text, and the teacher corrects the text online.

[0004] However, in actual scenarios, text recognition on homework images uploaded by students is greatly challenged due to the deformation of the homework paper, the interference of the shooting scene when students take pictures of the homework books with terminals such as mobile phones, and the perspective distortion caused by the imperfect shooting angle. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a method and device for grading homework based on image recognition, aiming to solve the technical problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions.

[0007] An embodiment of the present invention provides a method for marking homework based on image recognition, comprising the following steps: Acquire a student homework image from a student terminal, pre-process the student homework image, and obtain a plurality of target area images, wherein the target area image includes identification information, the identification information is used to represent the question information corresponding to the target area image, and use the identification information to retrieve the preset answer content of the question information; Construct an image text recognition model, the image text recognition model includes an encoder and a decoder, the encoder is integrated with an asymmetric convolution residual block and a hole convolution residual block, the asymmetric convolution residual block is used to extract features, and the hole convolution residual block is used to increase the receptive field; the decoder adopts a structure based on the Transformer network model, which is formed by stacking a multi-head attention layer and a feedforward neural network FNN; The plurality of target area images are sequentially used as inputs of the image text recognition model, the image text recognition model is used to perform text detection and recognition on the plurality of target area images, recognition information of each target area image is output, and the recognition information is used as the answer content of the target area image; The answer content of the target area image is compared with the preset answer content retrieved using the target area image identification information, the difference parts in the comparison result are marked, and the answer content with the marked information is output on the teacher side.

[0008] Furthermore, the step of preprocessing the student homework image includes: Performing region segmentation processing on the student homework image to obtain multiple region images, wherein the multiple region images include question regions and answer regions corresponding to the question regions one by one; Correction processing is performed on the regional image of the answering area to obtain a plurality of target regional images.

[0009] Furthermore, the step of correcting the regional image of the answering area to obtain multiple target regional images includes: The perspective transformation matrix is ​​calculated using the four vertex coordinates and target coordinates given in the region image, expressed as: ; Where (x, y) represents the coordinate point of the regional image; a represents the element of the perspective transformation matrix; The target coordinates of the transformation are The expanded representation is: ; Obtain the transformed target coordinate point through the vertices of the text line, where the inclination angle of the text line is calculated first: ; In the formula, θ represents the tilt angle, , Represent the coordinates of the upper left and upper right vertices of the text line respectively; then, according to the obtained inclination angle θ, transform to the target coordinate point in the horizontal position , the transformation matrix is ​​calculated using the eight coordinate points before and after the transformation, and the text line is transformed to the horizontal direction through the matrix to obtain the target area image for correction processing.

[0010] Furthermore, in the asymmetric convolution residual block, the input features are added to obtain the output features by convolution results of two channels, wherein one channel is a 7×5 asymmetric downsampling convolution layer and a 3×3 convolution layer, and the other channel is a 7×5 asymmetric downsampling convolution layer; In the dilated convolution residual block, the calculation formula of the receptive field of the dilated convolution residual block is expressed as: ; ; In the formula, Represents the receptive field value of the l-th layer of hole convolution residual block; represents the receptive field value of the l-1th layer of the dilated convolution residual block; l represents the number of layers, and l≥2; Represents the step size of the l-1th layer of dilated convolution residual block; Represents the convolution kernel size of the l-th layer of the dilated convolution residual block; is the receptive field value represented by 2D; and They represent the horizontal receptive field value and the vertical receptive field value respectively.

[0011] Furthermore, the structure of the decoder also includes a position encoding and word embedding layer structure for processing the output data of the encoder, and processes the output data from the encoder; and also includes a linearization layer and a Softmax layer for processing the data to be output from the decoder, and after the decoded feature vector passes through the fully connected layer of the Softmax activation function, the recognition information of the target area image is output.

[0012] Furthermore, in the decoder structure based on the Transformer network model, the input of the multi-head attention layer includes the query vector Q, the key vector K and the value vector V. , ,d=512,d represents the dimension, represents the number of Q; , , , t is the number of key vectors K and numeric vectors V; T represents the transpose operation of the vector; R represents the set of real numbers; multi-head attention is expressed as: ; In the formula, , , , α represents the attention weight; The multiple dot product attention results of the multi-head attention mechanism are concatenated, and then the value obtained after linear transformation is used as the output of the multi-head attention layer, expressed as: ; ; In the formula, represents multi-head attention operation; Represents a concatenation operation; where the parameter , , , ; d represents the dimension, h represents the number of heads; Represents the weight matrix of linear transformation after concatenating the attention results of multiple heads; Represents the linear transformation weight matrix of the query vector Q of the i-th head in the multi-head attention mechanism; Represents the linear transformation weight matrix of the key vector K of the i-th head in the multi-head attention mechanism; Represents the linear transformation weight matrix of the numerical vector V of the i-th head in the multi-head attention mechanism.

[0013] Furthermore, the feedforward neural network FNN includes two linear transformations, which are expressed as: ; Among them, the first linear transformation is expressed as: ; ReLu activation function, which is used to set all negative values ​​in the output of the first linear transformation to 0; The second linear transformation is expressed as: ; In the formula, represents the weight matrix of the first linear transformation; represents the weight matrix of the second linear transformation, represents the bias of the first linear transformation; represents the bias of the second linear transformation; x represents the input vector.

[0014] Furthermore, the steps of the homework marking method based on image recognition also include the following steps: Obtaining the teacher's correction result on the answer content with the marked information, wherein the correction result includes correction information and correction mark; Performing data statistics according to the correction information to obtain corresponding statistical results, and displaying the statistical results and the correction marks on a visual interface of the teacher's end; In response to the push instruction from the teacher terminal, the statistical results and the correction marks are pushed to the student terminal for display.

[0015] Another embodiment of the present invention provides a homework grading device based on image recognition, comprising the following modules: An image processing module is used to obtain a student homework image from a student terminal, pre-process the student homework image, and obtain a plurality of target area images, wherein the target area image includes identification information, the identification information is used to represent the question information corresponding to the target area image, and the preset answer content of the question information is retrieved using the identification information; A model building module is used to build an image text recognition model, wherein the image text recognition model includes an encoder and a decoder, wherein the encoder is integrated with an asymmetric convolution residual block and a hole convolution residual block, wherein the asymmetric convolution residual block is used to extract features, and the hole convolution residual block is used to increase the receptive field; the decoder adopts a structure based on a Transformer network model, which is formed by stacking a multi-head attention layer and a feedforward neural network FNN; An image recognition module is used to sequentially use the multiple target area images as inputs of an image text recognition model, perform text detection and recognition on the multiple target area images using the image text recognition model, output recognition information of each target area image, and use the recognition information as the answer content of the target area image; The homework correction module is used to compare the answer content of the target area image with the preset answer content retrieved using the target area image identification information, mark the different parts in the comparison result, and output the answer content with the marked information on the teacher side.

[0016] Compared with the prior art, the beneficial effects of the homework grading method and device based on image recognition of the present invention are: First, the present invention uses an image text recognition model to identify the answer area in the student's homework image, and can obtain the answer content of the student's homework, compare the answer content of the target area image with the preset answer content retrieved using the target area image identification information, mark the difference part in the comparison result, and output the answer content with the marked information on the teacher's side. The difference marking helps teachers quickly locate students' mistakes, which is convenient for teachers to correct homework, can reduce teachers' workload and improve correction efficiency; Second, the image text recognition model constructed by the present invention combines asymmetric convolution residual blocks and dilated convolution residual blocks to extract features from images. Asymmetric convolution can capture directional information in images, solving the problem of low text recognition accuracy caused by changes in image direction and shape. Dilated convolution residual blocks can capture a wider range of features in images by increasing the receptive field, which helps to recognize text in images. Third, the image text recognition model decoder of the present invention is built based on the Transformer network model. Through the structure of multi-head attention layer and feedforward neural network FNN, it can better understand the contextual relationship of the text in image text recognition and improve the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0018] Figure 1 This is a flow chart of an implementation method of a homework correction method based on image recognition according to the present invention; Figure 2 A sub-flow chart of the homework grading method based on image recognition of the present invention; Figure 3is another sub-flow chart of the homework grading method based on image recognition of the present invention; Figure 4 A schematic diagram of the structure of the asymmetric convolution residual block provided by the present invention; Figure 5 Schematic diagram of the structure of the dilated convolution residual block of the present invention; Figure 6 It is a structural block diagram of a homework correction device based on image recognition according to the present invention; Figure 7 The present invention is a structural block diagram of a terminal device. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions.

[0021] In the related automated homework grading technology, image recognition technology is often used to identify the content in students' homework images so that teachers can use computers to grade homework; this method is essentially to perform text recognition on the homework images uploaded by students, generate an online homework text, and the teacher grades the text online. In actual scenarios, due to the deformation of the homework paper, the interference of the shooting scene when students use mobile phones and other terminals to shoot the homework images, and the perspective distortion caused by the unsatisfactory shooting angle, text recognition on the homework images uploaded by students faces great challenges.

[0022] The specific implementation of the homework grading method and device based on image recognition of the present invention is described in detail below in conjunction with specific embodiments.

[0023] In one embodiment of the present invention, a homework grading method based on image recognition is provided. The homework grading method of the embodiment of the present invention, based on the technology of text recognition of student homework images, identifies the answer content of student homework, facilitates teachers to correct student homework online, and improves teachers' efficiency in correcting student homework.

[0024] For details, please refer to Figure 1 The homework marking method based on image recognition of the present invention comprises the following steps: S101, obtaining a student homework image from a student terminal, preprocessing the student homework image to obtain a plurality of target area images, wherein the target area image includes identification information, the identification information is used to represent the question information corresponding to the target area image, and the preset answer content of the question information is retrieved using the identification information; In step S101 of the embodiment of the present invention, the identification information may be a code of the question information, and each question information corresponds to a unique code, so that the preset question information and the preset answer content corresponding to the question information can be retrieved from the homework database by using the identification information; In addition, the student terminal of this embodiment is an intelligent terminal with a shooting function used by students, which can be various electronic devices with image processing capabilities and data processing capabilities, including but not limited to smart phones, tablet computers, laptop portable computers and desktop computers, etc.; Furthermore, the obtained student homework images are preprocessed as follows, please refer to Figure 2 The steps of preprocessing the student homework image in the embodiment of the present invention include: S201: performing region segmentation processing on the student homework image to obtain a plurality of region images, wherein the plurality of region images include question regions and answer regions corresponding to the question regions one by one; It can be understood that each student homework image generally includes multiple questions and multiple answer areas corresponding to the questions, and the students answer the questions in the answer areas; wherein, the embodiment of the present invention can obtain multiple area images by segmenting the student homework image, and for a single area image, there is only one question and an answer area corresponding to the question, and the answer content written by the student is written in the answer area; S202: performing correction processing on the regional image of the answering area to obtain a plurality of target regional images; Correction processing of the regional image can help to more accurately identify the answer content of the regional image in the subsequent step. Based on this, the present invention corrects the regional image of the answer area to obtain a plurality of target regional images, which are used to perform horizontal correction on the written text. Specifically, perspective transformation is used to correct the tilted text line. The correction processing steps include: using the four vertex coordinates given in the regional image and the target coordinates to calculate the perspective transformation matrix, which is expressed as: ; Where (x, y) represents the coordinate point of the regional image; a represents the element of the perspective transformation matrix; Furthermore, in the correction process, the target coordinates of the transformation are The expanded representation is: ; Obtain the transformed target coordinate point through the vertices of the text line, where the inclination angle of the text line is calculated first: ; In the formula, θ represents the tilt angle, , Represent the coordinates of the upper left and upper right vertices of the text line respectively; then, according to the obtained inclination angle θ, transform to the target coordinate point in the horizontal position , the transformation matrix is ​​calculated using the eight coordinate points before and after the transformation, and the text line is transformed to the horizontal direction through the matrix to obtain the target area image for correction processing.

[0025] Please continue to refer to Figure 1 The steps of the homework marking method based on image recognition of the present invention also include: S102, constructing an image text recognition model, the image text recognition model includes an encoder and a decoder, the encoder is integrated with an asymmetric convolution residual block and a hole convolution residual block, the asymmetric convolution residual block is used to extract features, and the hole convolution residual block is used to increase the receptive field; the decoder adopts a structure based on a Transformer network model, which is formed by stacking a multi-head attention layer and a feedforward neural network FNN; Further, such as Figure 4 As shown, in the asymmetric convolution residual block, the input features are added by the convolution results of two channels to obtain the output features, wherein one channel is a 7×5 asymmetric downsampling convolution layer and a 3×3 convolution layer, and the other channel is a 7×5 asymmetric downsampling convolution layer; In the dilated convolution residual block, the calculation formula of the receptive field of the dilated convolution residual block is expressed as: ; ; In the formula, Represents the receptive field value of the l-th layer of hole convolution residual block; represents the receptive field value of the l-1th layer of the dilated convolution residual block; l represents the number of layers, and l≥2; Represents the step size of the l-1th layer of dilated convolution residual block; Represents the convolution kernel size of the l-th layer of the dilated convolution residual block; is the receptive field value represented by 2D; and They represent the horizontal receptive field value and the vertical receptive field value respectively.

[0026] Please refer to Figure 5, an embodiment of the present invention provides a structure of a dilated convolution residual block, which sequentially passes through a dilated convolution layer, a BN layer, a ReLU layer, a dilated convolution layer, and a BN layer for an input feature vector, and then adds the input feature vector itself. Finally, the output obtained by passing through the ReLU layer is used as the input vector of the next layer, wherein the dilated convolution uses a convolution kernel with a size of 3×3, a step size (parameter s in the figure) of 1, and a dilation rate (parameter d in the figure) of 2.

[0027] The image text recognition model constructed by the present invention combines asymmetric convolution residual blocks and hole convolution residual blocks to extract features in the image. Asymmetric convolution can capture directional information in the image, solving the problem of low text recognition accuracy caused by changes in image direction and shape; the hole convolution residual block can capture a wider range of features in the image by increasing the receptive field, which is helpful to recognize the text in the image.

[0028] In addition, the decoder structure of the embodiment of the present invention also includes a position coding and word embedding layer structure for processing the output data of the encoder, and processes the output data from the encoder; and also includes a linearization layer and a Softmax layer for processing the data to be output of the decoder, and after the decoded feature vector passes through the fully connected layer of the Softmax activation function, the recognition information of the target area image is output.

[0029] Furthermore, in the decoder structure based on the Transformer network model, the input of the multi-head attention layer includes the query vector Q, the key vector K and the value vector V. , ,d=512,d represents the dimension, represents the number of Q; , , , t is the number of key vectors K and value vectors V; T represents the transpose operation of the vector; R represents the set of real numbers; Multi-head attention is expressed as: ; In the formula, , , , α represents the attention weight; The multiple dot product attention results of the multi-head attention mechanism are concatenated, and then the value obtained after linear transformation is used as the output of the multi-head attention layer, expressed as: ; ; In the formula, represents multi-head attention operation; Represents a concatenation operation; where the parameter , , , ; d represents the dimension, h represents the number of heads; Represents the weight matrix of linear transformation after concatenating the attention results of multiple heads; Represents the linear transformation weight matrix of the query vector Q of the i-th head in the multi-head attention mechanism; Represents the linear transformation weight matrix of the key vector K of the i-th head in the multi-head attention mechanism; Represents the linear transformation weight matrix of the numerical vector V of the i-th head in the multi-head attention mechanism.

[0030] Further, in a preferred embodiment of the present invention, the feedforward neural network FNN includes two linear transformations, which are expressed as: ; Among them, the first linear transformation is expressed as: ; ReLu activation function, which is used to set all negative values ​​in the output of the first linear transformation to 0; The second linear transformation is expressed as: ; In the formula, represents the weight matrix of the first linear transformation; represents the weight matrix of the second linear transformation, represents the bias of the first linear transformation; represents the bias of the second linear transformation; x represents the input vector.

[0031] Therefore, the image text recognition model decoder of the present invention is constructed based on the Transformer network model. Through the structure of multi-head attention layer and feedforward neural network FNN, it can better understand the contextual relationship of the text in image text recognition and improve the recognition accuracy.

[0032] Furthermore, in the process of constructing the image text recognition model in the embodiment of the present invention, the model network uses the Adam optimizer to control the gradient descent, and adaptively sets the learning rate for different parameters in the model network; wherein the initial learning rate is set to 0.0001, and an exponential decay method is adopted; Specifically, the calculation formula of the learning rate proposed in this embodiment is expressed as: ; In the formula, represents the learning rate, represents the initial learning rate, β represents the multiple of the exponential decay of the learning rate, s represents the step size, represents rounding down; e represents the number of iterations during training.

[0033] Please continue to refer to Figure 1The steps of the homework marking method based on image recognition of the present invention also include: S103, sequentially taking the multiple target area images as inputs of the image text recognition model, using the image text recognition model to perform text detection and recognition on the multiple target area images, outputting recognition information of each target area image, and taking the recognition information as the answer content of the target area image; S104, comparing the answer content of the target area image with the preset answer content retrieved using the identification information of the target area image, marking the difference in the comparison result, and outputting the answer content with the marked information on the teacher side.

[0034] In step S104 of the embodiment of the present invention, the teacher terminal is an intelligent terminal with information entry function used by the teacher. The intelligent terminal of the teacher terminal has a display screen and can be various electronic devices with image processing capabilities and data processing capabilities, including but not limited to smart phones, tablet computers, laptop portable computers and desktop computers, etc.; Furthermore, the student end and the teacher end of the embodiment of the present invention can be communicatively connected via a network; the network may include various types of wired communication links or wireless communication links, for example: the wired communication link includes optical fiber, twisted pair or coaxial cable, and the wireless communication link includes Bluetooth communication link and wireless fidelity (WIreless-FIdelity, Wi-Fi) communication link.

[0035] Therefore, the present invention utilizes the image text recognition model to identify the answer area in the student's homework image, and can obtain the answer content of the student's homework, compare the answer content of the target area image with the preset answer content retrieved using the target area image identification information, mark the difference part in the comparison result, and output the answer content with the marked information on the teacher side. The difference marking helps teachers to quickly locate errors in the student's answer text when correcting homework, which is convenient for teachers to correct homework, can reduce teachers' workload and improve correction efficiency.

[0036] Further, such as Figure 3 As shown, the steps of the homework grading method based on image recognition provided by the present invention, after the answer content with annotation information is output on the teacher side, also include the following steps: S301: obtaining the grading result of the answer content with annotation information on the teacher side, the grading result including the grading information and the grading mark; S302: performing data statistics according to the grading information to obtain corresponding statistical results, and displaying the statistical results and the grading mark on the visual interface of the teacher side; S303: in response to the push instruction of the teacher side, pushing the statistical results and the grading mark to the student side for display; In the embodiment of the present invention, by feeding back the correction structure to the student terminal, the student can receive the correction result in time through the student terminal, and the student can review and consolidate the corrected homework knowledge points in time to improve the mastery of the knowledge points.

[0037] like Figure 6 As shown, in another embodiment of the present invention, a homework grading device based on image recognition is provided, comprising the following modules: The image processing module 401 is used to obtain the student homework image from the student terminal, pre-process the student homework image, and obtain a plurality of target area images, wherein the target area image includes identification information, the identification information is used to represent the question information corresponding to the target area image, and the preset answer content of the question information is retrieved using the identification information; The model construction module 402 is used to construct an image text recognition model, wherein the image text recognition model includes an encoder and a decoder, wherein the encoder is integrated with an asymmetric convolution residual block and a hole convolution residual block, wherein the asymmetric convolution residual block is used to extract features, and the hole convolution residual block is used to increase the receptive field; the decoder adopts a structure based on a Transformer network model, wherein the structure is formed by stacking a multi-head attention layer and a feedforward neural network FNN; The image recognition module 403 is used to sequentially use the multiple target area images as inputs of the image text recognition model, perform text detection and recognition on the multiple target area images using the image text recognition model, output recognition information of each target area image, and use the recognition information as the answer content of the target area image; The homework correction module 404 is used to compare the answer content of the target area image with the preset answer content retrieved using the target area image identification information, mark the different parts in the comparison result, and output the answer content with the marked information on the teacher side.

[0038] In addition, if Figure 7 As shown, in another embodiment disclosed in the present invention, a terminal device is further provided. The terminal device 500 of the embodiment of the present invention includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the homework grading method based on image recognition according to any one of the above embodiments is implemented. Furthermore, the terminal device may also include: an input system and an output system; the processor, memory, input system and output system may be connected via a bus or other means, the input system may receive input digital or character information, and generate signal input related to the migration of homework correction based on image recognition.

[0039] The output system may include display devices such as display screens.

[0040] Furthermore, the present invention also provides a computer-readable storage medium, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor, the steps of the homework grading method based on image recognition as provided above are implemented.

[0041] In the several embodiments provided by the present invention, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0042] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0043] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0044] In a typical configuration of an embodiment of the present invention, the terminal, the device of the service network and the computing device include one or more processors (CPU), an input / output interface, a network interface and a memory.

[0045] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0046] Computer readable media include permanent and non-permanent, removable and non-removable media and can store information by any method or technology. The information can be computer readable instructions, data structures, program modules or other data.

[0047] Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0048] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0049] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments may still be modified, or some or all of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

[0050] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A homework grading method based on image recognition, It is characterized in that The following steps are involved: Acquire a student homework image from a student terminal, pre-process the student homework image, and obtain a plurality of target area images, wherein the target area image includes identification information, the identification information is used to represent the question information corresponding to the target area image, and use the identification information to retrieve the preset answer content of the question information; Construct an image text recognition model, the image text recognition model includes an encoder and a decoder, the encoder is integrated with an asymmetric convolution residual block and a hole convolution residual block, the asymmetric convolution residual block is used to extract features, and the hole convolution residual block is used to increase the receptive field; the decoder adopts a structure based on the Transformer network model, which is formed by stacking a multi-head attention layer and a feedforward neural network FNN; The plurality of target area images are sequentially used as inputs of the image text recognition model, the image text recognition model is used to perform text detection and recognition on the plurality of target area images, recognition information of each target area image is output, and the recognition information is used as the answer content of the target area image; The answer content of the target area image is compared with the preset answer content retrieved using the target area image identification information, the difference parts in the comparison result are marked, and the answer content with the marked information is output on the teacher side.

2. The homework marking method based on image recognition according to claim 1, It is characterized in that The step of preprocessing the student homework image includes: Performing region segmentation processing on the student homework image to obtain multiple region images, wherein the multiple region images include question regions and answer regions corresponding to the question regions one by one; Correction processing is performed on the regional image of the answering area to obtain a plurality of target regional images.

3. The homework marking method based on image recognition according to claim 2, It is characterized in that The step of correcting the regional image of the answering area to obtain multiple target regional images includes: The perspective transformation matrix is ​​calculated using the four vertex coordinates and target coordinates given in the region image, expressed as: ; Where (x, y) represents the coordinate point of the regional image; a represents the element of the perspective transformation matrix; The target coordinates of the transformation are The expanded representation is: ; Obtain the transformed target coordinate point through the vertices of the text line, where the inclination angle of the text line is calculated first: ; In the formula, θ represents the tilt angle, , Represent the coordinates of the upper left and upper right vertices of the text line respectively; then, according to the obtained inclination angle θ, transform to the target coordinate point in the horizontal position , the transformation matrix is ​​calculated using the eight coordinate points before and after the transformation, and the text line is transformed to the horizontal direction through the matrix to obtain the target area image for correction processing.

4. The homework marking method based on image recognition according to claim 3, It is characterized in that In the asymmetric convolution residual block, the input features are added by convolution results of two channels to obtain output features, wherein one channel is a 7×5 asymmetric downsampling convolution layer and a 3×3 convolution layer, and the other channel is a 7×5 asymmetric downsampling convolution layer; In the dilated convolution residual block, the calculation formula of the receptive field of the dilated convolution residual block is expressed as: ; ; In the formula, Represents the receptive field value of the l-th layer of hole convolution residual block; represents the receptive field value of the l-1th layer of the dilated convolution residual block; l represents the number of layers, and l≥2; Represents the step size of the l-1th layer of dilated convolution residual block; Represents the convolution kernel size of the l-th layer of the dilated convolution residual block; is the receptive field value represented by 2D; and They represent the horizontal receptive field value and the vertical receptive field value respectively.

5. The homework marking method based on image recognition according to claim 4, It is characterized in that The structure of the decoder also includes a position encoding and word embedding layer structure for processing the output data of the encoder, and processes the output data from the encoder; and also includes a linearization layer and a Softmax layer for processing the data to be output from the decoder, and after the decoded feature vector passes through the fully connected layer of the Softmax activation function, the recognition information of the target area image is output.

6. The homework marking method based on image recognition according to claim 5, It is characterized in that In the decoder structure based on the Transformer network model, the input of the multi-head attention layer includes the query vector Q, the key vector K and the value vector V. , ,d=512,d represents the dimension, represents the number of Q; , , , t is the number of key vectors K and value vectors V; T represents the transpose operation of the vector; R represents the set of real numbers; Multi-head attention is expressed as: ; In the formula, , , , α represents the attention weight; The multiple dot product attention results of the multi-head attention mechanism are concatenated, and then the value obtained after linear transformation is used as the output of the multi-head attention layer, expressed as: ; ; In the formula, represents multi-head attention operation; Represents a concatenation operation; where the parameter , , , ; d represents the dimension, h represents the number of heads; Represents the weight matrix of linear transformation after concatenating the attention results of multiple heads; Represents the linear transformation weight matrix of the query vector Q of the i-th head in the multi-head attention mechanism; Represents the linear transformation weight matrix of the key vector K of the i-th head in the multi-head attention mechanism; Represents the linear transformation weight matrix of the numerical vector V of the i-th head in the multi-head attention mechanism.

7. The homework marking method based on image recognition according to claim 6, It is characterized in that The feedforward neural network FNN includes two linear transformations, expressed as: ; Among them, the first linear transformation is expressed as: ; ReLu activation function is used to set all negative values ​​in the output of the first linear transformation to 0; The second linear transformation is expressed as: ; In the formula, represents the weight matrix of the first linear transformation; represents the weight matrix of the second linear transformation, represents the bias of the first linear transformation; represents the bias of the second linear transformation; x represents the input vector.

8. The homework marking method based on image recognition according to claim 7, It is characterized in that The steps of the homework marking method based on image recognition also include the following steps: Obtaining the teacher's correction result on the answer content with the marked information, wherein the correction result includes correction information and correction mark; Performing data statistics according to the correction information to obtain corresponding statistical results, and displaying the statistical results and the correction marks on a visual interface of the teacher's end; In response to the push instruction from the teacher terminal, the statistical results and the correction marks are pushed to the student terminal for display.

9. A device for implementing the homework grading method based on image recognition as claimed in any one of claims 1 to 8, It is characterized in that The device comprises the following modules: An image processing module is used to obtain a student homework image from a student terminal, pre-process the student homework image, and obtain a plurality of target area images, wherein the target area image includes identification information, the identification information is used to represent the question information corresponding to the target area image, and the preset answer content of the question information is retrieved using the identification information; A model building module is used to build an image text recognition model, wherein the image text recognition model includes an encoder and a decoder, wherein the encoder is integrated with an asymmetric convolution residual block and a hole convolution residual block, wherein the asymmetric convolution residual block is used to extract features, and the hole convolution residual block is used to increase the receptive field; the decoder adopts a structure based on a Transformer network model, which is formed by stacking a multi-head attention layer and a feedforward neural network FNN; An image recognition module is used to sequentially use the multiple target area images as inputs of an image text recognition model, perform text detection and recognition on the multiple target area images using the image text recognition model, output recognition information of each target area image, and use the recognition information as the answer content of the target area image; The homework correction module is used to compare the answer content of the target area image with the preset answer content retrieved using the target area image identification information, mark the different parts in the comparison result, and output the answer content with the marked information on the teacher side.

Images