Image processing method, computing device, storage medium and computer program product

Abstract

The embodiment of the invention relates to the technical field of image processing, in particular to an image processing method, computing equipment, a storage medium and a computer program product, and the image processing method comprises the following steps: determining an answer image corresponding to an answer sheet; when it is determined that the size information of the answer sheet does not meet the preset size condition, segmenting the answer image to obtain a plurality of sub-answer images meeting the preset size condition; sequentially determining the plurality of sub-answer images as to-be-processed answer images, and determining to-be-processed areas in the to-be-processed answer images, the to-be-processed areas being used for recording answer correction scores corresponding to questions in the answer sheet; and identifying the answer correction score from the to-be-processed area to obtain a target score corresponding to the answer image.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of image processing technology, and in particular, to image processing methods, computing devices, storage media, and computer program products. Background Art

[0002] In the field of education, in order to more accurately quantify the completion of students' homework and exams, students' homework or test papers are usually graded, and scores are used to represent the students' answers to each question. Usually, when teachers grade students' homework or test papers, they need to calculate the score of each question. Based on this, with the development of image processing technology, it is usually possible to identify the marking marks left by teachers on homework or test papers, so as to obtain the score of each question.

[0003] However, since the position of the marking marks left by teachers on homework or test papers is not fixed, they may overlap with the text of the questions and the students' answer handwriting. In addition, since the paper size may be different during the printing process of homework or test papers, there may also be layout problems such as overlapping and moving of the question area and the answer area, making it difficult to identify the marking marks and further difficult to accurately calculate the scores. Therefore, an effective technical solution is urgently needed to solve the above problems. Summary of the invention

[0004] In view of this, an embodiment of the present specification provides an image processing method. One or more embodiments of the present specification also relate to an image processing apparatus, a computing device, a computer-readable storage medium and a computer program product to solve the technical defects existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, there is provided an image processing method, including: Determine the answer image corresponding to the answer sheet; When it is determined that the size information of the answer sheet does not meet the preset size condition, segmenting the answer image to obtain a plurality of sub-answer images that meet the preset size condition; The plurality of sub-answer images are sequentially determined as answer images to be processed, and an area to be processed in the answer images to be processed is determined, wherein the area to be processed is used to record the answer correction score corresponding to the question in the answer sheet; The answer correction score is identified from the area to be processed, and a target score corresponding to the answer image is obtained.

[0006] According to a second aspect of the embodiments of this specification, there is provided an image processing apparatus, including: A first determination module is configured to determine the answer image corresponding to the answer sheet; A segmentation module is configured to segment the answer image to obtain a plurality of sub-answer images that meet the preset size condition when it is determined that the size information of the answer sheet does not meet the preset size condition; A second determination module is configured to sequentially determine the plurality of sub-answer images as answer images to be processed, and determine an area to be processed in the answer image to be processed, wherein the area to be processed is used to record the answer correction score corresponding to the question in the answer sheet; The recognition module is configured to recognize the answer correction score from the area to be processed and obtain the target score corresponding to the answer image.

[0007] According to a third aspect of an embodiment of this specification, a computing device is provided, including: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the steps of the above-mentioned image processing method are implemented.

[0008] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores a computer program / instruction, and when the computer program / instruction is executed by a processor, the steps of the above-mentioned image processing method are implemented.

[0009] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, comprising a computer program / instruction, which implements the steps of the above-mentioned image processing method when executed by a processor.

[0010] An embodiment of the present specification provides an image processing method, including: determining an answer image corresponding to an answer sheet; when it is determined that the size information of the answer sheet does not meet a preset size condition, segmenting the answer image to obtain a plurality of sub-answer images that meet the preset size condition; sequentially determining the plurality of sub-answer images as answer images to be processed, and determining a pending area in the pending answer image to be processed, wherein the pending area is used to record answer correction scores corresponding to questions in the answer sheet; identifying the answer correction score from the pending area to obtain a target score corresponding to the answer image.

[0011] In the above method, after determining the answer image corresponding to the answer sheet, the size information of the answer sheet can be determined. When it is determined that the size information of the answer sheet does not meet the preset size conditions, the answer image is segmented to obtain multiple sub-answer images that meet the preset size conditions, thereby avoiding inaccurate subsequent determination of the to-be-processed area in the to-be-processed answer image due to different paper sizes during the answer sheet printing process, ensuring the accuracy of determining the to-be-processed area in the to-be-processed answer image, and further achieving the accuracy of identifying the answer correction score from the to-be-processed area, thereby achieving the accuracy of calculating the target score of the answer image. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a schematic diagram of an application scenario of an image processing method provided by an embodiment of this specification; Figure 2 is a flow chart of an image processing method provided by an embodiment of this specification; Figure 3 is a schematic diagram of an answer image in an image processing method provided by an embodiment of this specification; Figure 4 is a schematic diagram of a reference question list image in an image processing method provided in one embodiment of the present specification; Figure 5 It is a schematic diagram of a question sheet matching image in an image processing method provided by an embodiment of this specification; Figure 6 It is an effect diagram of superimposing a reference question list image and a question list matching image in an image processing method provided in one embodiment of this specification; Figure 7 is a processing flow chart of an image processing method provided by an embodiment of this specification; Figure 8 is a structural schematic diagram of an image processing device provided by an embodiment of this specification; Fig. 9 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0013] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0014] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0015] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0016] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0017] In one or more embodiments of this specification, a large model refers to a deep learning model with large-scale model parameters, which usually contains hundreds of millions, tens of billions, hundreds of billions, trillions, or even more than 10 trillion model parameters. A large model can also be called a foundation model / foundation model. The large model is pre-trained with large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks, and the model has good generalization ability, such as a large-scale language model (LLM), a multi-modal pre-training model, etc.

[0018] When the big model is used in practice, only a small number of samples are needed to fine-tune the pre-trained model and it can be applied to different tasks. The big model can be widely used in natural language processing (NLP), computer vision and other fields. Specifically, it can be applied to computer vision tasks such as visual question answering (VQA), image caption (IC), image generation, as well as natural language processing tasks such as text-based sentiment classification, text summary generation, and machine translation. The main application scenarios of the big model include digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc.

[0019] First, the terms involved in one or more embodiments of this specification are explained.

[0020] Target detection model: Target detection model is an algorithm used in the field of computer vision to identify and locate the location and category of one or more targets (objects) in an image. Target detection model can accomplish two tasks at the same time: 1. Identify the targets present in the image and determine what they are (for example, people, cars, cats, etc.); 2. Locate the exact location of these targets in the image, usually by drawing bounding boxes (bbox).

[0021] OCR: Optical Character Recognition, Optical Character Recognition is a technology that allows a computer system to automatically recognize and convert text characters from an image or scanned document into machine-encoded text.

[0022] A4: A common specification of international standard paper size.

[0023] A5: A common specification of international standard paper size.

[0024] A3: A common specification of international standard paper size.

[0025] Affine transformation matrix: is a mathematical tool used to describe geometric transformations in two-dimensional or three-dimensional space. It combines linear transformations (such as rotation, scaling, and translation) with translation operations to deform graphics while keeping parallel lines unchanged.

[0026] HOG: Histogram of Oriented Gradients, a descriptor used in the field of computer vision and image processing, is particularly suitable for object detection tasks.

[0027] YOLO: You Only Look Once is a target detection algorithm. Its core idea is to solve the target detection task as a regression problem. The YOLO algorithm simultaneously predicts the probabilities of multiple bounding boxes and their categories in a single network, thereby achieving fast and efficient target detection.

[0028] IoU: Intersection over Union, is an indicator to measure the overlap degree of two bounding boxes, which is widely used in the field of object detection. It is the ratio of the intersection area of ​​two bounding boxes to the union area, that is, IoU = intersection area / union area.

[0029] IoU variants: including GIoU, DIoU and CIoU. GIoU not only measures the overlapping area, but also measures the distance between the two bounding boxes by considering the area of ​​the minimum circumscribed rectangular box. The value range of GIoU is [-1,1]. When the two images completely overlap, IoU=GIoU=1. When the two images do not intersect, IoU=0 and GIoU=-1. DIoU considers the distance between the center points of the two boxes and is suitable for processing non-overlapping and distant bounding boxes. CIoU also considers the distance between the center points of the two boxes and the similarity of the shapes / sizes of the two bounding boxes, and balances the two through the weight term α.

[0030] Homework score traces: Teacher's scoring information for paper homework.

[0031] Free score mark: The teacher marks the score at random positions in the assignment.

[0032] Score mark in the box: The teacher marks the student in or near the designated mark box.

[0033] Score box: A box used to specify where the teacher is to leave a mark.

[0034] In this specification, an image processing method is provided. This specification also relates to an image processing apparatus, a computing device, a computer-readable storage medium and a computer program product, which are described in detail one by one in the following embodiments.

[0035] See also Figure 1 , Figure 1 A schematic diagram of an application scenario of an image processing method provided according to an embodiment of the present specification is shown.

[0036] Figure 1 It includes a terminal side device 102 and a cloud side device 104.

[0037] In specific implementation, the teacher can select questions from the question database through the terminal device 102, and generate a reference question list image according to the questions and order selected by the teacher. The reference question list image contains the questions selected by the teacher, and generates a to-be-processed area (i.e., a score box) before the question number of each question in the reference question list image, and prints the reference question list image. After obtaining the answer sheet, it is sent to the student for answering. The teacher corrects the answer sheet after the student answers, and takes a photo of the corrected answer sheet through the terminal device 102 to obtain the answer image, and sends the answer image to the cloud device 104. The cloud device 104 can determine whether the answer image meets the preset size condition, segment the answer image that does not meet the preset size condition, obtain multiple sub-answer images that meet the preset size condition, and determine the multiple sub-answer images as the to-be-processed answer image at one time, identify the to-be-processed area for recording the answer correction score from the to-be-processed answer image, and identify the answer correction score from the to-be-processed area, thereby obtaining the target score corresponding to the answer image, and realizing the score calculation of the answer sheet.

[0038] The end-side device 102 may include a browser, an APP (Application), or a web application such as an H5 (Hyper Text Markup Language 5, version 5 of Hypertext Markup Language) application, or a light application (also known as a mini-program, a lightweight application) or a cloud application, etc. The end-side device may be based on a software development kit (SDK) of the corresponding service provided by the server, such as based on the real-time communication (RTC, Real Time Communication) SDK development and acquisition. The end-side device may be deployed in an electronic device and needs to rely on the device to run or some APPs in the device to run. The electronic device may have a display screen and support information browsing, such as a personal mobile terminal such as a mobile phone, a tablet computer, a personal computer, etc. Various other types of applications may also be configured in the electronic device, such as human-computer dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0039] The cloud-side device 104 can be understood as a server that provides various services, including physical servers and cloud servers, such as a server that provides communication services for multiple clients, a server for background training that supports the model used on the client, and a server that processes the data sent by the client. It should be noted that the cloud-side device 104 can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. The cloud-side device 104 can also be a server for a distributed system, or a server combined with a blockchain. The cloud-side device 104 can also be a cloud server for basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDNs), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0040] It is worth noting that the image processing method provided in the embodiments of this specification can be executed by the cloud-side device 104. In other embodiments of this specification, the image processing method provided in the embodiments of this specification can also be executed by the terminal-side device 102. In other embodiments, the image processing method provided in the embodiments of this specification can also be executed jointly by the terminal-side device 102 and the cloud-side device 104.

[0041] See also Figure 2 , Figure 2 A flowchart of an image processing method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0042] Step 202: Determine the answer image corresponding to the answer sheet.

[0043] Specifically, the image processing method provided in the embodiments of this specification can be applied to the server.

[0044] The answer sheet can be understood as a piece of paper used by students to answer questions in an exam or homework. The answer sheet can contain questions to be answered. The answer image can be understood as an image of the student answering the question on the answer sheet and corrected by the teacher. Figure 3 , Figure 3 FIG. 1 shows a schematic diagram of a response image in an image processing method provided according to an embodiment of the present specification, such as Figure 3As shown, the answer image may include a question area, an answer area and a pending area, wherein the question area may be used to display the question, the answer area may be used to fill in the answer to the corresponding question, and the pending area may be used to record the teacher's correction marks on the student's answer, and the correction marks may be the correction score for the answer to the question. It can be understood that each question may correspond to a question area, an answer area and a pending area.

[0045] In an optional embodiment, the size and position of the area to be processed can be set according to actual needs. For example, the area to be processed can be set before the question number of the question in the question area, or the area to be processed can also be set after the question in the question area. The embodiment of this specification does not limit the setting position of the area to be processed. By specifying an area to be processed for each question as a score box for filling in answers and correcting scores, since each question has its own score box as a score trace area, the correction traces in the score trace area are the correction traces of the question, and there is no need to set up complex attribution logic, to achieve auxiliary score collection and trace leaving, thereby eliminating interference from other information, and further improving the accuracy of score frame collection while ensuring that the answer sheet page is clean and tidy.

[0046] Furthermore, the answer sheet can be obtained by printing the reference question sheet image, and the reference question sheet image can include questions to be answered, see Figure 4 , Figure 4 A schematic diagram of a reference question list image in an image processing method provided according to an embodiment of the present specification is shown. Figure 4 As shown, the reference question sheet image may include a question area, an answer area, and a pending area, wherein the question area may be used to display the question, the answer area may be used to fill in the answer to the corresponding question, and the pending area may be used to record the teacher's correction score (i.e., correction trace) for the answer to the question. Unlike the answer image, the reference question sheet image does not have the trace of the student answering the question or the trace of the teacher correcting the student's answer.

[0047] In a specific implementation, determining the answer image corresponding to the answer sheet includes: Receiving an initial answer image sent by a client, wherein the initial answer image is obtained by capturing an image of an answer sheet after answering; The image direction of the initial answer image is adjusted using a preset image direction to obtain the answer image corresponding to the answer sheet.

[0048] The client can be understood as a client for photographing and uploading the answer sheet after the student's answer and teacher's correction, and the client includes but is not limited to scanners, cameras, mobile phones and other devices. The preset image direction can be understood as the positive direction of the text contained in the initial answer image.

[0049] Specifically, the teacher can select questions from the question database, and generate a reference question list image according to the questions and order selected by the teacher. The reference question list image includes the questions selected by the teacher, and generates a to-be-processed area (i.e., a score box) before the question number of each question in the reference question list image, and prints the reference question list image. After obtaining the answer sheet, it is distributed to the students for answering. The teacher corrects the answer sheet after the students answer, and takes a photo of the corrected answer sheet through the client to obtain an initial answer image, and sends the initial answer image to the server. After the server receives the initial answer image, it can adjust the image direction of the initial answer image according to the positive direction of the text contained in the initial answer image, and convert the initial answer image to the positive direction of the text to obtain the answer image corresponding to the answer sheet.

[0050] In practical applications, since the text on the corrected answer sheet may be inverted when photographed or scanned, it is necessary to correct the initial answer image to a direction in which the text is in the forward direction according to the preset image direction. In one embodiment of this specification, the reference question sheet image may include a direction mark, so the initial answer image also includes a direction mark, and the image direction of the initial answer image can be determined by detecting and identifying the direction mark. When it is determined that the image direction of the initial answer image is inverted, the initial answer image is corrected; in another embodiment of this specification, the text direction in the initial answer image can be determined based on OCR recognition technology, and whether the image direction of the initial answer image is inverted can be determined based on the text direction, thereby correcting the inverted initial answer image; in another embodiment of this specification, when the initial answer image is obtained by scanning the corrected answer sheet with a scanner, the scanning direction of the scanner can be set, and the scanner can automatically correct the scanned initial answer image according to the scanning direction. This embodiment of this specification does not limit this.

[0051] In summary, by positively processing the inverted initial answer image, the answer image corresponding to the answer sheet is obtained, which facilitates the subsequent accurate identification of the area to be processed in the answer image, and further realizes the identification of answer correction traces in the area to be processed, thereby ensuring the accuracy of subsequent score calculation.

[0052] Step 204: When it is determined that the size information of the answer sheet does not meet the preset size condition, the answer image is segmented to obtain a plurality of sub-answer images that meet the preset size condition.

[0053] Among them, the size information of the answer sheet can be understood as the paper size of the answer sheet, such as the paper size of the answer sheet is A3 style size, A4 style size or A5 style size, etc. The preset size condition can be understood as whether the paper size of the answer sheet is the size of the preset style. If the size information of the answer sheet does not meet the preset size condition, it can be understood that the paper size of the answer sheet is not the size of the preset style. Optionally, in order to reuse the A4 style image processing process, the preset style can be A4 style. It can be understood that the preset style can also be other styles, such as A3 style, A5 style, etc., which is not limited in the embodiments of this specification.

[0054] Specifically, when it is determined that the paper size of the answer sheet is not of A4 size, the answer image corresponding to the answer sheet may be segmented to obtain a plurality of sub-answer images of A4 size.

[0055] In practical applications, in the field of education, the test papers issued to students may be in A3 style, and the A3 style test papers are composed of two pages of A4 style question images. When the test paper is scanned by a scanner, the shorter side of the answer sheet usually enters the scanner. Therefore, after the aforementioned conversion of the collected initial answer image, in order to reuse the A4 style answer image processing flow, the A3 style answer image needs to be divided into multiple A4 style sub-answer images. Usually, an A3 style answer image can be switched to two A4 style sub-answer images.

[0056] In specific implementation, since the answer image may be an A3-style answer sheet or an A4-style answer sheet, the A3-style answer sheet needs to be segmented. Therefore, whether the answer image needs to be segmented can be determined according to the following judgment logic. The specific implementation method is as follows: When it is determined that the size information of the answer sheet does not meet the preset size condition, segmenting the answer image to obtain a plurality of sub-answer images that meet the preset size condition includes: When it is determined that the size information of the answer sheet does not meet the preset size conditions based on the paper size information corresponding to the answer sheet and / or the paper ratio information corresponding to the answer sheet, the answer image is segmented to obtain multiple sub-answer images that meet the preset size conditions.

[0057] Among them, the paper ratio information can be understood as the ratio of the length and width of the answer sheet.

[0058] In actual applications, the A3 style paper size is larger than the A4 style paper size, and the A3 style paper is longer than the A4 style paper, while the A4 style paper is longer than the A3 style paper. Based on this, it is possible to determine whether the answer sheet is A3 style or A4 style according to the paper size information and / or paper ratio information corresponding to the answer sheet, thereby determining whether the answer image corresponding to the answer sheet needs to be segmented to further obtain multiple sub-answer images in A4 style.

[0059] In summary, by judging the size information of the answer sheet, the answer image of the answer sheet that is not in A4 format is segmented to obtain multiple sub-answer images in A4 format, thereby realizing the reuse of the A4 format image processing flow. There is no need to perform different processing methods for answer images of different styles, which further reduces the computing resources required for image processing.

[0060] In a specific implementation, the segmentation process of the answer image to obtain a plurality of sub-answer images satisfying the preset size condition includes: The answer image is segmented according to a preset segmentation rule to obtain a plurality of sub-answer images that meet the preset size conditions.

[0061] Among them, the preset segmentation rule can be understood as a rule for segmenting answer images of other styles into preset styles (such as A4 style), for example, it can be segmented according to preset segmentation positions in the answer image and / or segmented using marker point detection.

[0062] Specifically, the segmentation process is performed on the answer image according to a preset segmentation rule to obtain a plurality of sub-answer images that meet the preset size condition, including: According to the preset segmentation positions in the answer image, the answer image is segmented to obtain a plurality of sub-answer images satisfying the preset size condition; and / or Mark point detection is performed on the answer image to obtain a detection result, and the answer image is segmented according to the detection result to obtain a plurality of sub-answer images that meet the preset size condition.

[0063] In practical applications, the A3-style answer image can be divided into two A4-style sub-answer images. The preset division position in the answer image can be, for example, the middle position in the answer image. Based on this, the answer image can be divided into two sub-answer images according to the middle position in the answer image.

[0064] Alternatively, the answer image may also include marking points used for segmentation, and the marking point detection may be performed on the answer image to obtain a detection result, and the answer image may be subjected to auxiliary segmentation processing based on the detection result to obtain multiple sub-answer images.

[0065] In summary, by segmenting the answer images according to the above segmentation strategy, it is possible to segment answer images of different styles into sub-answer images of preset styles, which facilitates subsequent processing of the sub-answer images.

[0066] In addition, after determining the answer image corresponding to the answer sheet, the method further includes: When it is determined that the size information of the answer sheet meets the preset size condition, the answer image is determined as the answer image to be processed.

[0067] Specifically, when it is determined that the size information of the answer sheet is the size of the preset style, it means that the size information of the answer sheet meets the preset size condition, and there is no need to perform the above segmentation process. The answer image can be directly determined as the answer image to be processed, and the next image processing step can be performed. For example, when it is determined that the answer sheet is of A4 style, the answer image can be directly determined as the answer image to be processed.

[0068] Step 206: sequentially determine the multiple sub-answer images as answer images to be processed, and determine the area to be processed in the answer image to be processed, wherein the area to be processed is used to record the answer correction scores corresponding to the questions in the answer sheet.

[0069] The answer image to be processed can be understood as each of the multiple answer sub-images, and the area to be processed can be understood as a score box in the answer image to be processed for recording the score of the answer corresponding to the question. It can be understood that the answer image to be processed can contain multiple questions, each of which corresponds to a region to be processed, and the score of the answer to the question is recorded in the region to be processed.

[0070] Specifically, each of the multiple sub-answer images can be used as an answer image to be processed in turn, and a pending area in the answer image to be processed for recording the answer correction score corresponding to the question is determined. For example, in the case where there are two sub-answer images, sub-answer image 1 can be used as the answer image to be processed, and pending area 1 for recording the answer correction score of question 1, pending area 2 for recording the answer correction score of question 2, and pending area 3 for recording the answer correction score of question 3 in the answer image to be processed are determined. Similar to the above, sub-answer image 2 can be used as the answer image to be processed, and pending area 4 for recording the answer correction score of question 4 and pending area 5 for recording the answer correction score of question 5 in the answer image to be processed are determined.

[0071] In a specific implementation, the step of determining the area to be processed in the answer image to be processed includes: Determine a reference question sheet image corresponding to the answer sheet, and determine question sheet matching information corresponding to the reference question sheet image, wherein the question in the reference question sheet image is consistent with the question in the answer sheet, and the question sheet matching information is used to mark the position of each element in the reference question sheet image; According to the reference question list image and the question list matching information, a to-be-processed area in the to-be-processed answer image is determined.

[0072] Among them, the reference question list image can be understood as a question list image containing the same questions as those in the answer image, the elements in the reference question list image can include the question area, the answer area and the area to be processed, and the matching information corresponding to the reference question list image can be understood as information used to mark the positions of the question area, the answer area and the area to be processed in the reference question list image.

[0073] Specifically, a reference question list image containing the same questions as the answer image can be determined, and question list matching information corresponding to the reference question list image can be determined. Based on the reference question list image and the question list matching information, the area to be processed can be determined from the answer image to be processed.

[0074] In actual applications, when determining the reference question sheet image corresponding to the answer sheet and determining the question sheet matching information corresponding to the reference question sheet image, it can be determined based on the association information between the answer image, the reference question sheet image, and the question sheet matching information. The reference question sheet image and the question sheet matching information can be stored in association in a database and obtained using the relevant information of the answer image. Specifically, after receiving the answer image sent by the client, the server can determine the reference question sheet image and the question sheet matching information corresponding to the answer sheet based on the association information specified by the teacher or student through the client, where the association information can be, for example, the homework name, homework number, etc.

[0075] In another embodiment of the present specification, the association information between the answer image, the reference question sheet image and the question sheet matching information can also be printed in the reference question sheet image, or the association information can be encrypted and transcoded and then printed in the reference question sheet image. The association information can be a string, or can be set in the reference question sheet image in the form of a QR code, a barcode, a feature image, etc. Then, the answer image will also include the association information. The server can obtain the association information by scanning and identifying the answer image, and determine the reference question sheet image and the question sheet matching information corresponding to the answer sheet based on the association information.

[0076] Furthermore, the question list matching information may be text information recording the position of each element, or the question list matching information may be a question list matching image, see Figure 5 , Figure 5 FIG. 1 is a schematic diagram showing a question list matching image in an image processing method provided according to an embodiment of the present specification. Figure 5As shown, the question sheet matching image contains different areas, and different areas can be distinguished by different colors, such as the colors of the question area, the answer area and the area to be processed are different. In addition, the question area, the answer area and the area to be processed can also be distinguished by other marks, such as by numbers, digital marks, letter marks, etc., which are not limited to the embodiments of this specification.

[0077] In practical applications, the question list matching image can be generated based on the layout rendering of the reference question list image. When generating the reference question list image, a corresponding solid color frame can be generated in the reference question list image at the location of the area to be processed. The area to be processed can be the red rectangular frame before the question number of each question. The question list matching image and the reference question list image can be generated in a 1:1 ratio to achieve superposition, see Figure 6 , Figure 6 A diagram showing the effect of superimposing a reference question list image and a question list matching image in an image processing method provided according to an embodiment of the present specification is shown.

[0078] In summary, by using the reference question list image and question list matching information to locate the area to be processed in the answer image to be processed, the accuracy of positioning is achieved, and the area to be processed is marked using the question list matching information to improve the robustness of detection and identification of the area to be processed and avoid the impact caused by the area to be processed being too dark or too shallow during the printing process.

[0079] Furthermore, in the field of education, in order to save printing costs, after the reference question sheet image is printed into an answer sheet, the answer sheet is usually photocopied to obtain multiple answer sheets. At this time, the question sheet content may be displaced or tilted, resulting in the inability to correspond one-to-one between the answer image corresponding to the answer sheet and the question position in the reference question sheet image. In this case, the question sheet matching image and the answer image to be processed cannot be directly overlapped, but the reference question sheet image needs to be used as an intermediate reference to realize the positioning of the question sheet matching image. The specific implementation method is as follows: The step of determining the to-be-processed area in the to-be-processed answer image according to the reference question list image and the question list matching information includes: Calculating a mapping matrix between the answer image to be processed and the reference question list image; According to the mapping matrix, the answer image to be processed and the question list matching information are superimposed to obtain a superimposed image; According to the question sheet matching information, the area to be processed in the superimposed image is determined, and the area to be processed in the superimposed image is determined as the area to be processed in the answer image to be processed.

[0080] Specifically, the mapping matrix between the answer image to be processed and the reference question list image can be calculated, and according to the mapping matrix, the answer image to be processed and the question list matching image are superimposed to obtain a superimposed image, and according to the question list matching information, the area to be processed in the superimposed image is determined, and the area to be processed in the superimposed image is determined as the area to be processed in the answer image to be processed.

[0081] In practical applications, the mapping matrix can be an affine transformation matrix. After calculating the mapping matrix between the answer image to be processed and the reference question sheet image, the question sheet matching image and the answer image to be processed can be superimposed according to the mapping matrix to obtain a superimposed image, and the question area and the area to be processed in the superimposed image can be segmented according to the rectangular frame area of ​​the question sheet matching image in the superimposed image. Each question area contains a region to be processed, thereby realizing the positioning of the area to be processed in the answer image to be processed, which is convenient for the subsequent designation of the answer correction score mark position and the auxiliary attribution score mark.

[0082] In summary, the question list matching image is transformed through the mapping matrix. Since the size, proportion and layout of the question list matching image and the reference question list image are consistent, the transformed question list matching image can overlap with the answer image to be processed better, thereby ensuring the accuracy of the subsequent designated answer correction score mark position and assisting in the accuracy of the attribution score mark.

[0083] In a specific implementation, the calculation of the mapping matrix between the answer image to be processed and the reference question sheet image includes: Performing feature extraction on the reference question sheet image to obtain multiple question sheet area features of the reference question sheet image, and performing feature extraction on the answer image to be processed to obtain multiple answer area features of the answer image to be processed; Calculating the regional similarity between the target question area feature and the target answer area feature, wherein the target question area feature is any one of the multiple question area features, and the target answer area feature is any one of the multiple answer area features; According to the regional similarity, determining a plurality of similar regional feature pairs from the plurality of question sheet regional features and the plurality of answer regional features; Based on the multiple similar area feature pairs, a mapping matrix between the answer image to be processed and the reference question sheet image is calculated.

[0084] Among them, a similar area feature pair may include a question area feature and an answer area feature.

[0085] Specifically, feature extraction can be performed in one or more specific response areas in the answer image to be processed to obtain multiple answer area features, feature extraction can be performed in one or more specific response areas in the reference question sheet image to obtain multiple question sheet area features, the regional similarity between the target question sheet area features and the target answer area features can be calculated, and based on the regional similarity, one or more similar area feature pairs whose regional similarities reach a preset similarity threshold can be determined from multiple question sheet area features and multiple answer area features, and a mapping matrix between the answer image to be processed and the reference question sheet image can be calculated based on the multiple similar area feature pairs. Alternatively, similar area feature pairs can be determined from multiple question sheet area features and multiple answer area features based on the comparison results between the regional similarities.

[0086] In practical applications, feature extraction can be performed based on a feature extractor, such as a HOG feature extractor. Furthermore, the same feature extractor can be used to perform feature extraction on the answer image to be processed and the reference question sheet image respectively.

[0087] In summary, by extracting features from the reference question sheet image and the answer image to be processed, the mapping matrix between the reference question sheet image and the answer image to be processed is calculated, which facilitates the subsequent question sheet matching image transformation according to the mapping matrix, and further realizes the identification and positioning of the area to be processed.

[0088] Step 208: Identify the answer correction score from the area to be processed, and obtain the target score corresponding to the answer image.

[0089] The answer correction score can be understood as the answer correction score of the question corresponding to the area to be processed. For example, if the answer correction score in the area to be processed corresponding to question 1 in the answer image to be processed is 5 points, then the score of question 1 is 5 points. The target score can be understood as the total score of the answer sheet corresponding to the answer image. For example, when the answer image is divided into sub-answer image 1 and sub-answer image 2, the score of sub-answer image 1 is 30 points, and the score of sub-answer image 2 is 50 points, then the total score of the answer sheet corresponding to the answer image is 80 points.

[0090] In a specific implementation, the step of identifying the answer correction score from the to-be-processed area and obtaining the target score corresponding to the answer image includes: Determine the answer correction traces corresponding to the area to be processed, and determine the answer correction scores of the questions corresponding to the area to be processed according to the answer correction traces; Determining an initial score corresponding to the answer image to be processed according to the answer correction scores of the questions corresponding to the multiple areas to be processed in the answer image to be processed; According to the initial score corresponding to the answer image to be processed, a target score corresponding to the answer image is determined.

[0091] Among them, the answer correction trace can be understood as the teacher's correction trace on the questions corresponding to the area to be processed. The answer correction trace can be the trace of the answer correction score. For example, if the answer correction trace in the area to be processed is the number 5, then it means that the answer correction score of the question corresponding to the area to be processed is 5 points. Or, if the answer correction trace in the area to be processed is "-5", then it can be said that the answer correction score of the question corresponding to the area to be processed is -5 points, that is, 5 points have been deducted.

[0092] Specifically, the target detection model can be used to identify answer correction traces in the area to be processed, and determine the answer correction score of the question corresponding to the area to be processed based on the answer correction traces; according to the answer correction scores of the questions corresponding to each area to be processed in the answer image to be processed, and according to the answer correction scores returned and collected by the questions, the initial score corresponding to the answer image to be processed is determined, that is, the initial score of each sub-answer image in multiple sub-answer images is determined, and the initial scores of the multiple sub-answer images are added together to obtain the total score of the answer sheet corresponding to the answer image.

[0093] In practical applications, the target detection model can be a large model or a detection and recognition model trained based on the YOLO detection algorithm.

[0094] In summary, by identifying the traces of answer correction in the area to be processed, the answer correction score is determined, which facilitates the subsequent calculation of the total score of the answer image.

[0095] Furthermore, the step of determining the answer correction trace corresponding to the area to be processed includes: Determining at least one answer correction trace contained in the answer image to be processed; Determine a bounding box corresponding to each answer correction trace in the at least one answer correction trace; According to the association relationship between the bounding boxes corresponding to the answer correction traces and the area to be processed, the answer correction trace corresponding to the area to be processed is determined from the at least one answer correction trace.

[0096] Among them, the correlation relationship between the bounding box corresponding to each answer correction trace and the area to be processed can be used to measure the proximity between the bounding box and the area to be processed. The correlation relationship can be the area distance information between the bounding box and the area to be processed. The area distance information can include the corner point distance, the nearest distance or the center distance between the bounding box and the area to be processed. The correlation relationship can also be the IoU between the bounding box and the area to be processed and various IoU variants. The embodiments of this specification do not limit this.

[0097] Specifically, one or more answer correction traces contained in the answer image to be processed can be determined, and the bounding box corresponding to each answer correction trace can be determined. According to the association between the bounding box corresponding to each answer correction trace and the area to be processed, the answer correction trace corresponding to the area to be processed can be determined from the one or more answer correction traces.

[0098] In summary, by calculating the correlation between the bounding box corresponding to each answer correction trace and the area to be processed, the attribution of the answer correction trace is achieved, and the accuracy of adding the scores of multiple questions is further achieved, avoiding the omission of scores or repeated recognition of scores.

[0099] Further, the determining the answer correction trace corresponding to the area to be processed from the at least one answer correction trace according to the association relationship between the bounding box corresponding to each answer correction trace and the area to be processed includes: Determine the association relationship value between the bounding box corresponding to each answer correction trace and the area to be processed according to the area distance information between the bounding box corresponding to each answer correction trace and the area to be processed; According to the association relationship value between the bounding box corresponding to each answer correction trace and the area to be processed, the answer correction trace corresponding to the area to be processed is determined from the at least one answer correction trace.

[0100] Specifically, the regional distance information between the bounding box corresponding to each answer correction trace and the area to be processed can be calculated, and the association relationship value between the bounding box corresponding to each answer correction trace and the area to be processed can be determined based on the regional distance information, and the answer correction trace in the bounding box whose association relationship value is greater than the preset association threshold is determined as the answer correction trace corresponding to the area to be processed. For example, the answer image to be processed contains answer correction trace 1, answer correction trace 2, and answer correction trace 3, then the regional distance information between the bounding box 1 corresponding to the answer correction trace 1 and the area to be processed can be calculated, the regional distance information between the bounding box 2 corresponding to the answer correction trace 2 and the area to be processed can be calculated, and the regional distance information between the bounding box 3 corresponding to the answer correction trace 3 and the area to be processed can be calculated, and based on the above regional distance information, the association relationship value 1 between the bounding box 1 and the area to be processed, the association relationship value 2 between the bounding box 2 and the area to be processed, and the association relationship value 3 between the bounding box 3 and the area to be processed can be calculated, wherein the association relationship value 2 is greater than the preset association threshold, then the answer correction trace 2 in the bounding box 2 can be determined as the answer correction trace corresponding to the area to be processed.

[0101] Furthermore, since only one answer correction score (i.e., score trace ownership) is allowed for an area to be processed, when the association relationship values ​​between multiple bounding boxes and the area to be processed are greater than the preset association threshold, the association relationship values ​​between each bounding box and the area to be processed can be sorted, and the answer correction traces in the bounding boxes with higher association relationship values ​​can be determined as the answer correction traces corresponding to the area to be processed according to the sorting results. Continuing with the above example, if both association relationship value 1 and association relationship value 2 are greater than the preset association threshold, then association relationship value 1 and association relationship value 2 can be sorted and compared, where association relationship value 1 is greater than association relationship value 2, then answer correction trace 1 in bounding box 1 can be determined as the answer correction trace corresponding to the area to be processed.

[0102] In practical applications, further, since each question has a corresponding area to be processed, and the scores corresponding to each question can be different, for example, the score of question 1 is 2 points, and the score of question 2 is 5 points, then an answer correction trace cannot be attributed to different areas to be processed at the same time, that is to say, an answer correction trace can only be attributed to one area to be processed, then, the answer correction trace of the area to be processed that has been determined can be marked for attribution, and the attribution mark is used to indicate that the area to be processed will not participate in the attribution of the answer correction traces of the remaining questions.

[0103] In summary, through the calculation and attribution logic of the above-mentioned association value, the attribution of the answer correction traces is realized, further ensuring that the identified answer correction scores correspond to the questions one by one, avoiding duplication or omission in the subsequent target score calculation, thereby ensuring the accuracy of the target score calculation.

[0104] In actual applications, due to different marking habits of different teachers, some teachers will not record the score of the questions answered correctly by students, but will deduct points for the questions answered incorrectly by students. In this case, there may be no trace of answer correction in the score box. The specific implementation method is as follows: When it is determined that there is no trace of answer correction in the to-be-processed area, determining the question answering area corresponding to the to-be-processed area in the to-be-processed answering image; According to the question answering information in the question answering area, the answer correction score of the question corresponding to the to-be-processed area is determined.

[0105] The question answering information can be understood as the answer content of the student's answer to the question in the question answering area. The question answering area can be understood as an area for answering questions. It can be understood that the answer of the student is recorded in the question answering area, and there is a one-to-one correspondence between the question answering area, the question area and the pending area.

[0106] Specifically, when there are no traces of answer correction in the area to be processed, it means that the area to be processed is blank. Then, the question answer area of ​​the question corresponding to the area to be processed can be determined, and the question answer information in the question answer area can be identified. According to the question answer information and the preset strategy, the answer correction score of the question corresponding to the area to be processed can be determined.

[0107] In practical applications, the question answer information in the question answer area can be identified based on OCR recognition technology. When it is determined that there is question answer information in the question answer area, the answer correction score of the question corresponding to the unprocessed area is determined as the score threshold of the question (i.e., the full mark). When it is determined that there is no question answer information in the question answer area, that is, the question answer area is blank and the student has not answered the question, then the answer correction score of the question corresponding to the unprocessed area can be determined as 0 points.

[0108] In summary, when there are no traces of answer correction in the area to be processed, the above strategy is used to determine the answer correction score based on the question answer information, taking into account the different correction habits of different teachers, and further ensuring the automation and efficiency of answer sheet correction.

[0109] Furthermore, determining the answer correction score of the question corresponding to the area to be processed according to the answer correction trace includes: Identify the score value and score type of the answer correction trace; The answer correction score corresponding to the answer correction trace is calculated according to the score value and the score type, and the answer correction score is determined as the answer correction score of the question corresponding to the area to be processed.

[0110] Among them, the score value can be understood as the specific value of the score, such as 5 points, 2 points, 10 points, etc. The score type can include plus type and minus type. The score type of the answer correction trace is a minus type, which means that the answer correction trace contains a minus sign "-". The score type of the answer correction trace is a plus type, which means that the answer correction trace contains a plus sign "+" or there is no sign before the score value.

[0111] Based on this, it is necessary to identify the score type. If the score type is a deduction type, the score threshold of the question corresponding to the area to be processed (i.e., what is the full score) can be determined, and the score threshold can be subtracted from the score value to obtain the answer correction score for the question. If the score type is a bonus type, the score value can be directly used as the answer correction score for the question.

[0112] In addition, when the answer correction traces in the identified area to be processed do not contain numbers, an abnormal result prompt message can be sent to the teacher's client, waiting for the teacher to confirm through the client, to avoid other interfering content from being misidentified and attributed to the score box, affecting the score result.

[0113] In summary, by identifying the score value and score type of the answer correction traces, the calculation of the answer correction score can be realized, which is compatible with the correction habits of different teachers and ensures the accuracy of the answer correction score calculation.

[0114] Furthermore, after determining the score of the answer to the question corresponding to the area to be processed, the method further includes: When it is determined that the answer correction score of the question corresponding to the to-be-processed area is greater than the score threshold corresponding to the question, the answer correction score is replaced with a preset score to obtain a replaced answer correction score; or When it is determined that the answer correction score of the question corresponding to the area to be processed is greater than the score threshold corresponding to the question, a score verification request is sent to the client, and an updated score returned by the client according to the score verification request is received, and the answer correction score is replaced with the updated score to obtain the replaced answer correction score.

[0115] The preset score can be the score threshold (full score) or zero of the question. If the score of the corrected answer to the question corresponding to the pending area is greater than the score threshold of the question, it means that the score corrected by the teacher for the question exceeds the upper limit of the question score, and there is an abnormality. For example, if the full score of the question is 5 points and the corrected answer score is 6 points, it means there is an abnormality.

[0116] In one embodiment of the present specification, in the case of the above-mentioned abnormality, the answer correction score can be directly replaced with a preset score to obtain the replaced answer correction score. For example, if the score type of the answer correction score is a plus-point type, the answer correction score can be directly replaced with a full score, and if the score type of the answer correction score is a deduction type, the answer correction score can be replaced with zero points.

[0117] In another embodiment of the present specification, a score verification request may also be sent to the teacher's client. The teacher may verify the score of the question through the client and send an updated score to the server through the client. The server may replace the answer correction score with the updated score returned by the client.

[0118] To summarize, in the above method, after determining the answer image corresponding to the answer sheet, the size information of the answer sheet can be determined. When it is determined that the size information of the answer sheet does not meet the preset size conditions, the answer image is segmented to obtain multiple sub-answer images that meet the preset size conditions, thereby avoiding inaccurate subsequent determination of the to-be-processed area in the to-be-processed answer image due to different paper sizes during the printing process of the answer sheet, ensuring the accuracy of determining the to-be-processed area in the to-be-processed answer image, and further achieving the accuracy of identifying the answer correction score from the to-be-processed area, thereby achieving the accuracy of calculating the target score of the answer image.

[0119] The following combination Figure 7 Taking the application of the image processing method provided in this specification in the collection of homework score traces as an example, the image processing method is further explained. Figure 7 A processing flow chart of an image processing method provided by an embodiment of the present specification is shown, which specifically includes the following steps.

[0120] Step 702: Generate a reference question list image and question list matching information corresponding to the reference question list image.

[0121] Specifically, the teacher can select questions from the question database, and generate a reference question list image according to the questions and order selected by the teacher. The reference question list image includes the questions selected by the teacher, and a to-be-processed area (i.e., a score box) is generated before the question number of each question in the reference question list image, and question list matching information corresponding to the reference question list image is generated.

[0122] Step 704: Receive the initial answer image sent by the client.

[0123] Specifically, the teacher can print the reference question sheet image, and after obtaining the answer sheet, distribute it to the students for answering. The teacher corrects the answer sheet after the students have answered, and takes a photo of the corrected answer sheet through the client to obtain the initial answer image, and sends the initial answer image to the server.

[0124] Step 706: adjusting the image direction of the initial answer image using the preset image direction to obtain the answer image.

[0125] Specifically, after receiving the initial answer image, the server can adjust the image direction of the initial answer image according to the positive direction of the text contained in the initial answer image, and convert the initial answer image to a direction with positive text to obtain the answer image corresponding to the answer sheet.

[0126] Step 708: When it is determined that the size information of the answer sheet does not meet the preset size condition, the answer image is segmented to obtain multiple sub-answer images that meet the preset size condition, and the multiple sub-answer images are sequentially determined as answer images to be processed.

[0127] Specifically, when it is determined that the paper size of the answer sheet is not of A4 size, the answer image corresponding to the answer sheet may be segmented to obtain a plurality of sub-answer images of A4 size.

[0128] Step 710: Obtain the reference question list image and question list matching information corresponding to the answer image to be processed.

[0129] Specifically, the reference question list image and question list matching information corresponding to the answer image to be processed can be obtained based on the associated information specified by the teacher or student when uploading the answer image, or the associated information contained in the answer image.

[0130] Step 712: Calculate the mapping matrix between the answer image to be processed and the reference question sheet image.

[0131] Specifically, feature extraction can be performed on one or more specific response areas in the answer image to be processed to obtain multiple answer area features, feature extraction can be performed on one or more specific response areas in the reference question sheet image to obtain multiple question sheet area features, the regional similarity between the target question sheet area features and the target answer area features can be calculated, and based on the regional similarity, one or more similar area feature pairs whose regional similarities reach a preset similarity threshold can be determined from the multiple question sheet area features and the multiple answer area features, and based on the multiple similar area feature pairs, a mapping matrix between the answer image to be processed and the reference question sheet image can be calculated.

[0132] Step 714: According to the mapping matrix, the answer image to be processed and the question sheet matching information are superimposed to obtain a superimposed image.

[0133] Step 716: Determine the area to be processed in the superimposed image according to the question list matching information, and determine the area to be processed in the superimposed image as the area to be processed in the answer image to be processed.

[0134] Step 718: Identify the answer correction score from the area to be processed and obtain the target score corresponding to the answer image.

[0135] Specifically, the target detection model can be used to identify answer correction traces in the area to be processed, and determine the answer correction score of the question corresponding to the area to be processed based on the answer correction traces; according to the answer correction scores of the questions corresponding to each area to be processed in the answer image to be processed, and according to the answer correction scores returned and collected by the questions, the initial score corresponding to the answer image to be processed is determined, that is, the initial score of each sub-answer image in multiple sub-answer images is determined, and the initial scores of the multiple sub-answer images are added together to obtain the total score of the answer sheet corresponding to the answer image.

[0136] To summarize, in the above method, after determining the answer image corresponding to the answer sheet, the size information of the answer sheet can be determined. When it is determined that the size information of the answer sheet does not meet the preset size conditions, the answer image is segmented to obtain multiple sub-answer images that meet the preset size conditions, thereby avoiding inaccurate subsequent determination of the to-be-processed area in the to-be-processed answer image due to different paper sizes during the printing process of the answer sheet, ensuring the accuracy of determining the to-be-processed area in the to-be-processed answer image, and further achieving the accuracy of identifying the answer correction score from the to-be-processed area, thereby achieving the accuracy of calculating the target score of the answer image.

[0137] Corresponding to the above method embodiment, this specification also provides an image processing device embodiment, Figure 8 FIG. 2 shows a schematic diagram of the structure of an image processing device provided by an embodiment of the present specification. Figure 8 As shown, the device comprises: The first determination module 802 is configured to determine the answer image corresponding to the answer sheet; The segmentation module 804 is configured to segment the answer image to obtain a plurality of sub-answer images that meet the preset size condition when it is determined that the size information of the answer sheet does not meet the preset size condition; The second determination module 806 is configured to sequentially determine the plurality of sub-answer images as answer images to be processed, and determine a to-be-processed area in the answer image to be processed, wherein the to-be-processed area is used to record the answer correction score corresponding to the question in the answer sheet; The identification module 808 is configured to identify the answer correction score from the area to be processed and obtain the target score corresponding to the answer image.

[0138] In an optional embodiment, the segmentation module 804 is further configured to: When it is determined that the size information of the answer sheet does not meet the preset size conditions based on the paper size information corresponding to the answer sheet and / or the paper ratio information corresponding to the answer sheet, the answer image is segmented to obtain multiple sub-answer images that meet the preset size conditions.

[0139] In an optional embodiment, the segmentation module 804 is further configured to: The answer image is segmented according to a preset segmentation rule to obtain a plurality of sub-answer images that meet the preset size conditions.

[0140] In an optional embodiment, the segmentation module 804 is further configured to: According to the preset segmentation positions in the answer image, the answer image is segmented to obtain a plurality of sub-answer images satisfying the preset size condition; and / or Mark point detection is performed on the answer image to obtain a detection result, and the answer image is segmented according to the detection result to obtain a plurality of sub-answer images that meet the preset size condition.

[0141] In an optional embodiment, the second determining module 806 is further configured to: When it is determined that the size information of the answer sheet meets the preset size condition, the answer image is determined as the answer image to be processed.

[0142] In an optional embodiment, the second determining module 806 is further configured to: Determine a reference question sheet image corresponding to the answer sheet, and determine question sheet matching information corresponding to the reference question sheet image, wherein the question in the reference question sheet image is consistent with the question in the answer sheet, and the question sheet matching information is used to mark the position of each element in the reference question sheet image; According to the reference question list image and the question list matching information, a to-be-processed area in the to-be-processed answer image is determined.

[0143] In an optional embodiment, the second determining module 806 is further configured to: Calculating a mapping matrix between the answer image to be processed and the reference question list image; According to the mapping matrix, the answer image to be processed and the question list matching information are superimposed to obtain a superimposed image; According to the question sheet matching information, the area to be processed in the superimposed image is determined, and the area to be processed in the superimposed image is determined as the area to be processed in the answer image to be processed.

[0144] In an optional embodiment, the second determining module 806 is further configured to: Performing feature extraction on the reference question sheet image to obtain multiple question sheet area features of the reference question sheet image, and performing feature extraction on the answer image to be processed to obtain multiple answer area features of the answer image to be processed; Calculating the regional similarity between the target question area feature and the target answer area feature, wherein the target question area feature is any one of the multiple question area features, and the target answer area feature is any one of the multiple answer area features; According to the regional similarity, determining a plurality of similar regional feature pairs from the plurality of question sheet regional features and the plurality of answer regional features; Based on the multiple similar area feature pairs, a mapping matrix between the answer image to be processed and the reference question sheet image is calculated.

[0145] In an optional embodiment, the identification module 808 is further configured to: Determine the answer correction traces corresponding to the area to be processed, and determine the answer correction scores of the questions corresponding to the area to be processed according to the answer correction traces; Determining an initial score corresponding to the answer image to be processed according to the answer correction scores of the questions corresponding to the multiple areas to be processed in the answer image to be processed; According to the initial score corresponding to the answer image to be processed, a target score corresponding to the answer image is determined.

[0146] In an optional embodiment, the identification module 808 is further configured to: Determining at least one answer correction trace contained in the answer image to be processed; Determine a bounding box corresponding to each answer correction trace in the at least one answer correction trace; According to the association relationship between the bounding boxes corresponding to the answer correction traces and the area to be processed, the answer correction trace corresponding to the area to be processed is determined from the at least one answer correction trace.

[0147] In an optional embodiment, the identification module 808 is further configured to: Determine the association relationship value between the bounding box corresponding to each answer correction trace and the area to be processed according to the area distance information between the bounding box corresponding to each answer correction trace and the area to be processed; According to the association relationship value between the bounding box corresponding to each answer correction trace and the area to be processed, the answer correction trace corresponding to the area to be processed is determined from the at least one answer correction trace.

[0148] In an optional embodiment, the identification module 808 is further configured to: Identify the score value and score type of the answer correction trace; The answer correction score corresponding to the answer correction trace is calculated according to the score value and the score type, and the answer correction score is determined as the answer correction score of the question corresponding to the area to be processed.

[0149] In an optional embodiment, the identification module 808 is further configured to: When it is determined that there is no trace of answer correction in the to-be-processed area, determining the question answering area corresponding to the to-be-processed area in the to-be-processed answering image; According to the question answering information in the question answering area, the answer correction score of the question corresponding to the to-be-processed area is determined.

[0150] In an optional embodiment, the identification module 808 is further configured to: When it is determined that the answer correction score of the question corresponding to the to-be-processed area is greater than the score threshold corresponding to the question, the answer correction score is replaced with a preset score to obtain a replaced answer correction score; or When it is determined that the answer correction score of the question corresponding to the area to be processed is greater than the score threshold corresponding to the question, a score verification request is sent to the client, and an updated score returned by the client according to the score verification request is received, and the answer correction score is replaced with the updated score to obtain the replaced answer correction score.

[0151] In an optional embodiment, the first determining module 802 is further configured to: Receiving an initial answer image sent by a client, wherein the initial answer image is obtained by capturing an image of an answer sheet after answering; The image direction of the initial answer image is adjusted using a preset image direction to obtain the answer image corresponding to the answer sheet.

[0152] To summarize, in the above-mentioned device, after determining the answer image corresponding to the answer sheet, the size information of the answer sheet can be determined. When it is determined that the size information of the answer sheet does not meet the preset size conditions, the answer image is segmented to obtain multiple sub-answer images that meet the preset size conditions, thereby avoiding inaccurate subsequent determination of the to-be-processed area in the to-be-processed answer image due to different paper sizes during the printing process of the answer sheet, ensuring the accuracy of determining the to-be-processed area in the to-be-processed answer image, further achieving the accuracy of identifying the answer correction score from the to-be-processed area, and then achieving the accuracy of calculating the target score of the answer image.

[0153] The above is a schematic scheme of an image processing device of this embodiment. It should be noted that the technical scheme of the image processing device and the technical scheme of the above-mentioned image processing method belong to the same concept, and the details not described in detail in the technical scheme of the image processing device can be referred to the description of the technical scheme of the above-mentioned image processing method.

[0154] Fig. 9 The structure block diagram of a computing device 900 provided according to an embodiment of the present specification is shown. The components of the computing device 900 include but are not limited to a memory 910 and a processor 920. The processor 920 is connected to the memory 910 via a bus 930, and a database 950 is used to store data.

[0155] The computing device 900 also includes an access device 940 that enables the computing device 900 to communicate via one or more networks 960. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 940 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0156] In one embodiment of the present application, the above components of the computing device 900 and Fig. 9 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig. 9 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.

[0157] The computing device 900 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 900 may also be a mobile or stationary server.

[0158] The processor 920 is used to execute the following computer program / instruction, which implements the steps of the above-mentioned image processing method when executed by the processor.

[0159] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the computing device embodiment, since it is basically similar to the image processing method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the image processing method embodiment.

[0160] An embodiment of the present specification further provides a computer-readable storage medium storing a computer program / instruction, which implements the steps of the above-mentioned image processing method when executed by a processor.

[0161] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the computer-readable storage medium embodiment, since it is basically similar to the image processing method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the image processing method embodiment.

[0162] An embodiment of the present specification further provides a computer program product, including a computer program / instruction, which implements the steps of the above-mentioned image processing method when executed by a processor.

[0163] The above is a schematic solution of a computer program product of this embodiment. It should be noted that the technical solution of the computer program product and the technical solution of the above-mentioned image processing method belong to the same concept, and the details not described in detail in the technical solution of the computer program product can be referred to the description of the technical solution of the above-mentioned image processing method.

[0164] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0165] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0166] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0167] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0168] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. An image processing method, characterized in that: include: Determine the answer image corresponding to the answer sheet; When it is determined that the size information of the answer sheet does not meet the preset size condition, segmenting the answer image to obtain a plurality of sub-answer images that meet the preset size condition; The plurality of sub-answer images are sequentially determined as answer images to be processed, and an area to be processed in the answer images to be processed is determined, wherein the area to be processed is used to record the answer correction score corresponding to the question in the answer sheet; The answer correction score is identified from the area to be processed, and a target score corresponding to the answer image is obtained.

2. The method according to claim 1, characterized in that: When it is determined that the size information of the answer sheet does not meet the preset size condition, segmenting the answer image to obtain a plurality of sub-answer images that meet the preset size condition includes: When it is determined that the size information of the answer sheet does not meet the preset size conditions based on the paper size information corresponding to the answer sheet and / or the paper ratio information corresponding to the answer sheet, the answer image is segmented to obtain multiple sub-answer images that meet the preset size conditions.

3. The method according to claim 1 or 2, characterized in that: The segmenting process of the answer image to obtain a plurality of sub-answer images satisfying the preset size condition includes: The answer image is segmented according to a preset segmentation rule to obtain a plurality of sub-answer images that meet the preset size conditions.

4. The method according to claim 3, characterized in that The step of segmenting the answer image according to a preset segmentation rule to obtain a plurality of sub-answer images satisfying the preset size condition comprises: According to the preset segmentation positions in the answer image, the answer image is segmented to obtain a plurality of sub-answer images satisfying the preset size condition; and / or Mark point detection is performed on the answer image to obtain a detection result, and the answer image is segmented according to the detection result to obtain a plurality of sub-answer images that meet the preset size condition.

5. The method according to claim 1, characterized in that After determining the answer image corresponding to the answer sheet, the method further includes: When it is determined that the size information of the answer sheet meets the preset size condition, the answer image is determined as the answer image to be processed.

6. The method according to any one of claims 1-2 and 5, characterized in that: The step of determining the area to be processed in the answer image to be processed includes: Determine a reference question sheet image corresponding to the answer sheet, and determine question sheet matching information corresponding to the reference question sheet image, wherein the question in the reference question sheet image is consistent with the question in the answer sheet, and the question sheet matching information is used to mark the position of each element in the reference question sheet image; According to the reference question list image and the question list matching information, a to-be-processed area in the to-be-processed answer image is determined.

7. The method according to claim 6, characterized in that The step of determining the to-be-processed area in the to-be-processed answer image according to the reference question list image and the question list matching information includes: Calculating a mapping matrix between the answer image to be processed and the reference question list image; According to the mapping matrix, the answer image to be processed and the question list matching information are superimposed to obtain a superimposed image; According to the question sheet matching information, the area to be processed in the superimposed image is determined, and the area to be processed in the superimposed image is determined as the area to be processed in the answer image to be processed.

8. The method according to claim 7, characterized in that The calculating of the mapping matrix between the answer image to be processed and the reference question list image includes: Performing feature extraction on the reference question sheet image to obtain multiple question sheet area features of the reference question sheet image, and performing feature extraction on the answer image to be processed to obtain multiple answer area features of the answer image to be processed; Calculating the regional similarity between the target question area feature and the target answer area feature, wherein the target question area feature is any one of the multiple question area features, and the target answer area feature is any one of the multiple answer area features; According to the regional similarity, determining a plurality of similar regional feature pairs from the plurality of question sheet regional features and the plurality of answer regional features; Based on the multiple similar area feature pairs, a mapping matrix between the answer image to be processed and the reference question sheet image is calculated.

9. The method according to any one of claims 1-2 and 5, characterized in that: The step of identifying the answer correction score from the to-be-processed area and obtaining a target score corresponding to the answer image includes: Determine the answer correction traces corresponding to the area to be processed, and determine the answer correction scores of the questions corresponding to the area to be processed according to the answer correction traces; Determining an initial score corresponding to the answer image to be processed according to the answer correction scores of the questions corresponding to the multiple areas to be processed in the answer image to be processed; According to the initial score corresponding to the answer image to be processed, a target score corresponding to the answer image is determined.

10. The method according to claim 9, characterized in that The step of determining the answer correction trace corresponding to the area to be processed includes: Determining at least one answer correction trace contained in the answer image to be processed; Determine a bounding box corresponding to each answer correction trace in the at least one answer correction trace; According to the association relationship between the bounding boxes corresponding to the answer correction traces and the area to be processed, the answer correction trace corresponding to the area to be processed is determined from the at least one answer correction trace.

11. The method according to claim 10, characterized in that The determining, based on the association relationship between the bounding boxes corresponding to the answer correction traces and the area to be processed, from the at least one answer correction trace, the answer correction trace corresponding to the area to be processed comprises: Determine the association relationship value between the bounding box corresponding to each answer correction trace and the area to be processed according to the area distance information between the bounding box corresponding to each answer correction trace and the area to be processed; According to the association relationship value between the bounding box corresponding to each answer correction trace and the area to be processed, the answer correction trace corresponding to the area to be processed is determined from the at least one answer correction trace.

12. The method according to claim 9, characterized in that Determining the answer correction score of the question corresponding to the to-be-processed area according to the answer correction trace includes: Identify the score value and score type of the answer correction trace; The answer correction score corresponding to the answer correction trace is calculated according to the score value and the score type, and the answer correction score is determined as the answer correction score of the question corresponding to the area to be processed.

13. The method according to claim 9, characterized in that Also includes: When it is determined that there is no trace of answer correction in the to-be-processed area, determining the question answering area corresponding to the to-be-processed area in the to-be-processed answering image; According to the question answering information in the question answering area, the answer correction score of the question corresponding to the to-be-processed area is determined.

14. The method according to claim 9, characterized in that After determining the answer correction score of the question corresponding to the to-be-processed area, the method further includes: When it is determined that the answer correction score of the question corresponding to the to-be-processed area is greater than the score threshold corresponding to the question, the answer correction score is replaced with a preset score to obtain a replaced answer correction score; or When it is determined that the answer correction score of the question corresponding to the area to be processed is greater than the score threshold corresponding to the question, a score verification request is sent to the client, and an updated score returned by the client according to the score verification request is received, and the answer correction score is replaced with the updated score to obtain the replaced answer correction score.

15. The method according to any one of claims 1 to 2 and 5, characterized in that: Determining the answer image corresponding to the answer sheet includes: Receiving an initial answer image sent by a client, wherein the initial answer image is obtained by capturing an image of an answer sheet after answering; The image direction of the initial answer image is adjusted using a preset image direction to obtain the answer image corresponding to the answer sheet.

16. A computing device, characterized in that include: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer program / instructions are executed by the processor, the steps of the method described in any one of claims 1 to 15 are implemented.

17. A computer-readable storage medium, characterized in that: It stores a computer program / instruction, which implements the steps of the method described in any one of claims 1 to 15 when executed by a processor.

18. A computer program product, characterized in that The method comprises a computer program / instruction which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 15.

Images