Image processing method and device

Abstract

The invention provides an image processing method and device, and the method comprises the steps: determining a question region in a target image and a to-be-processed region corresponding to the question region, and enabling the to-be-processed region to be used for recording answer correction traces corresponding to questions in the question region; determining an answer correction trace, and obtaining a trace type of the answer correction trace; according to trace position information of the answer correction traces and area position information of the to-be-processed area, target answer correction traces corresponding to the target to-be-processed area are determined, the target to-be-processed area is determined from the to-be-processed area, and the target answer correction traces are determined from the answer correction traces; determining a target question region corresponding to the target to-be-processed region, and determining an image processing result of the target image according to a target question in the target question region and a trace type of the target answer correction trace; the affiliation determination of the answer correction trace is conveniently and accurately realized, and the accuracy of the acquired image processing result is ensured.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an image processing method. The present invention also relates to an image processing device, a computing device, a computer-readable storage medium, and a computer program product. Background Art

[0002] In order to automatically count the completion of students' homework and summarize the correct and incorrect answers of each question into the students' personal learning information, it is necessary to collect the correction of each question in the homework. Specifically, judging the answer of each question based on the teacher's correction traces can ensure the accuracy of the collected student homework results.

[0003] In ordinary homework, the teacher's correction marks are randomly distributed, and the random marks will make the homework look very messy. Such correction marks will overlap with the questions in the homework and the students' answers. There will also be intersections and overlaps between the correction marks, thus affecting the accuracy of mark collection and attribution. Summary of the invention

[0004] In view of this, the embodiments of this specification provide an image processing method. This specification also relates to an image processing device, a computing device, a computer-readable storage medium and a computer program product to solve the above problems existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, there is provided an image processing method, including: Determine a question area in the target image, and a to-be-processed area corresponding to the question area, wherein the to-be-processed area is used to record answer correction traces corresponding to the question in the question area; Determine the answer correction trace, and obtain the trace type of the answer correction trace; Determine a target answer correction trace corresponding to a target area to be processed according to the trace position information of the answer correction trace and the area position information of the area to be processed, wherein the target area to be processed is determined from the area to be processed, and the target answer correction trace is determined from the answer correction trace; A target question area corresponding to the target area to be processed is determined, and an image processing result of the target image is determined according to the target question in the target question area and the trace type of the target answer correction trace.

[0006] According to a second aspect of an embodiment of this specification, there is provided an image processing method applied to a client, comprising: Determine a question area in the target image, and a to-be-processed area corresponding to the question area, wherein the to-be-processed area is used to record answer correction traces corresponding to the question in the question area; Determine the answer correction trace, and obtain the trace type of the answer correction trace; Determine a target answer correction trace corresponding to a target area to be processed according to the trace position information of the answer correction trace and the area position information of the area to be processed, wherein the target area to be processed is determined from the area to be processed, and the target answer correction trace is determined from the answer correction trace; A target question area corresponding to the target area to be processed is determined, and an image processing result of the target image is determined according to the target question in the target question area and the trace type of the target answer correction trace.

[0007] According to a third aspect of the embodiments of this specification, there is provided an image processing device, including: A region determination module is configured to determine a question region in a target image and a to-be-processed region corresponding to the question region, wherein the to-be-processed region is used to record answer correction traces corresponding to the question in the question region; A trace determination module is configured to determine the answer correction trace and obtain the trace type of the answer correction trace; a position determination module, configured to determine a target answer correction trace corresponding to a target to-be-processed area according to the trace position information of the answer correction trace and the area position information of the to-be-processed area, wherein the target to-be-processed area is determined from the to-be-processed area, and the target answer correction trace is determined from the answer correction trace; The result determination module is configured to determine a target question area corresponding to the target area to be processed, and determine an image processing result of the target image according to a target question in the target question area and a trace type of the target answer correction trace.

[0008] According to a fourth aspect of an embodiment of this specification, a computing device is provided, comprising a memory, a processor, and a computer program / instructions stored in the memory and executable on the processor, wherein the processor implements the steps of the image processing method when executing the computer program / instructions.

[0009] According to a fifth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores a computer program / instruction, and the steps of the image processing method are implemented when the computer program / instruction is executed by a processor.

[0010] According to a sixth aspect of the embodiments of this specification, a computer program product is provided, comprising a computer program / instruction, which implements the steps of the above-mentioned image processing method when executed by a processor.

[0011] In the image processing method provided in the present specification, a corresponding to-be-processed area exists in the question area of ​​the target image, which is used to record the answer correction traces corresponding to the questions in the question area. By recording the answer correction traces in the to-be-processed area, the target image can be made cleaner and neater, avoiding the intersection and overlap of the answer correction traces with the questions or answers in the question area, and matching the identified answer correction traces with the to-be-processed area in position, and eliminating the interference of other information as much as possible. Each question area has a corresponding to-be-processed area, which facilitates the use of the correspondence between the target to-be-processed area and the target answer correction traces to determine the correspondence between the target answer correction traces and the target question area corresponding to the target to-be-processed area, that is, conveniently and accurately realizing the determination of the attribution of the answer correction traces, and through the target question in the target question area and the trace type of the target answer correction traces, the answer situation of each question can be accurately judged, thereby ensuring the accuracy of the collected student homework results. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a scene schematic diagram of an image processing method provided by an embodiment of this specification; Figure 2 is a flow chart of an image processing method provided by an embodiment of this specification; Figure 3 is a processing flow chart of an image processing method provided in an embodiment of this specification; Figure 4a It is a schematic diagram of a bill of lading diagram provided in an embodiment of this specification; Figure 4b is a schematic diagram of a matching diagram provided in an embodiment of this specification; Figure 4c It is a schematic diagram of the effect after a bill of lading diagram and a matching diagram are superimposed, provided in an embodiment of this specification; Figure 5 It is a schematic diagram of a correction mark provided in an embodiment of this specification; Figure 6 is a structural schematic diagram of an image processing device provided by an embodiment of this specification; Figure 7 It is a structural block diagram of a computing device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0013] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0014] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0015] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0016] First, the terms involved in one or more embodiments of this specification are explained.

[0017] Target detection model: Target detection model is an algorithm used in the field of computer vision to identify and locate the location and category of one or more targets (objects) in an image. Target detection model can accomplish two tasks at the same time: 1) identify the targets present in the image and determine what they are (for example, people, cars, cats, etc.); 2) locate the exact location of these targets in the image, usually by drawing bounding boxes (bbox).

[0018] YOLO: YOLO (You Only Look Once) is a popular object detection algorithm. Its core idea is to solve the object detection task as a regression problem. Unlike traditional object detection methods, the YOLO algorithm predicts multiple bounding boxes and the probabilities of their categories in a single network, thereby achieving fast and efficient object detection. The main advantage of the YOLO algorithm is its speed, which is particularly suitable for real-time object detection applications.

[0019] IoU: Intersection over Union is an indicator that measures the degree of overlap between two bounding boxes, and is widely used in the field of object detection. It is the ratio of the intersection area of ​​two bounding boxes to the union area, that is, IoU = intersection area / union area.

[0020] Variants of IoU: 1. GIoU (Generalized IoU): GIoU not only measures the overlapping area, but also measures the distance between two boxes by considering the area of ​​the minimum enclosing rectangular box. The value range of GIoU is [-1,1]. When the two images completely overlap, IoU=GIoU=1. When the two images do not intersect, IoU=0, GIoU=-1.

[0021] 2. DIoU (Distance IoU): DIoU takes into account the distance between the center points of two boxes and is suitable for processing non-overlapping and distant bounding boxes.

[0022] 3.CIoU (Complete IoU): CIoU is one of the most complex IoU variants. It takes into account the distance between the center points of the two boxes, the similarity of the shapes / sizes of the two boxes, and balances the two through the weight term α.

[0023] Homework pictures, question list pictures and matching pictures: In the client-side question combination service (selecting questions from the online question bank to generate homework question lists), when generating the question list pdf format file, a matching picture with the same layout as the pdf format file will be generated. The key areas in the matching picture are marked with different boxes for collecting homework information and answer information. The picture of the question list pdf format file mentioned above is the question list picture, which is a lossless question list picture; the corresponding matching picture with the same layout is the matching picture; after the student prints the question list picture and answers the question, the picture uploaded by high-speed scanner or photo is the homework picture.

[0024] Homework collection: The paper version of the homework is entered into an electronic image through a high-speed scanner or by taking a photo and uploading it; and a series of algorithms are used to detect, identify, and correct the homework information and answer information in the homework image.

[0025] Homework correction traces: In paper homework, the teacher’s correction information such as √, ×, ○ is left.

[0026] Free marking: teachers leave marks at random locations in the homework.

[0027] Leaving marks in the box: The teacher leaves marks in or near the designated correction box. In the embodiment of this specification, the area to be processed is the area corresponding to the correction box.

[0028] Marking box: A box used to designate where the teacher is to leave marks.

[0029] In the existing scheme, the free traces in the homework image are detected to obtain the bbox and classification recognition results of the free traces, and the detection results of the free traces are attributed to each question according to the scope of the question, so as to obtain the correction results of each question. However, in this method, the free traces will overlap with the question stem and the student's answer in the homework, and there will be intersections and overlaps between the free traces, which will affect the detection and recognition of the model. Some free traces will span multiple question ranges, and some question ranges will have multiple traces. Therefore, complex logical processing is required to determine the homework correction trace corresponding to a question, and random traces make the homework look messy.

[0030] In this specification, an image processing method is provided. This specification also relates to an image processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail one by one in the following embodiments.

[0031] See also Figure 1 , Figure 1 A schematic diagram of a scene of an image processing method provided according to an embodiment of the present specification is shown.

[0032] Figure 1 The device includes a terminal device 102 and a server 104, wherein the server 104 can be used to execute the image processing method provided in the embodiment of this specification.

[0033] In the specific implementation, in the field of education, it is necessary to analyze the learning situation of the homework completed by the students. Based on this, the students can take a photo of the completed homework and upload the captured homework image to the terminal device 102. The terminal device 102 can send the homework image as a target image to the server 104, and the server 104 can process the target image.

[0034] Specifically, the server 104 can determine the question area in the target image, and the area to be processed corresponding to the question area, wherein the area to be processed is used to record the answer correction traces corresponding to the questions in the question area; determine the answer correction traces, and obtain the trace type of the answer correction traces; determine the target answer correction traces corresponding to the target area to be processed according to the trace position information of the answer correction traces and the area position information of the area to be processed, wherein the target area to be processed is determined from the area to be processed, and the target answer correction traces are determined from the answer correction traces; determine the target question area corresponding to the target area to be processed, and determine the image processing result of the target image according to the target question in the target question area and the trace type of the target answer correction traces. According to the image processing result, learning situation collection and analysis are realized.

[0035] The end-side device 102 may include a browser, an APP (Application), or a web application such as an H5 (Hyper Text Markup Language 5, version 5 of Hypertext Markup Language) application, or a light application (also known as a mini-program, a lightweight application) or a cloud application, etc. The end-side device may be based on a software development kit (SDK) of the corresponding service provided by the server, such as based on the real-time communication (RTC, Real Time Communication) SDK development and acquisition. The end-side device may be deployed in an electronic device and needs to rely on the device to run or some APPs in the device to run. The electronic device may have a display screen and support information browsing, such as a personal mobile terminal such as a mobile phone, a tablet computer, a personal computer, etc. Various other types of applications may also be configured in the electronic device, such as human-computer dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0036] Server 104 can be understood as a server that provides various services, including physical servers and cloud servers, such as a server that provides communication services to multiple clients, a server for background training that supports the model used on the client, and a server that processes the data sent by the client. It should be noted that server 104 can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. Server 104 can also be a server for a distributed system, or a server combined with a blockchain. Server 104 can also be a cloud server for basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDNs), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0037] It is worth noting that the image processing method provided in the embodiments of this specification can be executed by the server 104. In other embodiments of this specification, the terminal device 102 can have functions similar to those of the server 104, thereby executing the image processing method provided in the embodiments of this specification; in other embodiments, the image processing method provided in the embodiments of this specification can also be executed jointly by the terminal device 102 and the server 104.

[0038] In the image processing method provided by the embodiments of the present specification, a corresponding to-be-processed area exists in the question area of ​​the target image, which is used to record the answer correction traces corresponding to the questions in the question area. By recording the answer correction traces in the to-be-processed area, the target image can be made cleaner and neater, avoiding the intersection and overlap of the answer correction traces with the questions or answers in the question area, and matching the identified answer correction traces with the to-be-processed area in position, and eliminating interference from other information as much as possible. Each question area has a corresponding to-be-processed area, which facilitates the use of the correspondence between the target to-be-processed area and the target answer correction traces to determine the correspondence between the target answer correction traces and the target question area corresponding to the target to-be-processed area, that is, conveniently and accurately realizing the determination of the attribution of the answer correction traces, and through the target question in the target question area and the trace type of the target answer correction traces, the answer situation of each question can be accurately judged, thereby ensuring the accuracy of the collected student homework results.

[0039] Figure 2 A flowchart of an image processing method provided according to an embodiment of the present specification is shown, which specifically includes the following steps: Step 202: Determine a question area in the target image and a to-be-processed area corresponding to the question area, wherein the to-be-processed area is used to record answer correction traces corresponding to the questions in the question area.

[0040] Specifically, the image processing method provided in the embodiments of this specification can be applied to the processing of homework images in the field of education. For example, for an image of a test paper after a student has answered the questions, the answer status of each question in the test paper image can be determined according to the image processing method.

[0041] Among them, the target image is the image to be processed. In the embodiment of this specification, the target image can be a test paper image or a homework image after the student has answered the questions, etc. The target image includes the student's answers and the answer correction traces for the student's answers (the answer correction traces are the homework correction traces).

[0042] In practical applications, the image to be processed may be taken by a student through a photographing device, or may be scanned by an image scanning device, or may be obtained through other devices, such as a high-definition camera, an ink screen, etc. Alternatively, the image to be processed may also be a student's electronic homework, such as an electronic homework in PDF format, etc. This specification embodiment does not limit this.

[0043] The question area in the target image includes the question text and the answer area below the question; the area to be processed corresponding to the question area can be understood as the correction area corresponding to the correction box. The area to be processed can be inside the question area or outside the question area, that is, the size and position of the area to be processed can be freely defined and are not restricted here. However, when the area to be processed is used to record the answer correction traces corresponding to the questions in the question area and assist in the collection of answer correction traces, the area to be processed can be set in front of the question number corresponding to the question, which is convenient for matching the question area with the area to be processed and can also ensure aesthetics.

[0044] In specific implementation, since the area to be processed is an area separated from the question area, recording the answer correction traces in the area to be processed can avoid the intersection and overlap of the answer correction traces with the question information in the question area and the students' answer information, thereby improving the processing efficiency and accuracy when subsequently determining the question corresponding to the answer correction traces.

[0045] In one or more implementations of this specification, before determining the target image, a question sheet image is first generated, and the target image is obtained by answering and correcting on the question sheet image. When generating the question sheet image, a correction box can be set on the question sheet image, so that subsequent teachers can leave marks in the correction box when correcting. The specific implementation method is as follows: Before determining the topic area in the target image and the area to be processed corresponding to the topic area, the method further includes: Generate an initial image according to the questions selected from the question library and the selection order, wherein the initial image is an image including an initial question area, and the initial question area corresponds to an initial area to be processed; The first area position information of the initial topic area in the initial image and the second area position information of the initial area to be processed are obtained and stored in an information storage unit.

[0046] The initial image is a question sheet image, which can be understood as a test paper or homework consisting of questions. The initial question area is the area in the initial image that contains the question text and the answer part below the question, and the initial area to be processed is the correction area corresponding to the correction box in the initial image. The information storage unit can be understood as a database.

[0047] The first area position information is used to describe the area position of the initial question area, and the second area position information is used to describe the area position of the initial area to be processed. The first area position information and the second area position information can be expressed in the form of coordinates.

[0048] In actual applications, users can select questions from an online question bank (i.e., a question bank). The selected questions and the selection order can be automatically combined to generate a question list diagram, wherein the selection order can be used as the question number corresponding to each question. When generating a question list diagram, a correction box is generated in front of the question number of each question according to the custom setting selection.

[0049] When generating a question list diagram, the first area position information of the initial question area and the second area position information of the initial area to be processed can be obtained. Of course, according to actual needs, a score box can also be generated in front of the question number of each question in the question list diagram to facilitate the statistics of the score. At this time, the third area position information of the score box corresponding to the score area can also be obtained, which is not limited here. That is, the area position information is used to mark the position and area of ​​each element in the question list diagram. These area position information facilitate the subsequent regional matching of the uploaded homework diagram. Therefore, these area position information can be called matching information. The matching information can be displayed in the form of a matching diagram. The matching diagram is an image with the same layout as the question list diagram, and different color boxes can be used to mark different areas, such as using a red rectangular box to mark the initial area to be processed, and using a yellow rectangular box to mark the initial question area. The question list diagram and the corresponding matching information are associated and stored in the database.

[0050] In one or more embodiments of the present invention, matching information is obtained through an initial image corresponding to a target image, and the title area in the target image and the area to be processed corresponding to the title area are determined through the matching information. For example, when the matching information is represented in the form of a matching graph, the layout of the matching graph is consistent with that of the target image. By overlapping the matching graph with the work graph, the title area in the target image and the area to be processed can be determined based on the color frame in the matching graph. The specific implementation is as follows: The step of determining a topic area in a target image and a to-be-processed area corresponding to the topic area includes: Determine a target image, an initial image corresponding to the target image, first area position information of the initial question area in the initial image, and second area position information of the initial area to be processed, wherein the target image is an image obtained by an image acquisition device, with traces of answer correction added to the initial area to be processed of the initial image; A topic area in the target image is determined according to the first area position information, and a to-be-processed area in the target image corresponding to the topic area is determined according to the second area position information.

[0051] In actual applications, when the initial image is generated, the initial image can be printed, and students fill in answers on the printed initial image. Teachers make corrections in the correction box of the printed initial image and upload the target image. The corresponding initial image can be determined through the target image, that is, the corresponding question sheet image can be determined for the homework image. Since the layout of the initial image is consistent with that of the target image, the initial question area in the initial image can correspond to the question area in the target image, and the initial area to be processed in the initial image can correspond to the area to be processed in the target image.

[0052] Therefore, the title area in the target image can be determined based on the first area position information of the initial title area in the initial image, and the area to be processed corresponding to the title area in the target image can be determined based on the second area position information of the initial area to be processed.

[0053] The data processing method provided in the embodiments of this specification can efficiently and quickly determine the question area and the area to be processed in the target image corresponding to the initial image through the matching information corresponding to the initial image.

[0054] In one or more embodiments of the present specification, when submitting and uploading the work drawing, the corresponding question sheet image can be specified, so as to quickly determine the initial image corresponding to the target image; or the associated information corresponding to the question sheet image can be printed in the question sheet image, so that when the work drawing is subsequently uploaded, the associated information related to the initial image can be obtained by scanning and identifying the information printed on the paper, so as to accurately determine the initial image corresponding to the target image. The specific implementation is as follows: The determining of the target image, the initial image corresponding to the target image, the first area position information of the initial topic area in the initial image, and the second area position information of the initial area to be processed includes: In response to an image processing request sent by a client, determine a target image and associated information, wherein the image processing request carries the associated information, or the associated information is obtained by identifying the target image; An initial image is determined according to the association information, and first area position information of the initial topic area and second area position information of the initial area to be processed in the initial image are determined from the information storage unit according to the initial image.

[0055] Among them, the associated information can be understood as information related to the question image, which is used to determine the matching information corresponding to the target image. Because the question image and the matching information are stored in association, the matching information corresponding to the question image can be obtained based on the relevant information of the question image.

[0056] For example, the assignment name of the question sheet image can be determined as the associated information. By specifying the corresponding assignment name when uploading the assignment image, the initial image corresponding to the target image can be obtained according to the assignment name, and the matching information can be obtained according to the initial image.

[0057] In specific implementation, when uploading the homework image through the client, the question sheet image corresponding to the homework image can be specified by the job name and job identifier, so that the target image and related information are carried in the image processing request sent by the client. Alternatively, the related information or the encrypted and transcoded related information can be printed in the question sheet image, and when the user prints the question sheet image to answer the questions and obtains the homework image, the related information printed on the paper will not be changed. Therefore, by scanning and identifying the uploaded homework image, the related information can be obtained from the target image.

[0058] In actual applications, the associated information (assuming it is a string) can be printed directly, or it can be included in the work diagram in non-explicit ways such as QR codes, bar codes, feature images, etc. For example, each work diagram can carry a QR code in the upper right corner, which indicates the identification information of the work diagram corresponding to the question sheet diagram. A QR code recognition tool can be used to identify the QR code of the work image and obtain the identification information, thereby obtaining the corresponding initial image and the matching information corresponding to the initial image from the database.

[0059] The image processing method provided in the embodiments of the present specification, when the target image is determined, determines the corresponding initial image through the associated information, and can accurately and efficiently obtain the matching information corresponding to the initial image from the database (i.e., the first area position information of the initial question area in the initial image, and the second area position information of the initial area to be processed), so as to facilitate subsequent area matching.

[0060] In one or more embodiments of this specification, because schools are accustomed to printing the question sheet diagram and then photocopying it in order to save costs, the question sheet content may be displaced, tilted, etc., resulting in the position of the questions in the homework diagram and the question sheet diagram not being one-to-one corresponding, and the matching information corresponding to the question sheet diagram cannot be used to correctly match the area in the homework diagram, so it is necessary to adjust the question sheet diagram so that the adjusted question sheet diagram can correspond one-to-one with the homework diagram in terms of position. The specific implementation method is as follows: The determining of the topic area in the target image according to the first area position information, and the determining of the area to be processed corresponding to the topic area in the target image according to the second area position information, comprises: According to the target image and the initial image, the first region position information and the second region position information are adjusted to obtain first target region position information and second target region position information; determining a topic area in the target image according to the first target area position information; According to the second target area position information, a to-be-processed area corresponding to the question area in the target image is determined.

[0061] By adjusting the first area position information of the initial image and the second area position information according to the target image and the initial image, the adjusted first target area position information and the second target area position information are obtained, so as to determine the title area in the target image and the area to be processed corresponding to the title area using the first target area position information and the second target area position information.

[0062] Specifically, by means of feature extraction, an affine transformation matrix is ​​calculated, and the question sheet image is transformed using the affine transformation matrix so that the transformed question sheet image can perfectly overlap with the homework image. The specific implementation method is as follows: The adjusting the first region position information and the second region position information according to the target image and the initial image to obtain the first target region position information and the second target region position information includes: Performing feature extraction on the target image to obtain multiple topic area features of the target image, and performing feature extraction on the initial image to obtain multiple initial topic area features of the initial image; Calculating the regional similarity between the plurality of topic regional features and the plurality of initial topic regional features; According to the region similarity, a plurality of similar region feature pairs are determined from the plurality of topic region features and the plurality of initial topic region features, wherein a similar region feature pair includes a topic region feature and an initial topic region feature; Determining a mapping matrix between the target image and the initial image according to the plurality of similar region feature pairs; The initial image is adjusted using the mapping matrix, the first region position information is adjusted to the first target region position information, and the second region position information is adjusted to the second target region position information.

[0063] Specifically, because the target image and the initial image have the same question text and the same layout, the similarity between the target image and the area in the initial image can be calculated by extracting features, and the question area in the target image is corresponded to the initial question area in the initial image, and the to-be-processed area in the target image is corresponded to the initial to-be-processed area in the initial image; however, when the matching image is an image containing different color frames but no question text, the matching of the target image and the matching image cannot be achieved by extracting features from the target image and the matching image. It is necessary to use the question sheet image as an intermediate reference to locate the matching image, that is, to transform the question sheet image through the calculated mapping matrix. Since the matching image and the question sheet image are consistent in size, proportion, and layout, the mapping matrix can be used to directly transform the matching image so that the transformed matching image can perfectly overlap with the homework image.

[0064] In the specific implementation, feature extraction is performed on the target image and the initial image respectively. Feature descriptors can be used to measure the similarity between the features in the two images for the extracted features. Specifically, the distance between feature descriptors can be calculated using methods such as Euclidean distance and cosine similarity, which can be used as a measure of regional similarity. In addition, the most similar feature pairs can be found using a fast nearest neighbor search algorithm. Based on the calculated regional similarity, feature pairs that meet certain threshold conditions are screened out, which are similar regional feature pairs. These similar regional feature pairs should be features of corresponding positions in the two images. In order to improve matching accuracy, the RANSAC (random sampling consensus) algorithm is usually applied to remove false matches.

[0065] Using the similar region feature pairs determined above, a geometric transformation model, such as affine transformation and homography matrix, can be solved by the least squares method. This model can transform the initial image into the spatial coordinate system of the target image. This transformation model is the mapping matrix.

[0066] The obtained mapping matrix can be applied to the entire initial image to achieve image adjustment. Specifically, the mapping matrix is ​​applied to each pixel in the initial image to obtain its new position in the target image coordinate system. In this way, the first area position information is adjusted to the first target area position information, and the second area position information is adjusted to the second target area position information.

[0067] The image processing method provided in the embodiments of the present specification transforms the initial image by obtaining a mapping matrix between the target image and the initial image, so that the region position in the initial image is also transformed accordingly. When the first region position information is adjusted to the first target region position information and the second region position information is adjusted to the second target region position information, the question region and the region to be processed in the target image can be correctly matched based on the adjusted first target region position information and the second target region position information.

[0068] Step 204: Determine the answer correction trace, and obtain the trace type of the answer correction trace.

[0069] Specifically, a target detection model is obtained based on YOLO training. In the embodiment of this specification, the target detection model is used to obtain the bbox and trace type of the answer correction traces in the homework graph.

[0070] In fact, according to the collection needs and the teacher's grading habits, the answer correction traces are abstracted into the following trace types: correct, wrong, half correct, etc., that is, the common correction traces are classified into the above categories.

[0071] Step 206: Determine the target answer correction trace corresponding to the target area to be processed based on the trace position information of the answer correction trace and the area position information of the area to be processed, wherein the target area to be processed is determined from the area to be processed, and the target answer correction trace is determined from the answer correction trace.

[0072] Specifically, when the answer correction traces and the area to be processed are detected, the trace position information of the answer correction traces and the area position information of the area to be processed are obtained, wherein the trace position information and the area position information of the area to be processed are both coordinate information.

[0073] In fact, the trace position information of the answer correction trace is obtained, that is, the bbox of the answer correction trace is obtained (used to define the rectangular boundary of the answer correction trace); the area position information of the area to be processed is obtained, that is, the bbox of the correction box is obtained (used to define the rectangular boundary of the correction box).

[0074] The answer correction traces are attributed according to the relationship value between the bbox of the answer correction trace and the correction box bbox, that is, the target answer correction trace corresponding to the target area to be processed is determined, among which it is used to measure how close the relationship between the two bboxes is.

[0075] In one or more embodiments of the present specification, first, any one of the multiple to-be-processed areas is determined as a target to-be-processed area, and for the target to-be-processed area, a position distance is determined based on the area position information of the target to-be-processed area and the trace position information of multiple answer correction traces, thereby determining the target answer correction trace corresponding to the target to-be-processed area from the multiple answer correction traces based on the position distance. The specific implementation is as follows: The answer correction traces are multiple answer correction traces, and the to-be-processed area is multiple to-be-processed areas; The step of determining the target answer correction trace corresponding to the target area to be processed according to the trace position information of the answer correction trace and the area position information of the area to be processed includes: Determine a target area to be processed, wherein the target area to be processed is any one of the areas to be processed; Calculate the position distance between the target area to be processed and each answer correction trace in the multiple answer correction traces according to the area position information of the target area to be processed and the trace position information of the multiple answer correction traces; According to the position distance, a target answer correction trace corresponding to the target area to be processed is determined from the multiple answer correction traces.

[0076] The target pending area can be understood as the pending area currently being processed and whose corresponding answer correction traces are to be confirmed.

[0077] Specifically, the position distance between the target area to be processed and each answer correction trace is calculated through the area position information of the target area to be processed and the trace position information of multiple answer correction traces. The position distance can be understood as the relationship value in the above embodiment.

[0078] In practical applications, the relationship value can be calculated by the following methods: determine the correction box bbox corresponding to the target area to be processed and the bbox of each answer correction trace, and determine the relationship value by calculating the distance between the correction box bbox and each answer correction trace bbox, where the distance between bboxes can be the corner distance, the closest distance, the distance between centers, etc., which is not limited here. Calculate the relationship value by IoU or various IoU variants.

[0079] In one or more embodiments of the present specification, since the answer correction trace with the trace type of "correct", the teacher usually uses the check mark (√) symbol to represent it, and sometimes the tail of the check mark symbol may be too long, which may cause inaccurate attribution of the answer correction trace. For example, the tail of the check mark symbol covers the previous correction box of the correct correction box. At this time, the check mark symbol may be attributed to the previous correction box, resulting in inaccurate attribution. Therefore, corresponding target processing strategies can be determined for different trace types, and the trace position information of the answer correction trace can be processed according to the target processing strategy to obtain the target trace position information with a weight value, so as to calculate a more accurate position distance according to the target trace position information and the area position information of the target area to be processed. The specific implementation method is as follows: Calculating the position distance between the target area to be processed and each answer correction trace in the plurality of answer correction traces according to the area position information of the target area to be processed and the trace position information of the plurality of answer correction traces includes: Determining, according to the trace type of each answer correction trace among the plurality of answer correction traces, a target processing strategy corresponding to each answer correction trace; Processing the trace position information of each answer correction trace according to the target processing strategy to obtain a plurality of target trace position information of each answer correction trace, wherein each target trace position information of the plurality of target trace position information includes a weight value; According to the area position information of the target area to be processed and the multiple target trace position information of the answer correction traces, the position distance between the target area to be processed and each answer correction trace in the multiple answer correction traces is calculated.

[0080] Specifically, different trace attribution strategy algorithms (i.e., target processing strategies) are designed according to the trace types of answer correction traces. The trace attribution strategy algorithm can be understood as an algorithm that divides the answer correction traces into multiple parts and sets corresponding weights for each part, thereby calculating the position distance according to the position information after setting the weights. The position information of the multiple parts after division and containing weights is the position information of multiple target traces.

[0081] For example, the answer correction trace can be divided into four parts, and the corresponding weights can be determined for the four divided parts. The strategies for calculating weights are different for answer correction traces of different trace types. For example, for check marks and half-check marks, due to the teacher's marking habits, the weight of the lower left corner part can be set to the maximum weight, and the weights of other parts can be less than the weight of the lower left corner part. When the trace type of the answer correction trace is an error, the answer correction trace can usually be a cross, then the central area of ​​the answer correction trace can be set to the maximum weight.

[0082] When calculating the relationship value between the answer correction trace and the target area to be processed, the initial relationship values ​​are calculated between the multiple divided parts and the target area to be processed respectively, that is, according to the regional position information of the multiple parts after the answer correction trace is divided, and the regional position information of the target area to be processed is respectively calculated, to obtain multiple initial position distances (that is, initial relationship values), and the multiple initial position distances are weightedly averaged according to the weight values ​​in the position information of multiple target traces to obtain the relationship value between the answer correction trace and the target area to be processed.

[0083] The image processing method provided in the embodiments of this specification determines different target processing strategies according to different trace types. By processing the answer correction traces through the target processing strategies, multiple target trace position information with weight values ​​can be obtained. Therefore, when using the multiple target trace position information with weight values ​​and calculating the position distance with the target area to be processed, a more accurate position distance can be obtained, thereby ensuring the accuracy of the attribution of the answer correction traces.

[0084] In one or more embodiments of the present specification, in order to avoid misjudgment when attributing answer correction traces, a validity threshold is set, that is, only answer correction traces with a relationship value above the validity threshold are considered valid. The range of the answer correction trace is represented by a rectangular box bbox. When the handwriting of part of the answer correction trace is relatively long, its bbox will be very large, resulting in misjudgment when attributing the trace by means of bbox overlapping IoU. Therefore, it is necessary to set a validity threshold for the trace. And a correction box is only allowed to have one answer correction trace attributed. If there are multiple valid correction traces around the correction box, one of them needs to be selected as the final result. The specific implementation method is described as follows: Determining, according to the position distance, a target answer correction trace corresponding to the target area to be processed from the multiple answer correction traces, comprising: According to the position distance, determining a candidate answer correction trace that meets a distance threshold from the multiple answer correction traces; Sorting the candidate answer correction traces to obtain a sorting result; According to the sorting result, the target answer correction trace corresponding to the target area to be processed is determined from the candidate answer correction traces.

[0085] Among them, the candidate answer correction traces can be understood as the determined effective correction traces; the distance threshold is the effectiveness threshold, and the position distance between the answer correction traces and the target area to be processed and the answer correction traces above the distance threshold are determined as valid correction traces. The distance threshold can be set according to actual conditions and is not limited here.

[0086] Specifically, valid candidate answer correction traces are determined from multiple answer correction traces through location distance and a set distance threshold. According to actual conditions, one target area to be processed should correspond to one answer correction trace. If there are multiple candidate answer correction traces for a target area to be processed, the candidate answer correction traces can be sorted according to location distance to obtain a sorting result, and the candidate answer correction trace that is closest to the target area to be processed in the sorting result is determined as the target answer correction trace.

[0087] The image processing method provided in the embodiments of this specification can accurately and reasonably determine the target answer correction traces corresponding to the target area to be processed by first determining the valid candidate answer correction traces and then sorting the candidate answer correction traces according to the position distance.

[0088] In one or more embodiments of the present specification, for a target area to be processed without traces of answer correction, a basic logic is set, that is, to detect whether there is handwritten content in the question area. If not, the target answer correction trace corresponding to the target area to be processed is determined to be the first trace type. If so, the target answer correction trace corresponding to the target area to be processed is determined to be the second trace type. The specific implementation is as follows: After calculating the position distance between the target area to be processed and each answer correction trace in the multiple answer correction traces according to the area position information of the target area to be processed and the trace position information of the multiple answer correction traces, the method further includes: In the case that the target answer correction trace corresponding to the target area to be processed cannot be determined from the multiple answer correction traces according to the position distance, target object recognition is performed on the target question area corresponding to the target area to be processed to obtain a recognition result; In the case where the recognition result is that the target object is not detected, determining that the target answer correction trace corresponding to the target to-be-processed area is a first trace type; In the case where the recognition result is that the target object is detected, it is determined that the target answer correction trace corresponding to the target area to be processed is a second trace type.

[0089] The target object can be understood as the handwritten content, that is, the student's handwritten answer content. The first trace type is wrong, and the second trace type is right.

[0090] Specifically, when there are no corresponding valid candidate answer correction traces in the target area to be processed and the target answer correction traces cannot be determined, the target question area corresponding to the target area to be processed is detected to see whether there are student answers. If so, that is, the student has answered the question, then the student's answer to the question is determined to be correct, that is, the target answer correction traces corresponding to the target area to be processed are determined to be correct; if not, that is, the student has not answered the question, then the student's answer to the question is determined to be incorrect, that is, the target answer correction traces corresponding to the target area to be processed are determined to be incorrect.

[0091] The image processing method provided in the embodiments of this specification, when it is impossible to determine the target answer correction traces corresponding to the target area to be processed from multiple answer correction traces, performs a detection of the student's answer in the target question area, and determines the trace type of the target answer correction traces corresponding to the target area to be processed based on the detection results, thereby ensuring that each correction box has a corresponding correction trace affiliation and ensuring the correctness of the basic logic.

[0092] Step 208: Determine a target question area corresponding to the target area to be processed, and determine an image processing result of the target image according to the target question in the target question area and the trace type of the target answer correction trace.

[0093] Specifically, when a to-be-processed area corresponds to a question area, the target question area corresponding to the target to-be-processed area is determined. The target question exists in the target question area. Therefore, according to the target question and the target answer correction trace type, the student's answer to the target question can be determined. By collecting each question in the target image, the image processing result of the target image is obtained, that is, the student's answer to each question is determined.

[0094] In fact, since the topic is selected from the topic library when the bill of lading diagram is combined and generated, the bill of lading diagram and the associated topic information (topic number, corresponding topic area location information and specific topic content) can be stored in the database. When the target topic area is determined, the corresponding specific topic content can be determined from the database according to the area location information of the target topic area, without the need for topic text detection and recognition. Of course, in actual applications, text detection and recognition can also be performed on the target topic in the target topic area to obtain the corresponding target topic, which is not limited here.

[0095] In one or more embodiments of the present specification, the target acquisition result corresponding to the target question can be determined by the target question and the trace type of the target answer correction trace, and the image processing result of the entire target image can be obtained by the target acquisition result of the target question. The specific implementation is as follows: Determining the image processing result of the target image according to the target question in the target question area and the trace type of the target answer correction trace includes: Determine the target collection result corresponding to the target question according to the target question area in the target to-be-processed area and the trace type of the target answer correction trace; The image processing result of the target image is determined according to the target acquisition result corresponding to the target topic.

[0096] The target acquisition result can be understood as the student's answer to the target question. The image processing result can be understood as the answer to each question on the collected target image.

[0097] In fact, the target image includes multiple areas to be processed, multiple question areas, and multiple questions. The target acquisition result corresponding to each question is determined by determining the corresponding target answer correction traces for each area to be processed and the question corresponding to each area to be processed and the trace type of the target answer correction traces. The image processing result is determined by collecting the target acquisition results corresponding to each question.

[0098] Subsequently, based on the target collection results corresponding to each collected question, we can further determine the types of questions with higher student error rates, etc., which are not limited here.

[0099] The image processing method provided in the embodiments of this specification determines the target acquisition result corresponding to each question in the target image by determining the target acquisition result corresponding to the target question, thereby determining the image processing result of the target image, which is convenient for statistics on students' answers to homework pictures.

[0100] In the image processing method provided by the embodiments of the present specification, a corresponding to-be-processed area exists in the question area of ​​the target image, which is used to record the answer correction traces corresponding to the questions in the question area. By recording the answer correction traces in the to-be-processed area, the target image can be made cleaner and neater, avoiding the intersection and overlap of the answer correction traces with the questions or answers in the question area, and matching the identified answer correction traces with the to-be-processed area in position, and eliminating interference from other information as much as possible. Each question area has a corresponding to-be-processed area, which facilitates the use of the correspondence between the target to-be-processed area and the target answer correction traces to determine the correspondence between the target answer correction traces and the target question area corresponding to the target to-be-processed area, that is, conveniently and accurately realizing the determination of the attribution of the answer correction traces, and through the target question in the target question area and the trace type of the target answer correction traces, the answer situation of each question can be accurately judged, thereby ensuring the accuracy of the collected student homework results.

[0101] Another embodiment of the present specification also provides an image processing method applied to a client, including determining a question area in a target image, and an area to be processed corresponding to the question area, wherein the area to be processed is used to record answer correction traces corresponding to the questions in the question area; determining the answer correction traces, and obtaining the trace types of the answer correction traces; determining target answer correction traces corresponding to the target area to be processed according to the trace position information of the answer correction traces and the area position information of the area to be processed, wherein the target area to be processed is determined from the area to be processed, and the target answer correction traces are determined from the answer correction traces; determining a target question area corresponding to the target area to be processed, and determining the image processing result of the target image according to the target question in the target question area and the trace types of the target answer correction traces.

[0102] Specifically, the determining of the topic area in the target image and the area to be processed corresponding to the topic area includes: In response to an image upload instruction from a user through the image upload interface of the client, a target image uploaded by the user is determined; and for the target image, a topic area in the target image and a to-be-processed area corresponding to the topic area are determined.

[0103] The specific implementation method can be found in the above embodiments, which will not be described in detail here.

[0104] See also Figure 3 , Figure 3 The following is a flowchart of an image processing method provided by an embodiment of the present specification, which specifically includes the following steps: The image processing method is applied to an image processing system, which includes a client and a server.

[0105] Step 302: User group question list.

[0106] Step 304: Generate a question list graph and a matching graph.

[0107] For details, please refer to Figure 4a , Figure 4b , Figure 4a A schematic diagram showing a bill of lading diagram, Figure 4b A schematic diagram of a matching graph is shown.

[0108] Specifically, based on the group question layout, the question list is rendered, such as Figure 4a As shown, a question list diagram is generated based on the questions and order selected by the user when composing the questions, and a correction box is generated before the question number of each question.

[0109] Render the matching graph based on the question sheet layout, such as Figure 4bAs shown in the figure, when the question sheet is generated, a corresponding solid color box is generated at the position of the question area, and the correction box is the red rectangular box before the question number. It should be noted that the matching information is used to mark the position and area of ​​each element in the question sheet, and is not limited to the form. The matching map is a form of matching information, and the same effect can be achieved by recording the position of each box with plain text information.

[0110] The question list and matching information are generated one-to-one. For the superimposed display effect, please refer to Figure 4c ,Right now Figure 4c A schematic diagram showing the effect of superimposing a bill of lading image and a matching image is shown.

[0111] In practical applications, the question sheet graph and the corresponding matching information are stored in association in the database and can be obtained using the relevant information of the homework graph.

[0112] Step 306: The user prints the question sheet for use and uploads the homework diagram.

[0113] In fact, users print out the question sheets, students fill in the answers, teachers mark them and upload the homework pictures.

[0114] Step 308: Obtain the question list graph and matching graph.

[0115] Get the question sheet and matching information during the grading process (i.e. Figure 3 There are several ways to do this: Specify when submitting an assignment: The association information between the assignment graph, question sheet graph, and matching graph can be directly specified by the user when submitting the assignment (through the assignment name, assignment ID, etc.).

[0116] Obtaining through the work map: Print the associated information or the encrypted and transcoded associated information in the question list map. This information (assuming it is a string) can be printed directly, or included in the work map in a non-explicit way such as a QR code, barcode, feature image, etc., so that the corresponding associated information can be obtained by scanning the work map.

[0117] Step 310: Calculate the mapping matrix between the homework graph and the question list graph.

[0118] Specifically, in order to save costs, schools are used to printing the question sheet and then photocopying it. At this time, the question sheet content may be displaced or tilted, resulting in the position of the questions in the homework image and the question sheet image not being one-to-one corresponding. Therefore, the matching image and the homework image cannot be directly overlapped to determine the question area and correction box in the homework image; in order to solve this problem, the question sheet image is needed as an intermediate reference to locate the matching image. The process is as follows: (1) Calculating the features of the work graph: In the embodiments of this specification, the HOG feature (Histogram of Oriented Gradient, a feature description used for object detection in computer vision and image processing) is used to extract the question area in the work graph.

[0119] (2) Calculate the features of the problem sheet image: Use the same feature extractor to compare the extracted area with the area in (1), select several points with the highest similarity, and use them to calculate the affine transformation matrix. The affine transformation matrix can transform the problem sheet image so that it perfectly overlaps with the homework image.

[0120] Step 312: Superimpose the matching graph and the operation graph according to the mapping matrix.

[0121] Specifically, the affine transformation matrix calculated in step 310 is used for matching graph transformation. Since the matching graph and the question sheet graph are consistent in size, proportion, and layout, the transformed matching graph can perfectly overlap with the homework graph.

[0122] Step 314: Segment the title frame based on the matching graph.

[0123] The title frame (ie, the title area) is segmented according to the superimposed rectangular frame area. Each title area has a correction frame area (ie, the area to be processed in the above embodiment) for attributing corrections and leaving traces.

[0124] Step 316: Detection and identification of correction traces.

[0125] Specifically, based on the YOLO training detection and recognition model (i.e., target detection model), the bbox and trace type (i.e., trace type) of all the correction traces in the homework image are obtained. In fact, according to the collection requirements and the teacher's correction habits, the correction traces can be abstracted into the following categories: correct, wrong, half-correct, etc. Common correction traces are classified into the above categories.

[0126] Step 318: Correct and keep a record of the ownership.

[0127] The correction traces are attributed according to the relationship value between the bbox of the correction trace and the correction frame bbox.

[0128] Specifically, the relationship value is used to measure how close the relationship between two bboxes is. There are several ways to calculate it: calculate the distance between the two bboxes, which can be the corner distance, the closest distance, the distance between the centers, etc.; or calculate the IoU of the bbox and various IoU variants.

[0129] In fact, a threshold needs to be set. Only the correction marks with a relationship value above the threshold are considered valid. The range of the correction mark is represented by a rectangular box bbox. When the handwriting of the correction mark is relatively long, its bbox will be very large, resulting in misjudgment when the mark is attributed by the bbox overlap IoU method. See Figure 5 , Figure 5 A schematic diagram of a correction trace is shown. Therefore, it is necessary to set a validity threshold for the trace to correctly handle the example in the figure below.

[0130] against Figure 5 The following is an explanation of the correction traces shown.

[0131] Among them, situation 1 is: because the trace of question (2) is larger, question (1) is completely covered by the two trace bboxes. The expected correction results are (1) wrong and (2) right.

[0132] Case 2: One mark covers multiple correction boxes. The expected correction result is that both questions are wrong.

[0133] Case 3 is: Similar to Case 2, one trace covers multiple correction boxes, and the expected correction results are 5 correct and 6 half correct.

[0134] At this time, it is necessary to set the corresponding processing strategy according to the actual situation. For example, considering that the tail of the trace of the check mark style (√) is easy to be elongated, causing the bbox to cover the area outside the correction target; the center of gravity of the trace target of the check mark style is in the lower left corner, so the trace of the check mark style is partitioned: the bbox is divided into four blocks, the lower left corner block has a higher weight, and the remaining 3 blocks have a lower weight. When calculating the relationship value of the trace attribution, each block and the correction box are calculated separately, and finally the weighted average is calculated according to the weight. This strategy covers the check mark style (√) and the half check mark style, and the error style (×) is not affected. According to the processing strategy, the traces of case 1 and case 3 will only be attributed to the correction box below. The trace of case 2 will be attributed to both the upper and lower correction boxes at the same time.

[0135] Since only one correction mark is allowed to belong to a correction box, if there are multiple valid correction marks around the correction box, one of them needs to be selected as the final result, that is, first sort them according to the relationship value between the correction mark bbox and the correction box bbox; then take the correction mark with a higher relationship value as the trace result of the correction box.

[0136] For questions that have no correction traces, a basic logic is set to specify a default result for this type of situation. The basic logic provided in the embodiments of this specification is: no trace + no answer = wrong, no trace + answer = right. By judging whether there is a handwritten OCR detection result within the question area, it is determined whether the question has an answer.

[0137] Step 320: Return the grading result by topic.

[0138] The image processing method provided in the embodiment of this specification specifies a correction box for each question to assist in the collection of correction traces, and the size and position of the correction box for each question can be freely defined; in order to improve the accuracy of correction box collection, a matching map is used to assist detection and recognition; by leaving traces in the correction box in the above manner, interference from other information can be eliminated as much as possible, and each question has its own correction trace area, and the traces in the area are the traces of the question, without setting complex attribution logic, which is convenient and efficient; when the teacher's traces are concentrated in the designated area, the homework diagram will also look clean and tidy; and there are differences in printers in different schools, and the edge and background colors of the correction box in the printed question sheet homework are also different, which may be too dark to affect the trace recognition, or too light to affect the correction box area detection. By using the matching map to mark the correction box, it can be effectively recalled regardless of whether it is printed too dark or too light, thereby improving the robustness of detecting and identifying the correction box, 3. Using matching information to improve the detection and recognition effect of the correction box.

[0139] Corresponding to the above method embodiment, this specification also provides an image processing device embodiment, Figure 6 FIG. 1 shows a schematic diagram of the structure of an image processing device provided by an embodiment of the present specification. Figure 6 As shown, the device comprises: The region determination module 602 is configured to determine a question region in the target image and a to-be-processed region corresponding to the question region, wherein the to-be-processed region is used to record answer correction traces corresponding to the question in the question region; The trace determination module 604 is configured to determine the answer correction trace and obtain the trace type of the answer correction trace; The position determination module 606 is configured to determine the target answer correction trace corresponding to the target to-be-processed area according to the trace position information of the answer correction trace and the area position information of the to-be-processed area, wherein the target to-be-processed area is determined from the to-be-processed area, and the target answer correction trace is determined from the answer correction trace; The result determination module 608 is configured to determine a target question area corresponding to the target area to be processed, and determine an image processing result of the target image according to a target question in the target question area and a trace type of the target answer correction trace.

[0140] Optionally, the location determination module 606 is further configured to: Determine a target area to be processed, wherein the target area to be processed is any one of the areas to be processed; Calculate the position distance between the target area to be processed and each answer correction trace in the multiple answer correction traces according to the area position information of the target area to be processed and the trace position information of the multiple answer correction traces; According to the position distance, a target answer correction trace corresponding to the target area to be processed is determined from the multiple answer correction traces.

[0141] Optionally, the location determination module 606 is further configured to: Determining, according to the trace type of each answer correction trace among the plurality of answer correction traces, a target processing strategy corresponding to each answer correction trace; Processing the trace position information of each answer correction trace according to the target processing strategy to obtain a plurality of target trace position information of each answer correction trace, wherein each target trace position information of the plurality of target trace position information includes a weight value; According to the area position information of the target area to be processed and the multiple target trace position information of the answer correction traces, the position distance between the target area to be processed and each answer correction trace in the multiple answer correction traces is calculated.

[0142] Optionally, the location determination module 606 is further configured to: According to the position distance, determining a candidate answer correction trace that meets a distance threshold from the multiple answer correction traces; Sorting the candidate answer correction traces to obtain a sorting result; According to the sorting result, the target answer correction trace corresponding to the target area to be processed is determined from the candidate answer correction traces.

[0143] Optionally, the location determination module 606 is further configured to: In the case that the target answer correction trace corresponding to the target area to be processed cannot be determined from the multiple answer correction traces according to the position distance, target object recognition is performed on the target question area corresponding to the target area to be processed to obtain a recognition result; In the case where the recognition result is that the target object is not detected, determining that the target answer correction trace corresponding to the target to-be-processed area is a first trace type; In the case where the recognition result is that the target object is detected, it is determined that the target answer correction trace corresponding to the target area to be processed is a second trace type.

[0144] The device further comprises: An initial generation module is configured to generate an initial image according to the questions selected from the question library and the selection order, wherein the initial image is an image including an initial question area, and the initial question area corresponds to an initial area to be processed; The first area position information of the initial topic area in the initial image and the second area position information of the initial area to be processed are obtained and stored in an information storage unit.

[0145] Optionally, the area determination module 602 is further configured to: Determine a target image, an initial image corresponding to the target image, first area position information of the initial question area in the initial image, and second area position information of the initial area to be processed, wherein the target image is an image obtained by an image acquisition device, with traces of answer correction added to the initial area to be processed of the initial image; A topic area in the target image is determined according to the first area position information, and a to-be-processed area in the target image corresponding to the topic area is determined according to the second area position information.

[0146] Optionally, the area determination module 602 is further configured to: According to the target image and the initial image, the first region position information and the second region position information are adjusted to obtain first target region position information and second target region position information; determining a topic area in the target image according to the first target area position information; According to the second target area position information, a to-be-processed area corresponding to the question area in the target image is determined.

[0147] Optionally, the area determination module 602 is further configured to: Performing feature extraction on the target image to obtain multiple topic area features of the target image, and performing feature extraction on the initial image to obtain multiple initial topic area features of the initial image; Calculating the regional similarity between the plurality of topic regional features and the plurality of initial topic regional features; According to the region similarity, a plurality of similar region feature pairs are determined from the plurality of topic region features and the plurality of initial topic region features, wherein a similar region feature pair includes a topic region feature and an initial topic region feature; Determining a mapping matrix between the target image and the initial image according to the plurality of similar region feature pairs; The initial image is adjusted using the mapping matrix, the first region position information is adjusted to the first target region position information, and the second region position information is adjusted to the second target region position information.

[0148] Optionally, the area determination module 602 is further configured to: In response to an image processing request sent by a client, determine a target image and associated information, wherein the image processing request carries the associated information, or the associated information is obtained by identifying the target image; An initial image is determined according to the association information, and first area position information of the initial topic area and second area position information of the initial area to be processed in the initial image are determined from the information storage unit according to the initial image.

[0149] Optionally, the result determination module 608 is further configured to: Determine the target collection result corresponding to the target question according to the target question area in the target to-be-processed area and the trace type of the target answer correction trace; The image processing result of the target image is determined according to the target acquisition result corresponding to the target topic.

[0150] The image processing device provided in the present specification has a corresponding to-be-processed area for recording answer correction traces corresponding to the questions in the question area in the target image. By recording the answer correction traces in the to-be-processed area, the target image can be made cleaner and neater, avoiding intersection and overlap of the answer correction traces with the questions or answers in the question area, and matching the identified answer correction traces with the to-be-processed area in position, and eliminating interference from other information as much as possible. Each question area has a corresponding to-be-processed area, which facilitates the use of the correspondence between the target to-be-processed area and the target answer correction traces to determine the correspondence between the target answer correction traces and the target question area corresponding to the target to-be-processed area, that is, conveniently and accurately realizing the determination of the attribution of the answer correction traces, and through the target question in the target question area and the trace type of the target answer correction traces, the answer situation of each question can be accurately judged, thereby ensuring the accuracy of the collected student homework results.

[0151] The above is a schematic scheme of an image processing device of this embodiment. It should be noted that the technical scheme of the image processing device and the technical scheme of the above-mentioned image processing method belong to the same concept, and the details not described in detail in the technical scheme of the image processing device can be referred to the description of the technical scheme of the above-mentioned image processing method.

[0152] Figure 7 The structure block diagram of a computing device 700 provided according to an embodiment of the present specification is shown. The components of the computing device 700 include but are not limited to a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and the database 750 is used to store data.

[0153] The computing device 700 also includes an access device 740 that enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 740 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0154] In one embodiment of the present specification, the above components of the computing device 700 and Figure 7 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 7 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0155] The computing device 700 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 700 may also be a mobile or stationary server.

[0156] The processor 720 implements the steps of the image processing method when executing the computer program / instructions.

[0157] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned image processing method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above-mentioned image processing method.

[0158] An embodiment of the present specification further provides a computer-readable storage medium storing a computer program / instruction. When the computer program / instruction is executed by a processor, the steps of the image processing method described above are implemented.

[0159] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned image processing method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned image processing method.

[0160] An embodiment of the present specification further provides a computer program product, including a computer program / instruction, which implements the steps of the above-mentioned image processing method when executed by a processor.

[0161] The above is a schematic solution of a computer program product of this embodiment. It should be noted that the technical solution of the computer program product and the technical solution of the above-mentioned image processing method belong to the same concept, and the details not described in detail in the technical solution of the computer program product can be referred to the description of the technical solution of the above-mentioned image processing method.

[0162] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0163] The computer program / instruction includes a computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0164] It should be noted that, for the convenience of description, the aforementioned method embodiments are all described as a series of action combinations, but those skilled in the art should be aware that this specification is not limited by the order of the actions described, because according to this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this specification.

[0165] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0166] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. An image processing method, characterized in that: include: Determine a question area in the target image, and a to-be-processed area corresponding to the question area, wherein the to-be-processed area is used to record answer correction traces corresponding to the question in the question area; Determine the answer correction trace, and obtain the trace type of the answer correction trace; Determine a target answer correction trace corresponding to a target area to be processed according to the trace position information of the answer correction trace and the area position information of the area to be processed, wherein the target area to be processed is determined from the area to be processed, and the target answer correction trace is determined from the answer correction trace; A target question area corresponding to the target area to be processed is determined, and an image processing result of the target image is determined according to the target question in the target question area and the trace type of the target answer correction trace.

2. The image processing method according to claim 1, characterized in that: The answer correction traces are multiple answer correction traces, and the to-be-processed area is multiple to-be-processed areas; The step of determining the target answer correction trace corresponding to the target area to be processed according to the trace position information of the answer correction trace and the area position information of the area to be processed includes: Determine a target area to be processed, wherein the target area to be processed is any one of the areas to be processed; Calculate the position distance between the target area to be processed and each answer correction trace in the multiple answer correction traces according to the area position information of the target area to be processed and the trace position information of the multiple answer correction traces; According to the position distance, a target answer correction trace corresponding to the target area to be processed is determined from the multiple answer correction traces.

3. The image processing method according to claim 2, characterized in that: According to the area position information of the target area to be processed and the trace position information of the multiple answer correction traces, calculating the position distance between the target area to be processed and each answer correction trace in the multiple answer correction traces, including: Determining, according to the trace type of each answer correction trace among the plurality of answer correction traces, a target processing strategy corresponding to each answer correction trace; Processing the trace position information of each answer correction trace according to the target processing strategy to obtain a plurality of target trace position information of each answer correction trace, wherein each target trace position information of the plurality of target trace position information includes a weight value; According to the area position information of the target area to be processed and the multiple target trace position information of the answer correction traces, the position distance between the target area to be processed and each answer correction trace in the multiple answer correction traces is calculated.

4. The image processing method according to claim 2, characterized in that: Determining, according to the position distance, a target answer correction trace corresponding to the target area to be processed from the multiple answer correction traces, comprising: According to the position distance, determining a candidate answer correction trace that meets a distance threshold from the multiple answer correction traces; Sorting the candidate answer correction traces to obtain a sorting result; According to the sorting result, the target answer correction trace corresponding to the target area to be processed is determined from the candidate answer correction traces.

5. The image processing method according to claim 2, characterized in that: After calculating the position distance between the target area to be processed and each answer correction trace in the multiple answer correction traces according to the area position information of the target area to be processed and the trace position information of the multiple answer correction traces, the method further includes: In the case that the target answer correction trace corresponding to the target area to be processed cannot be determined from the multiple answer correction traces according to the position distance, target object recognition is performed on the target question area corresponding to the target area to be processed to obtain a recognition result; In the case where the recognition result is that the target object is not detected, determining that the target answer correction trace corresponding to the target to-be-processed area is a first trace type; In the case where the recognition result is that the target object is detected, it is determined that the target answer correction trace corresponding to the target area to be processed is a second trace type.

6. The image processing method according to claim 1, characterized in that: Before determining the topic area in the target image and the area to be processed corresponding to the topic area, the method further includes: Generate an initial image according to the questions selected from the question library and the selection order, wherein the initial image is an image including an initial question area, and the initial question area corresponds to an initial area to be processed; The first area position information of the initial topic area in the initial image and the second area position information of the initial area to be processed are obtained and stored in an information storage unit.

7. The image processing method according to claim 6, characterized in that: The step of determining a topic area in a target image and a to-be-processed area corresponding to the topic area includes: Determine a target image, an initial image corresponding to the target image, first area position information of the initial question area in the initial image, and second area position information of the initial area to be processed, wherein the target image is an image obtained by an image acquisition device, with traces of answer correction added to the initial area to be processed of the initial image; A topic area in the target image is determined according to the first area position information, and a to-be-processed area in the target image corresponding to the topic area is determined according to the second area position information.

8. The image processing method according to claim 7, characterized in that: The determining of the topic area in the target image according to the first area position information, and the determining of the area to be processed corresponding to the topic area in the target image according to the second area position information, comprises: According to the target image and the initial image, the first region position information and the second region position information are adjusted to obtain first target region position information and second target region position information; determining a topic area in the target image according to the first target area position information; According to the second target area position information, a to-be-processed area corresponding to the question area in the target image is determined.

9. The image processing method according to claim 7, characterized in that: The adjusting the first region position information and the second region position information according to the target image and the initial image to obtain the first target region position information and the second target region position information includes: Performing feature extraction on the target image to obtain multiple topic area features of the target image, and performing feature extraction on the initial image to obtain multiple initial topic area features of the initial image; Calculating the regional similarity between the plurality of topic regional features and the plurality of initial topic regional features; According to the region similarity, a plurality of similar region feature pairs are determined from the plurality of topic region features and the plurality of initial topic region features, wherein a similar region feature pair includes a topic region feature and an initial topic region feature; Determining a mapping matrix between the target image and the initial image according to the plurality of similar region feature pairs; The initial image is adjusted using the mapping matrix, the first region position information is adjusted to the first target region position information, and the second region position information is adjusted to the second target region position information.

10. The image processing method according to claim 7, characterized in that: The determining of the target image, the initial image corresponding to the target image, the first area position information of the initial topic area in the initial image, and the second area position information of the initial area to be processed includes: In response to an image processing request sent by a client, determine a target image and associated information, wherein the image processing request carries the associated information, or the associated information is obtained by identifying the target image; An initial image is determined according to the association information, and first area position information of the initial topic area and second area position information of the initial area to be processed in the initial image are determined from the information storage unit according to the initial image.

11. The image processing method according to any one of claims 1 to 10, characterized in that: Determining the image processing result of the target image according to the target question in the target question area and the trace type of the target answer correction trace includes: Determine the target collection result corresponding to the target question according to the target question area in the target to-be-processed area and the trace type of the target answer correction trace; The image processing result of the target image is determined according to the target acquisition result corresponding to the target topic.

12. An image processing method, characterized in that: Applied to the client, including: Determine a question area in the target image, and a to-be-processed area corresponding to the question area, wherein the to-be-processed area is used to record answer correction traces corresponding to the question in the question area; Determine the answer correction trace, and obtain the trace type of the answer correction trace; Determine a target answer correction trace corresponding to a target area to be processed according to the trace position information of the answer correction trace and the area position information of the area to be processed, wherein the target area to be processed is determined from the area to be processed, and the target answer correction trace is determined from the answer correction trace; A target question area corresponding to the target area to be processed is determined, and an image processing result of the target image is determined according to the target question in the target question area and the trace type of the target answer correction trace.

13. The image processing method according to claim 12, characterized in that: The step of determining a topic area in a target image and a to-be-processed area corresponding to the topic area includes: In response to an image upload instruction from a user through an image upload interface of the client, determining a target image uploaded by the user; For the target image, a topic area in the target image and a to-be-processed area corresponding to the topic area are determined.

14. An image processing device, characterized in that: include: A region determination module is configured to determine a question region in a target image and a to-be-processed region corresponding to the question region, wherein the to-be-processed region is used to record answer correction traces corresponding to the question in the question region; A trace determination module is configured to determine the answer correction trace and obtain the trace type of the answer correction trace; a position determination module, configured to determine a target answer correction trace corresponding to a target to-be-processed area according to the trace position information of the answer correction trace and the area position information of the to-be-processed area, wherein the target to-be-processed area is determined from the to-be-processed area, and the target answer correction trace is determined from the answer correction trace; The result determination module is configured to determine a target question area corresponding to the target area to be processed, and determine an image processing result of the target image according to a target question in the target question area and a trace type of the target answer correction trace.

15. A computing device comprising a memory, a processor, and a computer program / instruction stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program / instructions, the steps of the method according to any one of claims 1 to 13 are implemented.

16. A computer-readable storage medium storing a computer program / instruction, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

17. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

Images