Image processing method and device

Abstract

The invention provides an image processing method and device, and the method comprises the steps: obtaining a to-be-processed image containing a connection question, and inputting the to-be-processed image to a connection frame detection model and a connection detection model; determining line connection frame information corresponding to line connection frames in the line connection question through the line connection frame detection model, and generating a candidate line connection set corresponding to hand-drawn lines in the line connection question through the line connection detection model; selecting a target connection line corresponding to the hand-drawn connection line from the candidate connection line set, and determining connection line information corresponding to the target connection line; and constructing a connection information pair based on the connection frame information and the connection information, and loading question detection information matched with the connection question based on the connection information pair.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and particularly to an image processing method and apparatus. Background Art

[0002] With the development of computer technology, image processing technology is applied in more and more scenarios, such as image beautification, image enhancement, image restoration, etc., all of which involve image processing technology. Especially at present when intelligent devices are becoming more and more abundant, it can not only solve problems based on the images uploaded by users, but also correct homework and assist users in learning, bringing more convenient learning services to users. In the prior art, when it comes to correcting connection questions, since the connection questions not only contain question stem information but also connection information, and since the connections are hand-drawn by users, the quality of their hand-drawing is uneven. For the recognition of such questions in the prior art, most of them are to identify the connections in the test paper after identifying the connections; then detect and recognize the text in the connection boxes at both ends of the connections, and judge according to the formulated correction strategy to determine the correctness of the connections; finally, generate a correction result to provide feedback to students or teachers. However, this processing method is not only inefficient but also has low accuracy. In more complex connection question recognition scenarios, it is difficult to provide content that meets the needs of users to them. Therefore, an effective solution is urgently needed to solve the above problems. Summary of the Invention

[0003] In view of this, embodiments of the present application provide an image processing method to solve the technical defects existing in the prior art. Embodiments of the present application also provide an image processing apparatus, a computing device, and a computer-readable storage medium.

[0004] According to the first aspect of the embodiments of the present application, an image processing method is provided, including:

[0005] Obtain a to-be-processed image containing connection questions and input it into a connection box detection model and a connection detection model;

[0006] Determine the connection box information corresponding to the connection boxes in the connection questions through the connection box detection model, and generate a candidate connection set corresponding to the hand-drawn connections in the connection questions through the connection detection model;

[0007] Select the target connection corresponding to the hand-drawn connection from the candidate connection set and determine the connection information corresponding to the target connection;

[0008] Construct a connection information pair based on the connection box information and the connection information, and load the question detection information matching the connection questions based on the connection information pair.

[0009] According to the second aspect of the embodiments of the present application, an image processing apparatus is provided, including:

[0010] An acquisition module, configured to acquire a to-be-processed image including a connection question and input it into a connection box detection model and a connection detection model;

[0011] A detection module, configured to determine connection box information corresponding to a connection box in the connection question through the connection box detection model, and generate a candidate connection set corresponding to a hand-drawn connection in the connection question through the connection detection model;

[0012] A determination module, configured to select a target connection corresponding to the hand-drawn connection from the candidate connection set and determine connection information corresponding to the target connection;

[0013] A loading module, configured to construct a connection information pair based on the connection box information and the connection information, and load question detection information matching the connection question based on the connection information pair.

[0014] According to a third aspect of the embodiments of the present application, a computing device is provided, including:

[0015] A memory and a processor;

[0016] The memory is used to store computer-executable instructions, and when the processor executes the computer-executable instructions, the steps of the image processing method are implemented.

[0017] According to a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the image processing method are implemented.

[0018] The image processing method provided in this embodiment is to improve the recognition accuracy and efficiency of connection questions, so as to quickly determine the question detection information. First, an image to be processed containing connection questions can be obtained and input into a connection box detection model and a connection detection model respectively. Thereafter, the connection box information corresponding to the connection box in the connection question can be determined through the connection box detection model, and a candidate connection set corresponding to the hand-drawn connection in the connection question can be generated through the connection detection model. By using two different deep learning models to separately detect the connection box and the connection from two different dimensions, the detection task becomes more specific to ensure the detection accuracy. Thereafter, the target connection corresponding to the hand-drawn connection is selected from the candidate connection set, and the connection information corresponding to the target connection is determined. Based on this, a connection information pair can be constructed based on the connection box information and the connection information, which can represent the connection relationship in the connection question through information and be close to the connection relationship in the connection question. Thus, the question detection information matching the connection question can be loaded based on the connection information pair, effectively improving the recognition accuracy and detection efficiency of the connection question, and facilitating the downstream service to process the connection question using the highly matching question detection information. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a schematic diagram of an image processing method provided in an embodiment of the present application;

[0020] Figure 2 is a flowchart of an image processing method provided in an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of an image in an image processing method provided in an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of an image in another image processing method provided in an embodiment of the present application;

[0023] Figure 5 is a processing flowchart of an image processing method provided in an embodiment of the present application;

[0024] Figure 6 is a schematic structural diagram of an image processing device provided in an embodiment of the present application;

[0025] Figure 7 is a structural block diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] In the following description, numerous specific details are set forth to provide a thorough understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the present application. Therefore, the present application is not limited by the specific implementations disclosed below.

[0027] The terms used in one or more embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of the present application. The singular forms "a", "the", and "said" used in one or more embodiments of the present application and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0028] It should be understood that although the terms first, second, etc. may be used in one or more embodiments of the present application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.

[0029] First, the noun terms related to one or more embodiments of the present invention are explained.

[0030] The Bézier curve, also known as the Bezier curve or Bezier curve, is a mathematical curve applied to two-dimensional graphic applications. General vector graphics software uses it to accurately draw curves.

[0031] OCR: Optical Character Recognition, which is the process by which an electronic device (such as a scanner or digital camera) examines printed characters on paper, determines their shapes by detecting dark and bright patterns, and then translates the shapes into computer text using character recognition methods; that is, for printed characters, an optical method is used to convert the text in a paper document into an image file of black and white dot matrices, and the text in the image is converted into a text format by recognition software for further editing and processing by a word processing software.

[0032] In the present application, an image processing method is provided. The present application also relates to an image processing apparatus, a computing device, and a computer-readable storage medium, which will be described in detail one by one in the following embodiments.

[0033] In practical applications, the automatic recognition of connection questions is mainly divided into two types. One is to use the instance segmentation network MaskRCNN to implement instance segmentation, segment the connection area, and determine each connection by combining rules. The other is to detect the connection based on a quadratic or cubic function curve. Although both can achieve the detection effect of connection questions, their efficiency is low, and due to the uneven writing quality of users, it is difficult to accurately divide the connection segmentation diagram into corresponding connections through rules. For the solution based on quadratic or cubic curve detection, the quadratic or cubic curve cannot fit complex curves well either. The accuracy rates of both existing technologies are not high, resulting in a low accuracy rate for subsequent marking. And in the marking stage, since connection questions appear in various disciplines and are widely applicable to different grades, their types are diverse, and it is difficult to implement a unified marking scheme. Relying solely on marking rules can only better solve the marking of relatively simple connection questions. Therefore, an effective solution is urgently needed to solve the above problems.

[0034] See Figure 1 Referring to the schematic diagram shown, for the image processing method provided in this embodiment, in order to improve the recognition accuracy and efficiency of connection questions and thus quickly determine the question detection information, the to-be-processed image containing connection questions can be obtained first and input into the connection box detection model and the connection detection model respectively; thereafter, the connection box information corresponding to the connection box in the connection question can be determined through the connection box detection model, and the candidate connection set corresponding to the hand-drawn connection in the connection question can be generated through the connection detection model; it is realized that the connection box and the connection are detected separately from two different dimensions by two different deep learning models, making the detection task more specific to ensure the detection accuracy. Thereafter, the target connection corresponding to the hand-drawn connection is selected from the candidate connection set, and the connection information corresponding to the target connection is determined; based on this, a connection information pair can be constructed based on the connection box information and the connection information, which can realize the representation of the connection relationship in the connection question through information and be close to the connection relationship in the connection question, so that the question detection information matching the loaded connection question can be realized based on the connection information pair, effectively improving the recognition accuracy and detection efficiency of connection questions and facilitating the downstream service to process the connection question with the highly matching question detection information.

[0035] See Figure 2 , Figure 2 shows the flowchart of an image processing method provided according to an embodiment of the present application, which specifically includes the following steps:

[0036] Step S202, obtain the to-be-processed image containing connection questions and input it into the connection box detection model and the connection detection model.

[0037] The image processing method provided in this embodiment can be applied to the detection scenario of connection questions in any subject, such as connection questions in Chinese, connection questions in mathematics, connection questions in English, etc. Among them, a connection question specifically refers to a question that gives a set of questions and a set of answers, and connects the questions and answers by hand-drawn connection. In this embodiment, the image processing method is described by taking the connection questions in mathematics as an example. For other identical or similar descriptions, please refer to the description content of this embodiment, and this embodiment will not elaborate too much here.

[0038] Specifically, the image to be processed specifically refers to the image uploaded by the user after shooting a connection question. This image can be used to correct the connection questions answered by the user, or to analyze the connection questions answered wrongly by the user. And this image is uploaded through the terminal device held by the user. Correspondingly, the connection box detection model specifically refers to a model that boxes the answers on the answer side and the questions on the question side in the connection question. Through this model, the rectangular box corresponding to each answer and question can be determined, as well as the position information of each rectangular box; in practical applications, the connection box detection model can be implemented using the YOLO V5 detection model. In addition, when boxing the answers on the answer side and the questions on the question side through the connection box detection model, the characters in the rectangular box can also be recognized through the OCR recognition technology for subsequent use.

[0039] Correspondingly, the connection detection model specifically refers to a model that detects the hand-drawn connection in the connection question. Through this model, a candidate connection with a mapping relationship to the hand-drawn connection can be fitted, so that when the user's hand-drawn connection is not standard, the fitted candidate connection is more standard, thus facilitating subsequent correction of the connection question based on the candidate connection with a mapping relationship to the hand-drawn connection and the output of the connection box detection model. Among them, the connection detection model can use ResNet as the backbone network to ensure the connection detection accuracy.

[0040] That is to say, the connection box detection model is used to locate the rectangular box for characters in the connection question, and the connection detection model is used to locate the candidate connection corresponding to the hand-drawn connection in the connection question. Subsequently, combining the rectangular box and the candidate connection can complete the downstream correction task.

[0041] Based on this, in order to improve the recognition accuracy and efficiency of connection questions and quickly determine the question detection information, the to-be-processed image containing connection questions can be obtained first and input into a connection box detection model and a connection detection model respectively; thereafter, the connection box information corresponding to the connection box in the connection question can be determined through the connection box detection model, and a candidate connection set corresponding to the hand-drawn connection in the connection question can be generated through the connection detection model; it is realized to detect the connection box and the connection separately from two different dimensions by two different deep learning models, making the detection task more specific to ensure the detection accuracy. Thereafter, the target connection corresponding to the hand-drawn connection is selected from the candidate connection set, and the connection information corresponding to the target connection is determined; based on this, a connection information pair is constructed based on the connection box information and the connection information, and the connection relationship in the connection question can be represented by information and is close to the connection relationship in the connection question, so that it is possible to load the question detection information matching the connection question based on the connection information pair, effectively improving the recognition accuracy and detection efficiency of the connection question and facilitating the downstream service to process the connection question with the highly matched question detection information.

[0042] Step S204, determine the connection box information corresponding to the connection box in the connection question through the connection box detection model, and generate a candidate connection set corresponding to the hand-drawn connection in the connection question through the connection detection model.

[0043] Specifically, after the to-be-processed image is input into the connection detection model and the connection box detection model, further, at this time, the connection box detection model and the connection detection model can process the to-be-processed image simultaneously, realizing determining the connection box information corresponding to the connection box in the connection question through the connection box detection model, and generating a candidate connection set corresponding to the hand-drawn connection in the connection question through the connection detection model. That is to say, through the connection box detection model, the answers and questions in the connection question can be detected, the connection box for framing the answers and questions can be determined according to the detection results, and the connection box information corresponding to the connection box. At the same time, through the connection detection model, the hand-drawn connection between the answers and questions in the connection question can be detected, and a candidate connection set composed of candidate connections corresponding to the hand-drawn connection can be generated. Thus, it is convenient to complete the detection or marking of the connection question based on the connection box information and the candidate connection set subsequently.

[0044] Among them, the connection box specifically refers to the rectangular box that frames the questions and answers in the connection question. Correspondingly, the connection box information specifically refers to the size and coordinate information corresponding to the connection box. The coordinate system corresponding to this coordinate information can be constructed based on the long side and the short side of the image to be processed, that is, taking any corner point of the image to be processed as the origin, and then constructing the horizontal axis and the vertical axis with the long side and the short side to generate a coordinate system, so as to determine the coordinate information of each connection box in this coordinate system. At the same time, since the connection box is a rectangular box, the corresponding coordinate information can be located at any corner point of the connection box or at the center point of the connection box. This embodiment does not make any limitation here. Correspondingly, the hand-drawn connection specifically refers to the hand-drawn line used to connect the answers and questions in the connection question, and this line is the line drawn by the user when answering the question. Correspondingly, the candidate connection set specifically refers to the set composed of candidate connections that have a mapping relationship with the hand-drawn connection after predicting the hand-drawn connection based on the connection detection model. It should be noted that since there may be multiple hand-drawn connections in the connection question, there can also be multiple candidate connections in the candidate connection set. At the same time, in order to improve the detection accuracy, multiple candidate connections can be generated by the connection detection model for any hand-drawn connection, which is convenient for subsequent selection of one of them as the target connection corresponding to the hand-drawn connection for processing.

[0045] Furthermore, when detecting the connection box and determining the connection box information through the connection box detection model, it is actually a process of framing and positioning the questions and answers in the connection question. In this embodiment, the specific implementation method is as follows:

[0046] Detect the connection box in the connection question through the connection box detection model, and identify the characters in the connection box; determine the connection box information corresponding to the connection box according to the detection result and the recognition result.

[0047] Specifically, the detection result specifically refers to the result of positioning and adding a rectangular box to the questions and answers in the connection question through the connection detection model. Correspondingly, the recognition result specifically refers to the result after recognizing the characters in the connection box. Based on the detection result and the recognition result, the connection box information can be obtained, that is, the position of the connection box is located, and the content of the characters in the box is obtained, which is convenient for downstream services to use.

[0048] Based on this, after obtaining the image to be processed and inputting it into the connection box detection model, in order to improve the recognition accuracy of the connection question and facilitate the use of downstream services. The connection box in the connection question can be detected first through the connection box detection model, and at the same time, the characters in the connection box can be recognized. After the detection and recognition are completed, the connection box information corresponding to the connection box can be determined according to the detection result and the recognition result.

[0049] For example, after receiving the to-be-processed image uploaded by the user after shooting a math connection question, the image is as shown in Figure 3 a shown below. To improve the accuracy of grading the connection questions answered by the user, at this time, the connection question can be input into the connection box detection model, and the connection box detection model can frame the questions and answers in the image, and a schematic diagram as shown in Figure 3 b below can be generated, and the coordinates of each connection box in the image will be located, and the character content of the questions and answers will be recognized through the OCR recognition technology to facilitate the subsequent grading of the user's answer results.

[0050] In summary, by using the connection box detection model to specifically frame and locate the questions and answers in the connection questions, the specificity of model detection can be ensured. Based on this, the detection accuracy can be improved, and thus the connection questions can be graded more accurately.

[0051] Furthermore, when determining the candidate connection information through the connection detection model, in fact, the control points corresponding to the hand-drawn connection are predicted by the model, and the candidate connections are constructed based on the control points and form a set. In this embodiment, the specific implementation method is as follows:

[0052] Preprocess the to-be-processed image and input the preprocessed to-be-processed image into the connection detection model; detect the hand-drawn connection in the connection question through the connection detection model, and determine the set of control points associated with the hand-drawn connection according to the detection result; construct the candidate connection corresponding to the hand-drawn connection according to the set of control points, and form the candidate connection set.

[0053] Specifically, preprocessing specifically refers to the operation of processing the to-be-processed image according to the input rules preset by the connection detection model, including but not limited to adjusting the size, clarity, color, etc. of the image. Correspondingly, the set of control points specifically refers to the control points obtained by predicting the hand-drawn connection through the connection detection model. After the control points are connected in series, the candidate connection can be obtained. Each hand-drawn connection can correspond to multiple subsets of control points, and each subset can construct a candidate connection corresponding to the hand-drawn connection. Therefore, multiple candidate connections can be generated for one hand-drawn connection based on multiple subsets of control points, so that a candidate connection with a high matching degree with the hand-drawn connection can be selected for use in subsequent processing, thereby improving the connection detection accuracy.

[0054] Based on this, when detecting the hand-drawn connection lines in the image to be processed through the connection line detection model, the image to be processed can be preprocessed first to realize processing the image to be processed into an image corresponding to the input of the model and deleting redundant information, and then input it into the connection line detection model; by using the connection line detection model to detect the hand-drawn connection lines in the connection line questions, the set of control points associated with the hand-drawn connection lines can be determined according to the detection results; based on this, candidate connection lines corresponding to the hand-drawn connection lines can be constructed according to the set of control points and form a candidate connection line set to facilitate the use of downstream services.

[0055] In practical applications, the connection line detection model can use ResNet as the backbone network, and the input of the model is the image to be processed, and the output is multiple Bezier control points corresponding to each hand-drawn connection line, and multiple candidate connection lines corresponding to the hand-drawn connection lines can be drawn by the multiple Bezier control points. Due to the distribution characteristics of the multiple Bezier control points corresponding to each hand-drawn connection line, each hand-drawn connection line may correspond to multiple sets of Bezier control points, and thus multiple candidate connection lines corresponding to one hand-drawn connection line can be drawn. Among them, the image format input to the model can be a fixed size, such as 640*360, etc. Considering that the multiple Bezier control points output by the model may be multiple sets of control points corresponding to the hand-drawn connection lines, in order to avoid the computational redundancy caused by too many control points, the model can be set to output multiple sets of control points corresponding to a fixed number of lines, such as the control points corresponding to 40 candidate connection lines. In practical applications, it can be set according to actual needs, and this embodiment does not make any limitation here.

[0056] In addition, in order to improve the prediction accuracy of the connection line detection model, it is necessary to complete the training in combination with sample pairs in the training stage.

[0057] In this embodiment, the specific implementation method is as follows:

[0058] Obtain a sample image containing sample connection line questions, and determine the set of sample control points corresponding to the sample hand-drawn connection lines in the sample connection line questions; use a preset fitting algorithm to fit the sample control points included in the set of sample control points, and determine the target set of sample control points according to the fitting results; input the sample image into the initial connection line detection model for processing to obtain the predicted control points corresponding to the sample hand-drawn connection lines; calculate the loss value based on the target set of sample control points and the predicted control points, and adjust the parameters of the initial connection line detection model based on the loss value until the connection line detection model that meets the training stop condition is obtained.

[0059] Specifically, the sample image specifically refers to the image used to train the initial connection detection model, which contains sample connection questions. Correspondingly, the sample hand-drawn connection is the hand-drawn connection in the sample connection question, and the sample control point set specifically refers to the sample label corresponding to the hand-drawn connection. The fitting algorithm specifically refers to an algorithm that can fit the sample control point set to obtain the target sample control point set, which can be a Bezier curve. Correspondingly, the target sample control point set specifically refers to the control points obtained after fitting the control points in the sample control point set, and its distribution is more matched with the distribution of the sample hand-drawn connection. Correspondingly, the predicted control point specifically refers to the control point corresponding to the sample hand-drawn connection obtained after predicting the sample image using the initial connection detection model. Correspondingly, the training stop condition specifically refers to the condition for stopping the training of the initial connection detection model, including but not limited to the validation set condition, the number of iterations condition, and the loss value comparison condition. In practical applications, it can be selected according to actual needs, and this embodiment does not make any limitations here.

[0060] Based on this, in the training stage of the connection detection model, the sample image containing the sample connection question can be obtained first, and the sample control point set corresponding to the sample hand-drawn connection in the sample connection question can be determined. After that, considering that the number of points in the set may be relatively large, a preset fitting algorithm can be used to fit the sample control points included in the sample control point set, so as to determine the target sample control point set according to the fitting result, thereby ensuring that the obtained control points are all matched with the sample hand-drawn connection. After that, the sample image is input into the initial connection detection model for processing to obtain the predicted control points corresponding to the sample hand-drawn connection. Based on this, the loss value can be calculated based on the target sample control point set and the predicted control points, and the initial connection detection model can be tuned based on the loss value until a connection detection model that meets the training stop condition is obtained. If the conditions are not met, new samples can be continuously selected for training.

[0061] In practical applications, when constructing the sample set of the training model, the control points corresponding to the hand-drawn connection can be fitted using a Bezier curve, so that the required number of control points can be obtained. After that, the control points distributed in [0,1] can be taken to obtain the points on a curve, which can be used as the target sample control points of the corresponding sample hand-drawn curve. Then, when training the model, a third-order Bezier curve (i.e., four control points) can be used. And when calculating the loss function, uniformly sample [0,1], obtain a series of points through the control points, and calculate the Loss between this series of points and the real points as the loss function, so as to achieve the purpose of training the model until the connection detection model is obtained for deployment.

[0062] Continuing with the above example, when obtaining the user-uploaded such as Figure 3After the image to be processed shown in a, the image to be processed can be input into the connection detection model at the same time. The connection detection model is used to detect the user's hand-drawn connections in the image, and 40 control points of the connections can be obtained according to the detection results. Thereafter, a candidate connection can be drawn according to the control points corresponding to each line, so that 10 candidate connections corresponding to the 4 hand-drawn lines by the user can be obtained and form a candidate connection set for downstream services to use.

[0063] In summary, by using a well-trained connection detection model to detect the control points of the hand-drawn connections in the image, it is possible to fit the candidate connections corresponding to the hand-drawn connections according to the distribution of the control points. Based on this, the construction of candidate connections can be realized, and the trend of the candidate connections can be made to match the hand-drawn connections better for downstream services to use.

[0064] Step S206, select the target connection corresponding to the hand-drawn connection in the candidate connection set and determine the connection information corresponding to the target connection.

[0065] Specifically, after obtaining the candidate connection set corresponding to the hand-drawn connection as described above, further, considering that there is a one-to-many relationship between the candidate connections included in the candidate connection set and the hand-drawn connection, in order to achieve the purpose of connection question detection, the target connection having a mapping relationship with the hand-drawn connection can be selected from the candidate connection set, that is, the trend of the target connection is similar to the trend of the hand-drawn connection, so as to map the connection relationship of the hand-drawn connection without changing the connection relationship of the question. At this time, the connection information of the target connection can be determined to facilitate subsequent information matching of the connection question in combination with the connection box information.

[0066] Among them, the target connection specifically refers to the candidate connection with the highest confidence corresponding to the hand-drawn connection selected from the connection set. Correspondingly, the connection information specifically refers to the information including attributes such as the endpoint coordinates and length of the target connection. It is used for subsequent matching of the relationship between the connection box and the connection, so as to map the connection result after the user answers the connection question, so that the question detection information corresponding to the connection question can be accurately found in the matching stage for grading the answer result of the connection question.

[0067] Further, when selecting the target connection corresponding to the hand-drawn connection from the candidate connection set, the screening can be completed according to the confidence corresponding to each candidate connection. In this embodiment, the specific implementation method is as follows:

[0068] Determine the confidence corresponding to each candidate connection in the candidate connection set; compare the confidence corresponding to each candidate connection with the preset confidence threshold respectively, and select the target connection corresponding to the hand-drawn connection in the candidate connection set according to the comparison result; wherein, the confidence corresponding to each candidate connection is determined by the connection detection model.

[0069] Specifically, the confidence level specifically refers to a value representing the matching degree between each candidate connection line and the hand-drawn connection line. The higher the value, the more similar the candidate connection line is to the hand-drawn connection line. On the contrary, the lower the value, the less similar the candidate connection line is to the hand-drawn connection line. Correspondingly, the preset confidence level threshold specifically refers to the threshold for comparing the confidence level corresponding to each line. The size of this threshold can be set according to actual needs, and this embodiment does not make any limitation here.

[0070] Based on this, since there are multiple candidate connection lines in the candidate connection line set, in order to determine the target connection line that matches the hand-drawn connection line, the confidence level corresponding to each candidate connection line in the candidate connection line set can be determined; then, the confidence level corresponding to each candidate connection line is respectively compared with the preset confidence level threshold, and the target connection line corresponding to the hand-drawn connection line in the candidate connection line set is determined according to the comparison result; and the confidence level corresponding to each candidate connection line is determined by the connection line detection model.

[0071] Continuing with the above example, after obtaining the 10 candidate connection lines corresponding to the 4 hand-drawn lines by the user, the confidence level Z corresponding to each candidate connection line output by the connection line detection model can be compared with the threshold. According to the comparison result, the candidate connection line L1 corresponding to the first hand-drawn line by the user, the candidate connection line L2 corresponding to the second hand-drawn line by the user, the candidate connection line L3 corresponding to the third hand-drawn line by the user, and the candidate connection line L4 corresponding to the fourth hand-drawn line by the user are determined. According to the determined target connection line corresponding to each hand-drawn connection line, a schematic diagram as shown in Figure 3 Figure c is generated. The target connection line corresponding to each hand-drawn connection line is added to the image, and at the same time, the control point coordinates and the connection line length corresponding to each target connection line are recorded, which is convenient for subsequent marking of connection line questions.

[0072] In summary, by using the method of confidence level comparison to screen the target connection line corresponding to each hand-drawn connection line, it can be ensured that the selected target connection line has a high matching relationship with the hand-drawn connection line. Based on this, the subsequent matching of question detection information can be more accurate.

[0073] Step S208: Construct a connection line information pair based on the connection line box information and the connection line information, and load the question detection information matching the connection line question based on the connection line information pair.

[0074] Specifically, after obtaining the connection box information and connection information as described above, further, considering that the target connection is the line mapping the hand-drawn connection, and the connection box is a rectangular box for framing questions and answers, while the hand-drawn connection connects the questions and answers, it is necessary to match the connection box associated with the questions and answers and the target connection associated with the hand-drawn connection, so that the answer result of the user answering the connection question can be mapped according to the matching relationship. Based on this, a connection information pair for mapping the user's answer result can be constructed based on the connection box information and the connection information, that is, an information structure corresponding to the user's answer result is constructed from the target connection and the connection box. Based on this, the question detection information of the corresponding connection question can be loaded, and the answer result of the connection question can be corrected using the question detection information. That is to say, since the connection information pair reflects the answer result of the user answering the connection question, matching the question detection information according to the more standardized connection information pair can ensure the matching accuracy, and correcting based on this can improve the accuracy of question correction.

[0075] Among them, the connection information pair specifically refers to a pair of information structures constructed based on the connection information and the connection box information with a matching relationship, and this information structure can reflect the answer result of the current user for the connection question. By detecting the connection information pair, the detection result of the user's answer result can be determined. Correspondingly, the question detection information specifically refers to the information for correcting the connection question, which can be understood as the information corresponding to the standard answer of the connection question.

[0076] Further, in the process of constructing the connection information pair, in order to ensure that the constructed connection information pair matches the relationship of the user's hand-drawn connection, it can be achieved by calculating the coordinate distance in combination with the endpoint coordinates and the connection box coordinates. In this embodiment, the specific implementation method is as follows:

[0077] Determine the endpoint coordinates corresponding to the target connection according to the connection information, and determine the connection box coordinates corresponding to the connection box according to the connection box information; calculate the coordinate distance between the endpoint coordinates and the connection box coordinates, and construct the connection information pair according to the coordinate distance.

[0078] Specifically, the endpoint coordinates specifically refer to the coordinates of the points corresponding to both ends of the target connection determined according to the connection information. Correspondingly, the connection box coordinates specifically refer to the coordinates corresponding to the connection box. Correspondingly, the coordinate distance specifically refers to the distance between the endpoint coordinates and the connection coordinates. Selecting the closest endpoint and connection box can form the connection information pair.

[0079] Based on this, after obtaining the connection information corresponding to the target connection and the connection box information corresponding to the connection box, the endpoint coordinates corresponding to the target connection can be determined according to the connection information, and the connection box coordinates corresponding to the connection box can be determined according to the connection box information; thereafter, the coordinate distance between any endpoint coordinate and any connection box coordinate can be calculated, so as to obtain the matching degree between the line and the endpoint. The closer the coordinate distance is, the more the matching degree between the connection and the connection box conforms to the user's answer result. Therefore, the line and the connection box with the closest distance can be selected according to the coordinate distance to construct a connection information pair, so as to reflect the user's answer result through the connection information pair, thus facilitating subsequent marking.

[0080] In summary, by calculating the coordinate distance between the endpoint coordinates and the connection box coordinates, it can be obtained whether the connection between the connection and the connection box conforms to the user's answer result. Based on this, the connection information pairs that match the user's answer result can be screened out, and based on this, the subsequent marking accuracy is higher.

[0081] Furthermore, when determining the question detection information, it can be completed by calculating the information phase velocity. In this embodiment, the specific implementation method is as follows:

[0082] Determine the question database associated with the connection question; calculate the information similarity between the connection information pair and the question information included in the question database; extract the question detection information matching the connection question from the question database according to the information similarity; wherein, the question information included in the question database is composed of question connection box information and question connection information.

[0083] Specifically, the question database specifically refers to a database storing a large number of connection questions, and the questions stored in this database have the same subject as the question that the user is currently answering. Correspondingly, the information similarity specifically refers to a numerical value representing the similarity degree between the connection information pair and the question information in the question database. The higher the information similarity, the higher the matching degree between the connection information pair and the question information. In specific implementation, since the connection information pair represents the information of the connection relationship between the connection box and the target connection, and the question information is also the information of the connection relationship corresponding to the questions in the database, when calculating the information similarity, it can be directly completed from the graph dimension, that is, calculate the similarity according to the graph of the connection box and the target connection, and the graph corresponding to the question information in the question database, and the information similarity can be obtained. Or a matrix expression can be constructed according to the connection information pair, and a matrix expression can be constructed according to the question information, and the information similarity can be obtained by calculating the similarity between the matrices.

[0084] Based on this, when selecting the question detection information corresponding to the connection question, the question database associated with the connection question can be determined; thereafter, the information similarity between the connection information pair and the question information included in the question database can be calculated, and then the question detection information matching the connection question can be extracted from the question database according to the information similarity. Moreover, for the convenience of calculation, the question information included in the question database can also be composed of the question connection box information and the question connection information, and its construction process can refer to the construction process of the connection information pair, which is not limited in this embodiment.

[0085] In summary, by calculating the information similarity to extract the question detection information matching the connection question from the question database, it can be ensured that the detected question detection information is the same as the connection question, and the user's answer result can be marked based on this, which can improve the marking accuracy.

[0086] After obtaining the question detection information, the marking information can also be determined according to the question detection information, so as to complete the marking process of the connection question. In this embodiment, the specific implementation method is as follows:

[0087] In the case where the question detection information matches the connection question and the matching degree does not meet the matching degree threshold, the marking information is determined according to the question detection information; a question marking task is constructed and executed based on the marking information, and the marking result is fed back to the user terminal corresponding to the image to be processed according to the task execution result.

[0088] Specifically, the marking information specifically refers to the information for marking the answer result of the connection question. Correspondingly, the question marking task specifically refers to the task of adding marking traces in the image. Based on this, in the case where the question detection information matches the connection question and the matching degree does not meet the matching degree threshold, it means that the obtained question detection information matches the connection question at this time, and the user's connection answer result is incorrect. Therefore, the marking information can be determined according to the question detection information; then a question marking task is constructed and executed based on the marking information, and the marking result can be fed back to the user terminal corresponding to the image to be processed according to the task execution result.

[0089] In practical applications, on the basis of predicting the connection box information and the connection information through the connection detection model and the connection box detection model, the two endpoints of the connection can be used to calculate the connection box closest to the endpoints, and the two connection boxes corresponding to the endpoints can be selected as a pair of connection boxes, which match the user's answer result. Thereafter, the OCR recognition result in the connection box is extracted, and the connection information pair can be formed. Then, the question detection information matching the connection information pair is retrieved from the constructed question database.

[0090] In specific implementation, the question information included in the question database can be completed by constructing connection information pairs, and in the retrieval stage, it can be completed by calculating similarity, so as to effectively improve the information matching efficiency. At the same time, there is no need to update the cache on the local device.

[0091] Continuing with the above example, after determining the connection box information corresponding to the user's answer result and the connection information corresponding to the connection, the distances between the two endpoints of the connection and each connection box can be calculated. According to the distances, it is determined that line L1 corresponds to connection boxes 1-1 and 1-2, line L2 corresponds to connection boxes 2-1 and 2-2, line L3 corresponds to connection boxes 3-1 and 3-2, and line L3 corresponds to connection boxes 3-1 and 3-2. Thereafter, connection information pairs can be constructed by combining the connections, connection boxes, and characters within the connection boxes, and based on this, a search can be performed in the question database. In the retrieval stage, the similarity between the question information already stored in the question database and the paired connection information is calculated, and the answer corresponding to the math connection question is determined according to the similarity calculation result. Thereafter, the user's answer result can be corrected according to the answer, and the correction information can be added to the image to be processed, and then the corrected image as shown in Figure 4 the figure can be obtained and fed back to the user's terminal.

[0092] For the image processing method provided in this embodiment, in order to improve the recognition accuracy and efficiency of connection questions and thus quickly determine the question detection information, a to-be-processed image containing connection questions can be obtained first and input into a connection box detection model and a connection detection model respectively. Thereafter, the connection box information corresponding to the connection boxes in the connection question can be determined by the connection box detection model, and a candidate connection set corresponding to the hand-drawn connection in the connection question can be generated by the connection detection model. By using two different deep learning models to detect the connection boxes and connections separately from two different dimensions, the detection task becomes more specific to ensure the detection accuracy. Thereafter, the target connection corresponding to the hand-drawn connection is selected from the candidate connection set, and the connection information corresponding to the target connection is determined. Based on this, connection information pairs can be constructed based on the connection box information and the connection information, which can represent the connection relationship in the connection question through information and be close to the connection relationship in the connection question. Thus, it is possible to load the question detection information matching the connection question based on the connection information pair, effectively improving the recognition accuracy and detection efficiency of the connection question and facilitating downstream services to process the connection question using the highly matching question detection information.

[0093] The following combines the attached Figure 5 Taking the application of the image processing method provided in this application in the connection question recognition scenario as an example, the image processing method will be further described. Among them, Figure 5 shows a processing flow chart of an image processing method provided in an embodiment of this application, which specifically includes the following steps:

[0094] Step S502: Obtain the image to be processed containing connection questions and input it into the connection box detection model.

[0095] Step S504: Detect the connection boxes in the connection questions through the connection box detection model, and recognize the characters within the connection boxes.

[0096] Step S506: Determine the connection box information corresponding to the connection boxes according to the detection results and recognition results.

[0097] Step S508: Preprocess the image to be processed and input the preprocessed image to be processed into the connection detection model.

[0098] Step S510: Detect the hand-drawn connections in the connection questions through the connection detection model, and determine the set of control points associated with the hand-drawn connections according to the detection results.

[0099] Step S512: Construct candidate connections corresponding to the hand-drawn connections according to the set of control points and form a candidate connection set.

[0100] Step S514: Determine the confidence level corresponding to each candidate connection in the candidate connection set.

[0101] Step S516: Compare the confidence level corresponding to each candidate connection with the preset confidence threshold respectively, select the target connection corresponding to the hand-drawn connection in the candidate connection set according to the comparison results, and determine the connection information corresponding to the target connection.

[0102] Step S518: Determine the endpoint coordinates corresponding to the target connection according to the connection information, and determine the connection box coordinates corresponding to the connection box according to the connection box information.

[0103] Step S520: Calculate the coordinate distance between the endpoint coordinates and the connection box coordinates, and construct a pair of connection information according to the coordinate distance.

[0104] Step S522: Determine the question database associated with the connection questions, and calculate the information similarity between the pair of connection information and the question information contained in the question database.

[0105] Step S524: Extract the question detection information matching the connection questions from the question database according to the information similarity, where the question information contained in the question database is composed of question connection box information and question connection information.

[0106] The image processing method provided in this embodiment can improve the recognition accuracy and efficiency of connection questions, so as to quickly determine the question detection information. First, an image to be processed containing connection questions can be obtained and input into a connection box detection model and a connection detection model respectively. Then, the connection box information corresponding to the connection box in the connection question can be determined through the connection box detection model, and a candidate connection set corresponding to the hand-drawn connection in the connection question can be generated through the connection detection model. By using two different deep learning models to detect the connection box and the connection separately from two different dimensions, the detection task becomes more specific to ensure the detection accuracy. Then, the target connection corresponding to the hand-drawn connection is selected from the candidate connection set, and the connection information corresponding to the target connection is determined. Based on this, a connection information pair can be constructed based on the connection box information and the connection information, so that the connection relationship in the connection question can be represented by information and is close to the connection relationship in the connection question. Thus, the question detection information matching the connection question can be loaded based on the connection information pair, effectively improving the recognition accuracy and detection efficiency of the connection question, and facilitating downstream services to process the connection question with highly matched question detection information.

[0107] Corresponding to the above method embodiment, the present application also provides an embodiment of an image processing device. Figure 6 The structural schematic diagram of an image processing device provided by an embodiment of the present application is shown. As Figure 6 shown, the device includes:

[0108] An acquisition module 602, configured to acquire an image to be processed containing connection questions and input it into a connection box detection model and a connection detection model;

[0109] A detection module 604, configured to determine the connection box information corresponding to the connection box in the connection question through the connection box detection model, and generate a candidate connection set corresponding to the hand-drawn connection in the connection question through the connection detection model;

[0110] A determination module 606, configured to select the target connection corresponding to the hand-drawn connection from the candidate connection set and determine the connection information corresponding to the target connection;

[0111] A loading module 608, configured to construct a connection information pair based on the connection box information and the connection information, and load the question detection information matching the connection question based on the connection information pair.

[0112] In an optional embodiment, the detection module 604 is further configured to:

[0113] Detect the connection boxes in the connection questions through the connection box detection model, and recognize the characters within the connection boxes; determine the connection box information corresponding to the connection boxes according to the detection results and recognition results.

[0114] In an optional embodiment, the detection module 604 is further configured to: preprocess the to-be-processed image, and input the preprocessed to-be-processed image into the connection detection model; detect the hand-drawn connections in the connection questions through the connection detection model, determine the set of control points associated with the hand-drawn connections according to the detection results; construct candidate connections corresponding to the hand-drawn connections according to the set of control points, and form the candidate connection set.

[0115] In an optional embodiment, the determination module 606 is further configured to:

[0116] Determine the confidence level corresponding to each candidate connection in the candidate connection set; compare the confidence level corresponding to each candidate connection with a preset confidence threshold respectively, and select the target connection corresponding to the hand-drawn connection from the candidate connection set according to the comparison results; wherein, the confidence level corresponding to each candidate connection is determined by the connection detection model.

[0117] In an optional embodiment, the loading module 608 is further configured to:

[0118] Determine the endpoint coordinates corresponding to the target connection according to the connection information, and determine the connection box coordinates corresponding to the connection box according to the connection box information; calculate the coordinate distance between the endpoint coordinates and the connection box coordinates, and construct the connection information pair according to the coordinate distance.

[0119] In an optional embodiment, the loading module 608 is further configured to:

[0120] Determine the question database associated with the connection questions; calculate the information similarity between the connection information pair and the question information included in the question database; extract the question detection information matching the connection questions from the question database according to the information similarity; wherein, the question information included in the question database is composed of question connection box information and question connection information.

[0121] In an optional embodiment, the training of the connection detection model includes:

[0122] Obtain a sample image containing a sample connection question, and determine a set of sample control points corresponding to the sample hand-drawn connection in the sample connection question; use a preset fitting algorithm to fit the sample control points included in the set of sample control points, and determine a set of target sample control points according to the fitting result; input the sample image into an initial connection detection model for processing to obtain predicted control points corresponding to the sample hand-drawn connection; calculate a loss value based on the set of target sample control points and the predicted control points, and adjust the parameters of the initial connection detection model based on the loss value until the connection detection model that meets the training stop condition is obtained.

[0123] In an optional embodiment, the device further includes:

[0124] A matching module, configured to determine marking information according to the question detection information in the case where the question detection information matches the connection question and the matching degree does not meet the matching degree threshold; construct and execute a question marking task based on the marking information, and feedback a marking result to the user terminal corresponding to the image to be processed according to the task execution result.

[0125] In order to improve the recognition accuracy and efficiency of connection questions and thus quickly determine question detection information, the image processing device provided in this embodiment can first obtain an image to be processed containing a connection question and input it into a connection box detection model and a connection detection model respectively; thereafter, the connection box information corresponding to the connection box in the connection question can be determined through the connection box detection model, and a candidate connection set corresponding to the hand-drawn connection in the connection question can be generated through the connection detection model; it is realized that the connection box and the connection are detected separately from two different dimensions by two different deep learning models, making the detection task more specific to ensure the detection accuracy. Thereafter, select a target connection corresponding to the hand-drawn connection from the candidate connection set, and determine the connection information corresponding to the target connection; based on this, a connection information pair can be constructed based on the connection box information and the connection information, which can realize the representation of the connection relationship in the connection question through information and be close to the connection relationship in the connection question, so that it is possible to realize the loading of question detection information matching the connection question based on the connection information, effectively improving the recognition accuracy and detection efficiency of the connection question, and facilitating the downstream service to process the connection question using the highly matching question detection information.

[0126] The above is a schematic solution of an image processing device according to this embodiment. It should be noted that the technical solution of this image processing device and the technical solution of the above image processing method belong to the same concept. For the details not described in detail in the technical solution of the image processing device, reference can be made to the description of the technical solution of the above image processing method. In addition, each component in the device embodiment should be understood as a functional module that must be established to implement each step of the program flow or each step of the method. Each functional module is not an actual functional segmentation or separation limitation. The device claim defined by such a set of functional modules should be understood as mainly implementing the functional module architecture of the solution through the computer program recorded in the specification, rather than understanding as an entity device mainly implementing the solution through hardware means.

[0127] Figure 7 FIG. shows a structural block diagram of a computing device 700 according to an embodiment of the present application. The components of the computing device 700 include, but are not limited to, a memory 710 and a processor 720. The processor 720 is connected to the memory 710 through a bus 730, and a database 750 is used to store data.

[0128] The computing device 700 further includes an access device 740, which enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 740 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0129] In an embodiment of the present application, the above components of the computing device 700 and Figure 7Other components not shown may also be connected to each other, for example, via a bus. It should be understood that Figure 7 The block diagram of the computing device shown is for illustrative purposes only and is not a limitation on the scope of the present application. Those skilled in the art can add or replace other components as needed.

[0130] The computing device 700 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.) or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 700 can also be a mobile or stationary server.

[0131] Among them, the processor 720 is used to execute the computer-executable instructions of the image processing method.

[0132] The above is a schematic solution of a computing device according to this embodiment. It should be noted that the technical solution of the computing device and the technical solution of the above image processing method belong to the same concept. For the details not described in detail in the technical solution of the computing device, reference can be made to the description of the technical solution of the above image processing method.

[0133] An embodiment of the present application also provides a computer-readable storage medium, which stores computer instructions that are executed by a processor for an image processing method.

[0134] The above is a schematic solution of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of the storage medium and the technical solution of the above image processing method belong to the same concept. For the details not described in detail in the technical solution of the storage medium, reference can be made to the description of the technical solution of the above image processing method.

[0135] The computer instructions include computer program code, which may be in the form of source code, object code, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0136] It should be noted that for the foregoing method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, some steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily all essential to this application.

[0137] In the above embodiments, the descriptions of each embodiment have their own emphases. For the parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0138] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The alternative embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made according to the content of this application. The present application selects and specifically describes these embodiments in order to better explain the principle and practical application of the present application, so that those skilled in the art can well understand and utilize the present application. The present application is only limited by the claims and their full scope and equivalents.

Claims

1. An image processing method, characterized in that, comprising: Obtain the to-be-processed image containing the connection question and input it into the connection box detection model and the connection detection model; Determine the connection box information corresponding to the connection box in the connection question through the connection box detection model, and generate a candidate connection set corresponding to the hand-drawn connection in the connection question through the connection detection model; Select the target connection corresponding to the hand-drawn connection from the candidate connection set, and determine the connection information corresponding to the target connection; Construct a connection information pair based on the connection box information and the connection information, and load the question detection information matching the connection question based on the connection information pair.

2. The method according to claim 1, characterized in that, The step of determining the connection box information corresponding to the connection box in the connection question through the connection box detection model includes: Detect the connection box in the connection question through the connection box detection model, and recognize the characters in the connection box; Determine the connection box information corresponding to the connection box according to the detection result and the recognition result.

3. The method according to claim 1, characterized in that, Before the step of generating the candidate connection set corresponding to the hand-drawn connection in the connection question through the connection detection model, it further includes: Preprocess the to-be-processed image and input the preprocessed to-be-processed image into the connection detection model; The step of generating the candidate connection set corresponding to the hand-drawn connection in the connection question through the connection detection model includes: Detect the hand-drawn connection in the connection question through the connection detection model, and determine the control point set associated with the hand-drawn connection according to the detection result; Construct the candidate connection corresponding to the hand-drawn connection according to the control point set, and form the candidate connection set.

4. The method according to claim 1, characterized in that, The step of selecting the target connection corresponding to the hand-drawn connection from the candidate connection set includes: Determine the confidence corresponding to each candidate connection in the candidate connection set; Compare the confidence corresponding to each candidate connection with the preset confidence threshold respectively, and select the target connection corresponding to the hand-drawn connection from the candidate connection set according to the comparison result; Wherein, the confidence corresponding to each candidate connection is determined by the connection detection model.

5. The method according to claim 1, characterized in that, The step of constructing the connection information pair based on the connection box information and the connection information includes: Determine the endpoint coordinates corresponding to the target connection according to the connection information, and determine the connection box coordinates corresponding to the connection box according to the connection box information; Calculate the coordinate distance between the endpoint coordinates and the connection box coordinates, and construct the connection information pair according to the coordinate distance.

6. The method according to claim 1, characterized in that, The step of loading the question detection information matching the connection question based on the connection information pair includes: Determine the question database associated with the connection question; Calculate the information similarity between the connection information pair and the question information included in the question database; Extract the question detection information matching the connected questions in the question database according to the information similarity; Among them, the question information included in the question database is composed of question connection box information and question connection information.

7. The method according to claim 3, characterized in that the training of the connection detection model includes: obtain a sample image containing sample connected questions, and determine a set of sample control points corresponding to the sample hand-drawn connection in the sample connected questions; fit the sample control points included in the set of sample control points by using a preset fitting algorithm, and determine a set of target sample control points according to the fitting result; input the sample image into an initial connection detection model for processing to obtain predicted control points corresponding to the sample hand-drawn connection; calculate a loss value based on the set of target sample control points and the predicted control points, and adjust the parameters of the initial connection detection model based on the loss value until the connection detection model that meets the training stop condition is obtained.

8. The method according to any one of claims 1-7, characterized in that after the step of loading the question detection information matching the connected questions based on the connection information, it further includes: in the case where the question detection information matches the connected questions and the matching degree does not meet the matching degree threshold, determine correction information according to the question detection information; construct a question correction task based on the correction information and execute it, and feedback the correction result to the user terminal corresponding to the image to be processed according to the task execution result.

9. An image processing device, characterized in that it includes: an acquisition module configured to acquire an image to be processed containing connected questions and input it into a connection box detection model and a connection detection model; a detection module configured to determine connection box information corresponding to the connection box in the connected questions through the connection box detection model, and generate a candidate connection set corresponding to the hand-drawn connection in the connected questions through the connection detection model; a determination module configured to select a target connection corresponding to the hand-drawn connection from the candidate connection set and determine connection information corresponding to the target connection; a loading module configured to construct a pair of connection information based on the connection box information and the connection information, and load the question detection information matching the connected questions based on the connection information.

10. A computing device, characterized in that it includes: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing computer instructions, characterized in that when the instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images