Connection question correcting method and device, electronic equipment, medium and product

Abstract

The invention relates to the technical field of data processing, and particularly provides a connection question correcting method and device, electronic equipment, a medium and a product. The method for correcting the connection question comprises the following steps: determining each first target object in a target standard image and a target area of a matched second target object in a to-be-corrected image; detecting an end point of at least one connecting line in the to-be-corrected image to obtain an end point position corresponding to the at least one connecting line; according to the position of each endpoint and the target area of each second target object, screening out a target connection object matched with each endpoint from each second target object; according to the target connection object of each endpoint, forming a target connection object combination of at least one connection line; and comparing the respective target connection object combination of the at least one connection line with the at least one standard object combination to obtain a connection line question correction result of the to-be-corrected image. Therefore, the problem that the connection problem is difficult to correct effectively is solved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, electronic equipment, medium and product for correcting connecting-line questions. Background Art

[0002] Currently, photographing and judging questions has been widely used in the field of education. Under the existing technology, the test paper submitted by the examinee can usually be photographed so that the test paper image can be automatically marked by the marking device.

[0003] However, the above method can only handle logical operation type questions, such as arithmetic questions. Under the existing technology, it is usually difficult to effectively mark the connection questions through the marking equipment. Therefore, how to correctly and effectively mark the connection questions is a problem that needs to be solved. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a method, device, electronic device, medium and product for correcting connecting-line questions, so as to solve the problem that it is difficult to effectively correct connecting-line questions.

[0005] On the one hand, an embodiment of the present application provides a method for correcting a connecting question, the method comprising:

[0006] Determine each first target object in the target standard image and a target area of ​​a second target object matched in the image to be corrected; the first target object is an object to be connected;

[0007] By using an endpoint detection model, the endpoints of at least one connecting line in the image to be corrected are detected to obtain the endpoint positions corresponding to each of the at least one connecting line; the endpoint detection model is used to detect the endpoint positions of the two endpoints of the connecting line;

[0008] According to the positions of the endpoints and the target areas of the second target objects, the target connection objects that match the endpoints are selected from the second target objects;

[0009] According to the respective target connection objects of the respective endpoints, forming at least one respective target connection object combination of the respective connection lines;

[0010] The target connection object combination of at least one connection line is compared with at least one standard object combination corresponding to the target standard image to obtain the connection question correction result of the image to be corrected.

[0011] In one embodiment, before determining the target area of ​​each first target object in the target standard image and the second target object matched in the image to be corrected, the method further includes:

[0012] Acquire a standard image set; the standard image set includes at least one standard image;

[0013] Determine the similarity between the image to be corrected and each standard image in the standard image set;

[0014] The target standard image is selected from the standard image set according to the similarity of each standard image.

[0015] In one embodiment, the step of determining the target region of the second target object that matches each first target object in the target standard image in the image to be corrected includes:

[0016] According to the key point detection model, a plurality of key point pairs matching the target standard image and the image to be corrected are detected, each key point pair comprising a first key point in the target standard image and a second key point corresponding to the first key point in the image to be corrected;

[0017] For each first target object, perform the following steps:

[0018] According to the positions of the first key points and the image region where the first target object is located, a plurality of target key points located in the image region of the first target object are screened out from the first key points;

[0019] According to each key pair, determine the second key point corresponding to each target key point;

[0020] The second key points corresponding to each target key point are connected to obtain a target area of ​​the second target object corresponding to the first target object.

[0021] In one implementation, the step of filtering out target connection objects that match each endpoint from each second target object according to the position of each endpoint and the target area of ​​each second target object includes:

[0022] For each endpoint separately, perform the following steps:

[0023] Determine, according to the endpoint position of the endpoint and the target area of ​​each second target object, the connection distance between the endpoint and each second target object;

[0024] Determine the minimum distance among the connection distances;

[0025] The second target object corresponding to the minimum distance is determined as the target connection object for endpoint matching.

[0026] In one embodiment, comparing the target connection object combination of the at least one connection line with at least one standard object combination corresponding to the target standard image to obtain the connection question correction result of the image to be corrected includes:

[0027] For each connection, the following steps are performed respectively: if it is determined that there is a standard object combination matching the target connection object combination of the connection in the at least one standard object combination, then the correction result of the connection is determined to be a correct connection; otherwise, the correction result of the connection is determined to be an incorrect connection;

[0028] According to the correction results of each connection, the correction result of the connection question is obtained.

[0029] In one embodiment, the endpoint detection model is trained by following the steps below:

[0030] Acquire training data; the training data includes a plurality of connection samples and their corresponding endpoint sample positions;

[0031] Input each link sample in the training data into the initial detection model to obtain the predicted endpoint position corresponding to each link sample;

[0032] Obtain the prediction loss according to the predicted endpoint positions and endpoint sample positions corresponding to each connection sample;

[0033] According to the predicted endpoint positions corresponding to the link samples, the first link vectors corresponding to the link samples are obtained;

[0034] According to the endpoint sample positions corresponding to the line samples, the second connection vectors corresponding to the line samples are obtained;

[0035] Obtaining a vector loss according to the first connection vector and the second connection vector corresponding to each connection sample;

[0036] According to the prediction loss and the vector loss, the parameters of the initial detection model are adjusted until the endpoint detection model that meets the set training conditions is obtained.

[0037] In one implementation manner, obtaining the vector loss according to the first connection vector and the second connection vector corresponding to each connection sample includes:

[0038] Use any of the following:

[0039] Determine the vector loss according to the similarity between the first connection vector and the second connection vector corresponding to each connection sample;

[0040] Determine the vector loss according to a difference between a first modulus length of a first connection vector and a second modulus length of a second connection vector corresponding to each of the connection samples;

[0041] The vector loss is determined according to the similarity between the first connection vector and the second connection vector corresponding to each connection sample, and the difference between the first modulus length of the first connection vector and the second modulus length of the second connection vector corresponding to each connection sample.

[0042] On the one hand, an embodiment of the present application provides a device for correcting a connecting question, comprising:

[0043] A determination unit, used to determine, for each first target object in the target standard image, a target area of ​​a second target object matched in the image to be corrected; the first target object is an object to be connected;

[0044] A detection unit, used to detect the endpoints of at least one line in the image to be corrected by using an endpoint detection model, and obtain the endpoint positions corresponding to each of the at least one line; the endpoint detection model is used to detect the endpoint positions of two endpoints of the line;

[0045] A matching unit, configured to filter out target connection objects that match each endpoint from each second target object according to the position of each endpoint and the target area of ​​each second target object;

[0046] A composition unit, configured to compose a target connection object combination of the at least one connection according to the target connection objects of the respective endpoints;

[0047] The obtaining unit is used to compare the target connection object combination of the at least one connection line with at least one standard object combination corresponding to the target standard image to obtain the connection question correction result of the image to be corrected.

[0048] In one implementation, the determining unit is further configured to:

[0049] Acquire a standard image set; the standard image set includes at least one standard image;

[0050] Determine the similarity between the image to be corrected and each standard image in the standard image set;

[0051] The target standard image is selected from the standard image set according to the similarity of each standard image.

[0052] In one implementation, the determining unit is used to:

[0053] According to the key point detection model, a plurality of key point pairs matching the target standard image and the image to be corrected are detected, each key point pair comprising a first key point in the target standard image and a second key point corresponding to the first key point in the image to be corrected;

[0054] For each first target object, perform the following steps:

[0055] According to the positions of the first key points and the image region where the first target object is located, a plurality of target key points located in the image region of the first target object are screened out from the first key points;

[0056] According to each key pair, determine the second key point corresponding to each target key point;

[0057] The second key points corresponding to each target key point are connected to obtain a target area of ​​the second target object corresponding to the first target object.

[0058] In one implementation, the matching unit is used to:

[0059] For each endpoint separately, perform the following steps:

[0060] Determine, according to the endpoint position of the endpoint and the target area of ​​each second target object, the connection distance between the endpoint and each second target object;

[0061] Determine the minimum distance among the connection distances;

[0062] The second target object corresponding to the minimum distance is determined as the target connection object for endpoint matching.

[0063] In one embodiment, the obtaining unit is used to:

[0064] For each connection, the following steps are performed respectively: if it is determined that there is a standard object combination matching the target connection object combination of the connection in the at least one standard object combination, then the correction result of the connection is determined to be a correct connection; otherwise, the correction result of the connection is determined to be an incorrect connection;

[0065] According to the correction results of each connection, the correction result of the connection question is obtained.

[0066] In one embodiment, the detection unit is further used for:

[0067] The endpoint detection model is trained using the following steps:

[0068] Acquire training data; the training data includes a plurality of connection samples and their corresponding endpoint sample positions;

[0069] Input each link sample in the training data into the initial detection model to obtain the predicted endpoint position corresponding to each link sample;

[0070] Obtain the prediction loss according to the predicted endpoint positions and endpoint sample positions corresponding to each connection sample;

[0071] According to the predicted endpoint positions corresponding to the link samples, the first link vectors corresponding to the link samples are obtained;

[0072] According to the endpoint sample positions corresponding to the line samples, the second connection vectors corresponding to the line samples are obtained;

[0073] Obtaining a vector loss according to the first connection vector and the second connection vector corresponding to each connection sample;

[0074] According to the prediction loss and the vector loss, the parameters of the initial detection model are adjusted until the endpoint detection model that meets the set training conditions is obtained.

[0075] In one embodiment, the detection unit is further used for:

[0076] Use any of the following:

[0077] Determine the vector loss according to the similarity between the first connection vector and the second connection vector corresponding to each connection sample;

[0078] Determine the vector loss according to a difference between a first modulus length of a first connection vector and a second modulus length of a second connection vector corresponding to each connection sample;

[0079] The vector loss is determined according to the similarity between the first connection vector and the second connection vector corresponding to each connection sample, and the difference between the first modulus length of the first connection vector and the second modulus length of the second connection vector corresponding to each connection sample.

[0080] On the one hand, an embodiment of the present application provides an electronic device, including:

[0081] Processor; and

[0082] The memory stores computer instructions, wherein the computer instructions are used to cause the processor to execute the steps of the method provided in any of the various optional implementations of the connecting-the-dot question correction described above.

[0083] On the one hand, an embodiment of the present application provides a computer-readable storage medium storing computer instructions, which are used to enable a computer to execute the steps of the method provided in any of the various optional implementations of the connecting-the-dot question grading described above.

[0084] On the one hand, a computer program product is provided in an embodiment of the present application, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the steps of the method provided in any of the various optional implementations of the above-mentioned connecting-the-dot question grading.

[0085] The method for correcting the connection questions in the embodiment of the present application includes determining the target area of ​​the second target object matched in the image to be corrected for each first target object in the target standard image; the first target object is an object for being connected; through the endpoint detection model, the endpoint of at least one connection in the image to be corrected is detected to obtain the endpoint position corresponding to each of the at least one connection; the endpoint detection model is used to detect the endpoint positions of the two endpoints of the connection; according to the positions of each endpoint and the target area of ​​each second target object, the target connection objects that each endpoint matches are screened out from each second target object; according to the target connection objects of each endpoint, the target connection object combination of each of the at least one connection is formed; the target connection object combination of each of the at least one connection is compared with at least one standard object combination corresponding to the target standard image to obtain the connection question correction result of the image to be corrected. In this way, through the method of endpoint detection and target object matching, the connection questions can be corrected effectively and accurately, solving the problem of difficulty in effectively correcting the connection questions. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0087] Figure 1 It is a flow chart of a method for correcting connecting-line questions in an embodiment of the present application.

[0088] Figure 2 It is an example diagram of a target standard image in an embodiment of the present application.

[0089] Figure 3 This is an example diagram of key points in an embodiment of the present application.

[0090] Figure 4 It is a detailed flow chart of a method for correcting connecting questions in an embodiment of the present application.

[0091] Figure 5 It is a structural block diagram of a device for correcting connecting questions in an embodiment of the present application.

[0092] Figure 6 It is a structural schematic diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0093] The technical solution of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described implementation methods are part of the implementation methods of the present application, rather than all of the implementation methods. Based on the implementation methods in the present application, all other implementation methods obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In addition, the technical features involved in the different implementation methods of the present application described below can be combined with each other as long as they do not conflict with each other.

[0094] Under traditional technology, usually only logical operation type questions in the test paper image can be marked by marking equipment, but it is difficult to effectively mark the connection questions.

[0095] Based on the defects of the above-mentioned related technologies, the embodiments of the present application provide a method, device, electronic device, medium and product for correcting connecting-line questions, aiming to solve the problem that it is difficult to effectively correct connecting-line questions.

[0096] A method for correcting connecting-line questions is provided in an embodiment of the present application. The method can be applied to electronic devices. The present application does not limit the type of electronic device, which can be any type of device suitable for implementation, such as a terminal device and a server, etc. The present application will not go into details about this.

[0097] See also Figure 1 As shown in the figure, it is a flow chart of the method for correcting the connecting question in the embodiment of the present application. Figure 1 The method is described below. The specific implementation process of the method is as follows:

[0098] Step 101: Determine each first target object in the target standard image and a target area of ​​a second target object to be matched in the image to be corrected; the first target object is an object to be connected.

[0099] In one implementation, an image to be corrected input by a user is obtained, and a target standard image corresponding to the image to be corrected is searched for, and a target region of a second target object in the image to be corrected is determined according to a first target object in the target standard image.

[0100] In one implementation, when obtaining a target standard image corresponding to the image to be corrected, the following steps may be used:

[0101] S101 - 11: Acquire a standard image set; the standard image set includes at least one standard image.

[0102] In one implementation, when it is determined that a connect-the-line question correction request for an image to be corrected is received, the image to be corrected in the connect-the-line question correction request is obtained, and a standard image set is obtained from a database.

[0103] The image to be corrected is an image of a user's (e.g., a test taker's) connection question, which can also be called a user image. The image to be corrected includes the connection question and usually also includes the connection drawn by the user. For example, after the user answers the connection question, he / she takes a photo of it, obtains and uploads the image to be corrected.

[0104] The standard image may be an image containing one or more test questions. For example, each test question in one or more test papers is photographed to obtain the standard image corresponding to each test question. Furthermore, the standard answer corresponding to each standard image may be obtained, such as the standard object combination corresponding to the target standard image, that is, the standard answer to the connection question in the target standard image.

[0105] In this way, each standard image can be obtained from the question bank.

[0106] S101 - 12 : Determine the similarity between the image to be corrected and each standard image in the standard image set.

[0107] In one implementation, a convolutional neural network is used to extract features from the image to be corrected, obtain first image features, and determine the similarity between the first image features and the second image features corresponding to each standard image.

[0108] Optionally, the second image features corresponding to each standard image may be extracted in advance to simplify the operation and improve the efficiency of similarity determination.

[0109] Optionally, when determining the similarity between images, the Euclidean distance or cosine similarity may be used, or other methods may be used for determination, which are not limited here.

[0110] S101 - 13 : Filtering the target standard image from the standard image set according to the similarity of each standard image.

[0111] In one implementation, if the target similarity that meets the similarity screening condition is screened out, the target standard image corresponding to the target similarity can be obtained, and the search is determined to be successful, and the standard answer can be obtained.

[0112] For example, the similarity screening condition may be higher than a set similarity threshold. Further, if it is determined that there is no target similarity that meets the similarity screening condition, it is determined that the search has failed, for example, an error code may be returned.

[0113] In actual applications, the similarity screening conditions and the similarity thresholds can be set according to the actual application scenarios and are not limited here.

[0114] See also Figure 2 The figure shows an example of a target standard image. Figure 2 In, multiple first target objects are included, namely Figure 2 The house pattern and the chicken pattern in the figure. Each house pattern contains a different operation expression, and each chicken pattern contains a different calculation result.

[0115] In this way, the standard image corresponding to any image to be corrected can be automatically searched from the question bank.

[0116] In one implementation, when determining the target area of ​​the second target object, the following steps may be used:

[0117] S101-21: Detect multiple key point pairs matching the target standard image and the image to be corrected according to the key point detection model, each key point pair comprising a first key point in the target standard image and a second key point corresponding to the first key point in the image to be corrected.

[0118] The key point detection model is used to match the key points of the two images to obtain multiple pairs of matched key points. Each key point pair is the corresponding relationship between each first key point and each second key point.

[0119] Optionally, the number of key point pairs may be preset, such as 2048 pairs.

[0120] For example, see Figure 3 The following is an example diagram of key points. Figure 3 In the figure, there is an image to be corrected on the left and a target standard image on the right. Figure 3 An example of a set of key point pairs in is given, the target standard image contains a first key point, and the image to be corrected contains its corresponding second key point. For the convenience of explanation, the first key point and the second key point are connected by a dotted line to indicate that the first key point and the second key point are a matched set of key point pairs, that is, the two have a corresponding relationship.

[0121] S101-22: For each first target object, perform the following steps:

[0122] S101-22-1: Based on the positions of the first key points and the image region where the first target object is located, a plurality of target key points located in the image region of the first target object are screened out from the first key points.

[0123] In one implementation, a first coordinate system is established for the target standard image, so that the key point coordinates of each first key point (i.e., the position of the first key point) can be obtained, and the regional coordinate range of the image area where the first target object is located can be obtained, and then the target key points corresponding to the key point coordinates within the regional coordinate range can be screened out.

[0124] For example, Figure 3 The first key point in is located in the house pattern containing "8+5" in the target standard image (ie, the first target object).

[0125] In this way, the target key point set contained in each first target object can be obtained.

[0126] S101-22-2: According to each key pair, determine the second key point corresponding to each target key point.

[0127] In one implementation, the second key point corresponding to each target key point is obtained according to the correspondence between the first key point and the second key point.

[0128] In this way, the corresponding positions of the first key points of the first target object in the image to be corrected can be obtained.

[0129] S101-22-3: Connect the second key points corresponding to each target key point to obtain the target area of ​​the second target object corresponding to the first target object.

[0130] In one implementation, the minimum bounding rectangle of the second key point corresponding to each target key point is determined (a rectangular box containing "8+5" in the target standard image), and the area of ​​the minimum bounding rectangle is determined as the target area of ​​the second target object corresponding to the first target object.

[0131] For example, the target key points contained in the first target object are p1, p2 and p3 respectively, and their corresponding second key points are q1, q2 and q3 respectively. Then, the minimum enclosing rectangle connecting q1, q2 and q3 can be obtained, and the area of ​​the minimum enclosing rectangle can be determined as the target area of ​​the second target object.

[0132] In this way, the object area (i.e., the target area) found is relatively stable, avoiding the instability problem of the result (i.e., the target area) predicted by the neural network model in the traditional way, and reducing the problem of multiple frames or fewer frames.

[0133] Step 102: Detect the endpoints of at least one line in the image to be corrected by using an endpoint detection model to obtain the endpoint positions corresponding to each of the at least one line; the endpoint detection model is used to detect the endpoint positions of two endpoints of a line.

[0134] In one implementation, a second coordinate system is established for the target standard image, and the endpoint coordinates (i.e., endpoint positions) of each endpoint can be obtained, that is, the image to be corrected can be input into an endpoint detection model to obtain the two endpoint positions corresponding to each connecting line in the image to be corrected.

[0135] Optionally, the endpoint detection model can be modified based on the target detection model (Detection Transformer, DETR).

[0136] In other words, the endpoint detection model is a modified version of the target detection model based on the Transformers architecture. The traditional target detection model usually predicts a rectangular box of an object, and the predicted values ​​are usually the coordinates (x, y) of the center point of the rectangular box, the width w and the height h. In the embodiment of the present application, the traditional target detection model is adjusted so that the predicted values ​​are the coordinates of the two endpoints of the connecting line.

[0137] Furthermore, the first endpoint (e.g., the upper endpoint of the connection line) and the second endpoint (e.g., the lower endpoint of the connection line) of the two endpoints can also be distinguished according to the relative positional relationship between the two endpoints. For example, the first endpoint and the second endpoint can be distinguished according to the height. Assuming that the first endpoint is the upper endpoint of the connection line and the second endpoint is the lower endpoint of the connection line, the coordinates of the first endpoint can be expressed as (x1, y1), and the coordinates of the second endpoint can be expressed as (x2, y2), then y1>y2, therefore, the first endpoint and the second endpoint can be distinguished according to the size of the y of the coordinates of the two endpoints.

[0138] Furthermore, the endpoint detection model may also be trained. In one embodiment, the endpoint detection model is trained by the following steps:

[0139] Acquire training data; the training data includes a plurality of line samples and their corresponding endpoint sample positions; input each line sample in the training data into an initial detection model to obtain a predicted endpoint position corresponding to each line sample; obtain a prediction loss based on the predicted endpoint position and endpoint sample position corresponding to each line sample; obtain a first connection vector corresponding to each line sample based on the predicted endpoint position corresponding to each line sample; obtain a second connection vector corresponding to each line sample based on the endpoint sample position corresponding to each line sample; obtain a vector loss based on the first connection vector and the second connection vector corresponding to each line sample; adjust parameters of the initial detection model based on the prediction loss and the vector loss until the endpoint detection model that meets the set training conditions is obtained.

[0140] In one implementation, the following steps may be used to determine the predicted loss:

[0141] Determine the absolute value of the difference between the predicted endpoint position and the endpoint sample position to obtain the prediction loss, which can be called Loss_L1.

[0142] In one implementation, when determining the vector loss, any of the following methods may be used:

[0143] Method 1: Determine the vector loss according to the similarity between the first connection vector and the second connection vector corresponding to each connection sample.

[0144] For example, the first connection vector may be expressed as a vector (x2-x1, y2-y1), and the second connection vector may be expressed as a vector (x4-x3, y4-y3).

[0145] In one implementation, the vector loss function is a negative cosine similarity function, and the negative cosine similarity function can be used to determine the negative cosine similarity between the first connection vector and the second connection vector as the vector loss.

[0146] Among them, the value of negative cosine similarity is [-1, 1]. When the value is closer to -1, it indicates that the angle between the two vectors is smaller and the directions are more consistent, that is, the vector loss is smaller.

[0147] Method 2: determining the vector loss according to the difference between the first modulus length of the first connection vector and the second modulus length of the second connection vector corresponding to each connection sample.

[0148] Method three: Determine the vector loss according to the similarity between the first connection vector and the second connection vector corresponding to each connection sample, and the difference between the first modulus length of the first connection vector and the second modulus length of the second connection vector corresponding to each connection sample.

[0149] Among them, the above similarity can be called Loss_sim, the above difference can be called Loss_len, then vector loss = Loss_sim + Loss_len.

[0150] In one implementation, when adjusting the parameters of the initial detection model according to the prediction loss and the vector loss, the following steps may be adopted:

[0151] The prediction loss and the sum of the vector loss are determined through the model loss function to obtain the loss function value, and the parameters of the initial detection model are adjusted according to the loss function value.

[0152] Among them, the model loss function can be: Loss_L1+Loss_sim+Loss_len.

[0153] It should be noted that in the embodiments of the present application, only the sequential execution of step 101 and step 102 is used as an example for illustration. In actual applications, since step 101 and step 102 are independent of each other, the two can be executed in any order, for example, they can also be executed in parallel, and there is no limitation here.

[0154] Step 103: According to the positions of the endpoints and the target areas of the second target objects, the target connection objects that match the endpoints are screened out from the second target objects.

[0155] In one implementation, for each endpoint, the following steps are performed:

[0156] According to the endpoint position of the endpoint and the target area of ​​each second target object, the connection distance between the endpoint and each second target object is determined; the minimum distance among the connection distances is determined; and the second target object corresponding to the minimum distance is determined as the target connection object matched by the endpoint.

[0157] For example, the distances between the endpoint coordinates of the endpoint and the center point coordinates of the target area of ​​each second target object are determined to obtain a plurality of connection distances.

[0158] In this way, the objects to which the endpoints of each connection are respectively connected (ie, the matching target connection objects) can be obtained.

[0159] Step 104: According to the target connection objects of each endpoint, a target connection object combination of each of the at least one connection line is formed.

[0160] In one implementation, for each connection line, the following steps are performed: target connection objects to which two endpoints of the connection line are respectively connected to form a target connection object combination.

[0161] For example, Figure 3 Each house pattern (i.e., part of the second target object) in the image to be corrected can be represented as F1, F2, ..., F6 in sequence. Figure 3 Each bird pattern (ie, part of the second target object) in the image to be corrected can be represented as B1, B2, ..., B6 in sequence, such as: Figure 3 The target connection object combination of a line in is (F1, B4).

[0162] Furthermore, the following steps may be performed for each connection line:

[0163] According to the endpoint position of one endpoint of the connection line and the target area of ​​each second target object, the connection distance between the endpoint and each second target object is determined; the minimum distance among the connection distances is determined; the second target object corresponding to the minimum distance is determined as the target connection object matched by the endpoint, and similarly, the target connection object according to the other endpoint of the connection line is determined; then, the target connection objects respectively connected to the two endpoints of the connection line form a target connection object combination.

[0164] In this way, each pair of target connection objects of the user connection can be obtained.

[0165] Step 105: Compare the target connection object combination of the at least one connection line with at least one standard object combination corresponding to the target standard image to obtain the connection question correction result of the image to be corrected.

[0166] In one implementation, for each connection, the following steps are performed:

[0167] If it is determined that there is a standard object combination that matches the target connection object combination of the connection in the at least one standard object combination, then the correction result of the connection is determined to be a correct connection; otherwise, the correction result of the connection is determined to be an incorrect connection; and the correction result of the connection question is obtained according to the correction results of each connection.

[0168] For example, the score of the connecting-the-line question can be obtained based on the number of correctly connected lines and the score of each connected line.

[0169] According to the correction results of each connection, when obtaining the correction results of the connection question, the following steps can be adopted:

[0170] Determine the difference between the number of the target connection object combinations and the number of the standard object combinations; and obtain the connection question correction result according to the correction result of each of the at least one connection and the difference.

[0171] For example, if the above difference is zero, that is, the number of target connection object combinations is the same as the number of standard object combinations, that is, there are no extra or fewer connections, and all correction results are correct connections, then it is determined that the answer to the connection question is correct, otherwise, the answer is incorrect.

[0172] Combine the following Figure 4 , further describing the embodiments of the present application, see Figure 4 As shown in FIG. 1 , a detailed flow chart of a method for correcting a connecting question is shown. The flow of the method includes:

[0173] Step 401: Obtain the image to be corrected input by the user.

[0174] Step 402: Search the question bank for a target standard image that matches the image to be corrected.

[0175] Step 403: Determine whether the search is successful. If so, execute steps 404 and 405 in parallel. Otherwise, execute step 408.

[0176] Step 404 : Obtain the target area of ​​the second target object in the image to be corrected by matching key points between images, and execute step 406 .

[0177] Step 405: Detect the endpoint positions of each connecting line from the image to be corrected by endpoint detection.

[0178] Step 406: Determine a target connection object combination connected to each link according to each target area and the end point position of each link.

[0179] Step 407: Compare each target connection object combination with the standard object combination to obtain and output the connection question correction result.

[0180] Step 408: Output search error results.

[0181] When executing step 401 to step 408, the above steps 101 to 105 may be referred to, and will not be described in detail here.

[0182] In the embodiment of the present application, there is no need to directly perform frame detection on the object. Instead, the target area of ​​the second target object is determined by using key point pairs and minimum rectangle connection. It is only necessary to detect the two endpoints of the connecting line, and determine the two target connection objects respectively connected by the connecting line based on the distance between the endpoints and the target area. In addition, comparison is performed based on the combination of target connection objects instead of comparing the connecting lines. This can solve the problem of difficulty in grading connecting line questions, is applicable to a variety of scenarios, has a wide range of applications, and can achieve high-precision grading of connecting line questions.

[0183] Based on the same inventive concept, the embodiment of the present application also provides a device for correcting a connecting question. Since the principle of solving the problem by the above device and equipment is similar to that of a method for correcting a connecting question, the implementation of the above device can refer to the implementation of the method, and the repeated parts will not be repeated. The device can be applied to electronic devices. The present application does not limit the type of electronic devices. It can be any type of device suitable for implementation, such as terminal devices and servers, etc., and the present application will not repeat them.

[0184] See also Figure 5 , which is a structural block diagram of a device for correcting a connect-the-dot question in an embodiment of the present application. In some implementations, the device for correcting a connect-the-dot question in an example of the present application includes:

[0185] A determination unit 501 is used to determine, for each first target object in the target standard image, a target area of ​​a second target object to be matched in the image to be corrected; the first target object is an object to be connected;

[0186] The detection unit 502 is used to detect the endpoints of at least one line in the image to be corrected by using an endpoint detection model to obtain the endpoint positions corresponding to each of the at least one line; the endpoint detection model is used to detect the endpoint positions of two endpoints of the line;

[0187] The matching unit 503 is used to filter out target connection objects that match each endpoint from each second target object according to the position of each endpoint and the target area of ​​each second target object;

[0188] A composition unit 504 is used to compose a target connection object combination of the at least one connection according to the target connection objects of the respective endpoints;

[0189] The obtaining unit 505 is used to compare the target connection object combination of the at least one connection line with at least one standard object combination corresponding to the target standard image to obtain the connection question correction result of the image to be corrected.

[0190] In one implementation, the determining unit 501 is further configured to:

[0191] Acquire a standard image set; the standard image set includes at least one standard image;

[0192] Determine the similarity between the image to be corrected and each standard image in the standard image set;

[0193] The target standard image is selected from the standard image set according to the similarity of each standard image.

[0194] In one implementation, the determining unit 501 is configured to:

[0195] According to the key point detection model, a plurality of key point pairs matching the target standard image and the image to be corrected are detected, each key point pair comprising a first key point in the target standard image and a second key point corresponding to the first key point in the image to be corrected;

[0196] For each first target object, perform the following steps:

[0197] According to the positions of the first key points and the image region where the first target object is located, a plurality of target key points located in the image region of the first target object are screened out from the first key points;

[0198] According to each key pair, determine the second key point corresponding to each target key point;

[0199] The second key points corresponding to each target key point are connected to obtain a target area of ​​the second target object corresponding to the first target object.

[0200] In one implementation, the matching unit 503 is used to:

[0201] For each endpoint separately, perform the following steps:

[0202] Determine, according to the endpoint position of the endpoint and the target area of ​​each second target object, the connection distance between the endpoint and each second target object;

[0203] Determine the minimum distance among the connection distances;

[0204] The second target object corresponding to the minimum distance is determined as the target connection object for endpoint matching.

[0205] In one implementation, the obtaining unit 505 is used to:

[0206] For each connection, the following steps are performed respectively: if it is determined that there is a standard object combination matching the target connection object combination of the connection in the at least one standard object combination, then the correction result of the connection is determined to be a correct connection; otherwise, the correction result of the connection is determined to be an incorrect connection;

[0207] According to the correction results of each connection, the correction result of the connection question is obtained.

[0208] In one implementation, the detection unit 502 is further configured to:

[0209] The endpoint detection model is trained using the following steps:

[0210] Acquire training data; the training data includes a plurality of connection samples and their corresponding endpoint sample positions;

[0211] Input each link sample in the training data into the initial detection model to obtain the predicted endpoint position corresponding to each link sample;

[0212] Obtain the prediction loss according to the predicted endpoint positions and endpoint sample positions corresponding to each connection sample;

[0213] According to the predicted endpoint positions corresponding to the link samples, the first link vectors corresponding to the link samples are obtained;

[0214] According to the endpoint sample positions corresponding to the line samples, the second connection vectors corresponding to the line samples are obtained;

[0215] Obtaining a vector loss according to the first connection vector and the second connection vector corresponding to each connection sample;

[0216] According to the prediction loss and the vector loss, the parameters of the initial detection model are adjusted until the endpoint detection model that meets the set training conditions is obtained.

[0217] In one implementation, the detection unit 502 is further configured to:

[0218] Use any of the following:

[0219] Determine the vector loss according to the similarity between the first connection vector and the second connection vector corresponding to each connection sample;

[0220] Determine the vector loss according to a difference between a first modulus length of a first connection vector and a second modulus length of a second connection vector corresponding to each connection sample;

[0221] The vector loss is determined according to the similarity between the first connection vector and the second connection vector corresponding to each connection sample, and the difference between the first modulus length of the first connection vector and the second modulus length of the second connection vector corresponding to each connection sample.

[0222] The method for correcting the connection question in the embodiment of the present application includes determining each first target object in the target standard image and the target area of ​​the second target object matched in the image to be corrected; the first target object is an object for being connected; through the endpoint detection model, the endpoint of at least one connection line in the image to be corrected is detected to obtain the endpoint position corresponding to each of the at least one connection line; the endpoint detection model is used to detect the endpoint positions of the two endpoints of the connection line; according to the positions of each endpoint and the target area of ​​each second target object, the target connection object that matches each endpoint is screened out from each second target object; according to the target connection object of each endpoint, a target connection object combination of each connection line is formed; the target connection object combination of each connection line is compared with at least one standard object combination corresponding to the target standard image to obtain the connection question correction result of the image to be corrected. In this way, through the method of endpoint detection and target object matching, the connection question can be corrected effectively and accurately, solving the problem of difficulty in effectively correcting the connection question.

[0223] In an embodiment of the present application, an electronic device is provided, including:

[0224] Processor; and

[0225] The memory stores computer instructions, where the computer instructions are used to enable the processor to execute the method of any of the above embodiments.

[0226] In an embodiment of the present application, a computer-readable storage medium is provided, which stores computer instructions, and the computer instructions are used to enable a computer to execute a method in any of the above-mentioned embodiments.

[0227] An embodiment of the present application also provides a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device implements any of the above-mentioned methods.

[0228] Figure 6 FIG. 6 is a schematic diagram showing the structure of an electronic device 6000. Figure 6 As shown, the electronic device 6000 includes: a processor 6010 and a memory 6020, and optionally, may also include a power supply 6030, a display unit 6040, and an input unit 6050.

[0229] The processor 6010 is the control center of the electronic device 6000. It connects various components using various interfaces and lines, and performs various functions of the electronic device 6000 by running or executing software programs and / or data stored in the memory 6020, thereby monitoring the electronic device 6000 as a whole.

[0230] In the embodiment of the present application, the processor 6010 executes the various steps in the above embodiment when calling the computer program stored in the memory 6020.

[0231] Optionally, the processor 6010 may include one or more processing units; preferably, the processor 6010 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and applications, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 6010. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on separate chips.

[0232] The memory 6020 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, various applications, etc.; the data storage area may store data created according to the use of the electronic device 6000, etc. In addition, the memory 6020 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices, etc.

[0233] The electronic device 6000 also includes a power source 6030 (such as a battery) for supplying power to various components. The power source can be logically connected to the processor 6010 through a power management system, thereby managing functions such as charging, discharging, and power consumption through the power management system.

[0234] The display unit 6040 may be used to display information input by a user or information provided to a user, various menus of the electronic device 6000, and in the embodiment of the present application, is mainly used to display the display interface of each application in the electronic device 6000 and objects such as text and pictures displayed in the display interface. The display unit 6040 may include a display panel 6041. The display panel 6041 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), and the like.

[0235] The input unit 6050 may be used to receive information such as numbers or characters input by the user. The input unit 6050 may include a touch panel 6051 and other input devices 6052. The touch panel 6051, also known as a touch screen, may collect touch operations of the user on or near it (such as operations of the user using a finger, a stylus, or any other suitable object or accessory on or near the touch panel 6051).

[0236] Specifically, the touch panel 6051 can detect the user's touch operation, detect the signal brought by the touch operation, convert these signals into touch point coordinates, send them to the processor 6010, and receive and execute the command sent by the processor 6010. In addition, the touch panel 6051 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. Other input devices 6052 can include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, power on / off keys, etc.), a trackball, a mouse, a joystick, etc.

[0237] Of course, the touch panel 6051 can cover the display panel 6041. When the touch panel 6051 detects a touch operation on or near it, it is transmitted to the processor 6010 to determine the type of touch event. Then the processor 6010 provides corresponding visual output on the display panel 6041 according to the type of touch event. Figure 6 In the embodiment, the touch panel 6051 and the display panel 6041 are used as two independent components to realize the input and output functions of the electronic device 6000, but in some embodiments, the touch panel 6051 and the display panel 6041 can be integrated to realize the input and output functions of the electronic device 6000.

[0238] The electronic device 6000 may also include one or more sensors, such as a pressure sensor, a gravity acceleration sensor, a proximity light sensor, etc. Of course, according to the needs of specific applications, the electronic device 6000 may also include other components such as a camera. Since these components are not the key components used in the embodiments of the present application, Figure 6 It is not shown and will not be described in detail.

[0239] Those skilled in the art will understand that Figure 6 The electronic device is merely an example and does not limit the electronic device, and may include more or less components than those shown in the figure, or may combine certain components, or may include different components.

[0240] For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Of course, when implementing this application, the functions of each module (or unit) can be implemented in the same or multiple software or hardware.

Claims

1. A method for correcting connecting questions, characterized in that: The method comprises: Determine each first target object in the target standard image and a target area of ​​a second target object matched in the image to be corrected; the first target object is an object to be connected; By using an endpoint detection model, the endpoints of at least one connecting line in the image to be corrected are detected to obtain the endpoint positions corresponding to each of the at least one connecting line; the endpoint detection model is used to detect the endpoint positions of the two endpoints of the connecting line; According to the positions of the endpoints and the target areas of the second target objects, the target connection objects that match the endpoints are selected from the second target objects; According to the target connection objects of each endpoint, a target connection object combination of each of the at least one connection line is formed; The target connection object combination of the at least one connection line is compared with at least one standard object combination corresponding to the target standard image to obtain a connection question correction result of the image to be corrected.

2. The method according to claim 1, characterized in that Before determining the target area of ​​each first target object in the target standard image and the second target object matched in the image to be corrected, the method further includes: Acquire a standard image set; the standard image set includes at least one standard image; Determine the similarity between the image to be corrected and each standard image in the standard image set; The target standard image is selected from the standard image set according to the similarity of each standard image.

3. The method according to claim 1 or 2, characterized in that: The step of determining the target area of ​​the second target object matched in the image to be corrected for each first target object in the target standard image includes: According to the key point detection model, a plurality of key point pairs matching the target standard image and the image to be corrected are detected, each key point pair comprising a first key point in the target standard image and a second key point corresponding to the first key point in the image to be corrected; For each first target object, perform the following steps: According to the positions of the first key points and the image region where the first target object is located, a plurality of target key points located in the image region of the first target object are screened out from the first key points; According to each key pair, determine the second key point corresponding to each target key point; The second key points corresponding to each target key point are connected to obtain a target area of ​​the second target object corresponding to the first target object.

4. The method according to claim 1 or 2, characterized in that: The step of screening out target connection objects that match each endpoint from each second target object according to the position of each endpoint and the target area of ​​each second target object includes: For each endpoint separately, perform the following steps: Determine, according to the endpoint position of the endpoint and the target area of ​​each second target object, the connection distance between the endpoint and each second target object; Determine the minimum distance among the connection distances; The second target object corresponding to the minimum distance is determined as the target connection object for endpoint matching.

5. The method according to claim 1 or 2, characterized in that: The step of comparing the target connection object combination of the at least one connection line with the at least one standard object combination corresponding to the target standard image to obtain the connection question correction result of the image to be corrected includes: For each connection, the following steps are performed respectively: if it is determined that there is a standard object combination matching the target connection object combination of the connection in the at least one standard object combination, then the correction result of the connection is determined to be a correct connection; otherwise, the correction result of the connection is determined to be an incorrect connection; According to the correction results of each connection, the correction result of the connection question is obtained.

6. The method according to claim 1 or 2, characterized in that: The endpoint detection model is trained using the following steps: Acquire training data; the training data includes a plurality of connection samples and their corresponding endpoint sample positions; Input each link sample in the training data into the initial detection model to obtain the predicted endpoint position corresponding to each link sample; Obtain the prediction loss according to the predicted endpoint positions and endpoint sample positions corresponding to each connection sample; According to the predicted endpoint positions corresponding to the link samples, the first link vectors corresponding to the link samples are obtained; According to the endpoint sample positions corresponding to the line samples, the second connection vectors corresponding to the line samples are obtained; Obtaining a vector loss according to the first connection vector and the second connection vector corresponding to each connection sample; According to the prediction loss and the vector loss, the parameters of the initial detection model are adjusted until the endpoint detection model that meets the set training conditions is obtained.

7. The method according to claim 6, characterized in that The step of obtaining the vector loss according to the first connection vector and the second connection vector corresponding to each connection sample includes: Use any of the following: Determine the vector loss according to the similarity between the first connection vector and the second connection vector corresponding to each connection sample; Determine the vector loss according to a difference between a first modulus length of a first connection vector and a second modulus length of a second connection vector corresponding to each connection sample; The vector loss is determined according to the similarity between the first connection vector and the second connection vector corresponding to each connection sample, and the difference between the first modulus length of the first connection vector and the second modulus length of the second connection vector corresponding to each connection sample.

8. A device for correcting connecting questions, characterized in that: The device comprises: A determination unit, used to determine, for each first target object in the target standard image, a target area of ​​a second target object matched in the image to be corrected; the first target object is an object to be connected; A detection unit, used to detect the endpoints of at least one line in the image to be corrected by using an endpoint detection model, and obtain the endpoint positions corresponding to each of the at least one line; the endpoint detection model is used to detect the endpoint positions of two endpoints of the line; A matching unit, configured to filter out target connection objects that match each endpoint from each second target object according to the position of each endpoint and the target area of ​​each second target object; A composition unit, configured to compose a target connection object combination of the at least one connection according to the target connection objects of the respective endpoints; The obtaining unit is used to compare the target connection object combination of the at least one connection line with at least one standard object combination corresponding to the target standard image to obtain the connection question correction result of the image to be corrected.

9. An electronic device, characterized in that: include: processor; as well as A memory storing computer instructions, wherein the computer instructions are used to enable the processor to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Computer instructions are stored, and the computer instructions are used to make a computer execute the method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The invention comprises a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code is executed in a processor of an electronic device, the processor in the electronic device performs the method according to any one of claims 1 to 7.

Images