Topic alignment method and device, computer equipment and storage medium

Abstract

The invention provides a question alignment method and device, computer equipment and a storage medium. The method comprises the steps of determining at least two candidate answer pages corresponding to a to-be-recognized image from an answer page library according to initial similarity between the to-be-recognized image and each answer page in the answer page library; determining a target answer page from at least two candidate answer pages according to the target similarity between the to-be-recognized image and each candidate answer page; and for each to-be-identified question in the to-be-identified image, determining a target question corresponding to the to-be-identified question from the target answer page. According to the method, the accuracy of the searched target topic can be ensured while the searching efficiency is improved, so that the efficiency and the accuracy of correcting the to-be-recognized image are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a topic alignment method, apparatus, computer equipment and storage medium. Background Art

[0002] With the continuous advancement of educational informatization, especially with the rapid development of artificial intelligence and deep learning technologies, homework grading systems have gradually become an important part of intelligent education. These systems can efficiently handle a large number of student assignments, automatically grade and provide feedback. Question alignment, as a core link in the homework grading system, is directly related to the accuracy of the grading results. Its main task is to accurately match each question in the student's test paper with the corresponding question in the question bank, so as to extract the correct answer and analysis to grade the questions in the test paper.

[0003] Traditional question alignment methods are mainly divided into single question alignment and full page alignment. Single question alignment identifies the text information of a single question and then compares it with the questions in the question bank to determine the corresponding answer question. This method has high accuracy, but slow processing speed, especially when there are a large number of questions, there is a problem of long time consumption; full page alignment is based on the full page image of the test paper for matching, which can process a large number of questions at one time. This method has high processing efficiency, but is usually greatly affected by layout changes, especially when facing different versions of test papers, the alignment effect of the questions in the test paper is not good and the accuracy is low. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a topic alignment method, apparatus, computer equipment and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, the present application provides a topic alignment method, the method comprising:

[0006] Determine at least two candidate answer pages corresponding to the image to be identified from the answer page library according to the initial similarity between the image to be identified and each answer page in the answer page library, wherein the answer page library includes a plurality of answer pages, and the initial similarity includes the page similarity between the image to be identified and each answer page in the answer page library;

[0007] Determining a target answer page from at least two candidate answer pages according to a target similarity between the image to be identified and each candidate answer page, wherein the target similarity includes a question similarity between the image to be identified and each candidate answer page;

[0008] For each question to be identified in the image to be identified, a target question corresponding to the question to be identified is determined from the target answer page.

[0009] In combination with any embodiment of the present disclosure, the determining, from the answer page library, at least two candidate answer pages corresponding to the image to be identified includes:

[0010] Determine the number of text content lines and the number of text content lines in the image to be recognized, wherein the text content includes text content and non-text content;

[0011] Determining the initial similarity between the image to be identified and each answer page in the answer page library according to the number of text content lines and the number of text content lines;

[0012] If the initial similarity between the image to be identified and any answer page in the answer page library is greater than a first similarity threshold, any answer page is used as a candidate answer page.

[0013] In combination with any embodiment of the present disclosure, determining the initial similarity between the image to be identified and each answer page in the answer page library according to the number of lines of text content and the number of lines of word content includes:

[0014] When the number of lines of the text content is greater than or equal to the first line number threshold, and the number of lines of the text content is greater than or equal to the second line number threshold, determining the page similarity between the image to be identified and each answer page in the answer library, and using the page similarity as the initial similarity;

[0015] When the number of lines of text content is less than the first line number threshold, or the number of lines of text content is less than the second line number threshold, determine the page similarity between the image to be identified and each answer page in the answer page library, and the line text similarity between the image to be identified and each answer page in the answer page library, and determine the initial similarity between the image to be identified and each answer page in the answer page library based on the page similarity and the line text similarity.

[0016] In combination with any embodiment of the present disclosure, determining a target answer page from at least two candidate answer pages according to the target similarity between the image to be identified and each candidate answer page includes:

[0017] For each candidate answer page, determining the line text similarity and question similarity between the image to be identified and the candidate answer page;

[0018] Determine a target similarity between the image to be identified and each candidate answer page according to the line text similarity and the question similarity between the image to be identified and each candidate answer page;

[0019] The candidate answer page with the largest target similarity is used as the target answer page.

[0020] In combination with any embodiment of the present disclosure, the determining of the line text similarity between the image to be recognized and the candidate answer page includes:

[0021] For each line of text in the image to be recognized, determine the similarity between the line of text and each line of text in the candidate answer page, and use the maximum similarity among the obtained similarities as the similarity of the line of text;

[0022] The sum of the similarities of each line of text in the image to be recognized is taken as the line of text similarity between the image to be recognized and the candidate answer page.

[0023] In combination with any embodiment of the present disclosure, the determining the question similarity between the image to be identified and each candidate answer page includes:

[0024] For each question to be identified in the image to be identified, determine the similarity between the question to be identified and each question in the candidate answer page, and use the maximum similarity among the obtained similarities as the similarity of the question to be identified;

[0025] The sum of the similarities of each question to be identified in the image to be identified is taken as the question similarity between the image to be identified and the candidate answer page.

[0026] In combination with any embodiment of the present disclosure, determining the target similarity between the image to be identified and each candidate answer page according to the line text similarity and the question similarity between the image to be identified and each candidate answer page includes:

[0027] The sum of the line text similarity and the question similarity between the image to be identified and each candidate answer page is used as the target similarity between the image to be identified and each candidate answer page.

[0028] In combination with any embodiment of the present disclosure, for each question to be identified in the image to be identified, determining a target question corresponding to the question to be identified from the target answer page includes:

[0029] For each question to be identified in the image to be identified, determining the line text similarity and question similarity between the question to be identified and each candidate question in the target answer page;

[0030] For each question to be identified in the image to be identified, the candidate question with the largest sum of the line text similarity and the question similarity in the target answer page is used as the target question corresponding to the question to be identified.

[0031] In combination with any embodiment of the present disclosure, the step of taking the candidate question with the largest sum of the line text similarity and the question similarity in the target answer page as the target question corresponding to the question to be identified includes:

[0032] In the case that the question to be identified does not contain non-text content, the candidate question having the largest sum of the similarity between the line text and the question in the target answer page is used as the target question corresponding to the question to be identified;

[0033] In the case where the question to be identified contains non-text content, the feature vector similarity between the non-text content and the non-text content of each candidate question in the target answer page is determined, and based on the line text similarity, the question similarity and the feature vector similarity in the target answer page, the target question corresponding to the question to be identified is determined from each candidate question on the target answer page.

[0034] In combination with any embodiment of the present disclosure, the method further includes:

[0035] In the case that there is a question to be identified for which no corresponding target question is found in the image to be identified, determining at least two candidate questions corresponding to the question to be identified from a question library, wherein the question library includes a plurality of questions and an answer corresponding to each question;

[0036] According to the similarity between the to-be-identified topic and each candidate topic in the topic library, a target topic corresponding to the to-be-identified topic is determined from at least two candidate topics.

[0037] In a second aspect, the present application also provides a title alignment device, comprising:

[0038] a first determination module, configured to determine at least two candidate answer pages corresponding to the image to be identified from the answer page library according to an initial similarity between the image to be identified and each answer page in the answer page library, wherein the answer page library includes a plurality of answer pages, and the initial similarity includes a page similarity between the image to be identified and each answer page in the answer page library;

[0039] a second determination module, configured to determine a target answer page from at least two candidate answer pages according to a target similarity between the image to be identified and each candidate answer page, wherein the target similarity includes a question similarity between the image to be identified and each candidate answer page;

[0040] The third determination module is used to determine, for each question to be identified in the image to be identified, a target question corresponding to the question to be identified from the target answer page.

[0041] In one embodiment, the first determining module is specifically configured to:

[0042] Determine the number of text content lines and the number of text content lines in the image to be recognized, wherein the text content includes text content and non-text content;

[0043] Determining the initial similarity between the image to be identified and each answer page in the answer page library according to the number of text content lines and the number of text content lines;

[0044] If the initial similarity between the image to be identified and any answer page in the answer page library is greater than a first similarity threshold, any answer page is used as a candidate answer page.

[0045] In one embodiment, the first determining module is specifically configured to:

[0046] When the number of lines of the text content is greater than or equal to the first line number threshold, and the number of lines of the text content is greater than or equal to the second line number threshold, determining the page similarity between the image to be identified and each answer page in the answer library, and using the page similarity as the initial similarity;

[0047] When the number of lines of text content is less than the first line number threshold, or the number of lines of text content is less than the second line number threshold, determine the page similarity between the image to be identified and each answer page in the answer page library, and the line text similarity between the image to be identified and each answer page in the answer page library, and determine the initial similarity between the image to be identified and each answer page in the answer page library based on the page similarity and the line text similarity.

[0048] In one embodiment, the second determining module is specifically configured to:

[0049] For each candidate answer page, determining the line text similarity and question similarity between the image to be identified and the candidate answer page;

[0050] Determine a target similarity between the image to be identified and each candidate answer page according to the line text similarity and the question similarity between the image to be identified and each candidate answer page;

[0051] The candidate answer page with the largest target similarity is used as the target answer page.

[0052] In one embodiment, the second determining module is specifically configured to:

[0053] For each line of text in the image to be recognized, determine the similarity between the line of text and each line of text in the candidate answer page, and use the maximum similarity among the obtained similarities as the similarity of the line of text;

[0054] The sum of the similarities of each line of text in the image to be recognized is taken as the line of text similarity between the image to be recognized and the candidate answer page.

[0055] In one embodiment, the second determining module is specifically configured to:

[0056] For each question to be identified in the image to be identified, determine the similarity between the question to be identified and each question in the candidate answer page, and use the maximum similarity among the obtained similarities as the similarity of the question to be identified;

[0057] The sum of the similarities of each question to be identified in the image to be identified is taken as the question similarity between the image to be identified and the candidate answer page.

[0058] In one embodiment, the second determining module is specifically configured to:

[0059] The sum of the line text similarity and the question similarity between the image to be identified and each candidate answer page is used as the target similarity between the image to be identified and each candidate answer page.

[0060] In one embodiment, the third determination module is specifically used to:

[0061] For each question to be identified in the image to be identified, determining the line text similarity and question similarity between the question to be identified and each candidate question in the target answer page;

[0062] For each question to be identified in the image to be identified, the candidate question with the largest sum of the line text similarity and the question similarity in the target answer page is used as the target question corresponding to the question to be identified.

[0063] In one embodiment, the third determination module is specifically used to:

[0064] In the case that the question to be identified does not contain non-text content, the candidate question having the largest sum of the similarity between the line text and the question in the target answer page is used as the target question corresponding to the question to be identified;

[0065] In the case where the question to be identified contains non-text content, the feature vector similarity between the non-text content and the non-text content of each candidate question in the target answer page is determined, and based on the line text similarity, the question similarity and the feature vector similarity in the target answer page, the target question corresponding to the question to be identified is determined from each candidate question on the target answer page.

[0066] In one embodiment, the title alignment device further comprises:

[0067] A topic determination module is used to determine at least two candidate topics corresponding to the topic to be identified from a topic library when there is a topic to be identified in the image to be identified for which no corresponding target topic is found, wherein the topic library includes multiple topics and answers corresponding to each topic; and determine the target topic corresponding to the topic to be identified from the at least two candidate topics based on the similarity between the topic to be identified and each candidate topic in the topic library.

[0068] In a third aspect, the present application provides a computer program product, comprising a computer program / instructions, which implement the steps of the method described in any embodiment when the computer program / instructions are executed by a processor.

[0069] In a fourth aspect, the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any of the above embodiments when executing the program.

[0070] In a fifth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the above embodiments.

[0071] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0072] In the disclosed embodiment, at least two candidate answer pages corresponding to the image to be identified are determined from the answer page library, and the target answer page corresponding to the image to be identified is determined from the at least two candidate answer pages, and further, the target question corresponding to each question to be identified in the identified image is determined from the target answer page. Compared with the prior art which only searches for the target answer page for each overall answer page, or only searches for the corresponding target question for each question to be identified, the present application first searches for the target answer page based on the overall page, and then searches for the target question corresponding to each question to be identified from the target answer page, which can improve the search efficiency while ensuring the accuracy of the target questions found, thereby improving the efficiency and accuracy of correcting the image to be identified.

[0073] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0075] Figure 1 The present invention is a flowchart of a method for aligning questions according to some exemplary embodiments.

[0076] Figure 2 is a flow chart of another topic alignment method shown in some exemplary embodiments.

[0077] Figure 3a FIG. 4 is a schematic diagram of an image to be recognized shown in some exemplary embodiments.

[0078] Figure 3b is a schematic diagram of another image to be recognized shown in some exemplary embodiments.

[0079] Figure 4 The following is a flowchart of another method for aligning topics according to some exemplary embodiments.

[0080] Figure 5 is a flow chart of yet another topic alignment method shown in some exemplary embodiments.

[0081] Figure 6 The figure is a block diagram of a title alignment device showing some exemplary embodiments.

[0082] Figure 7 The figure is a hardware structure diagram of a computer device shown in some exemplary embodiments. DETAILED DESCRIPTION

[0083] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0084] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0085] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0086] Traditional question alignment methods are mainly divided into two methods: single question alignment and full page alignment. Single question alignment extracts and recognizes the text information of each question, and then compares it with the questions in the question bank to determine the corresponding answer question. This method has high accuracy, but because it needs to be processed question by question, especially when there are a large number of questions, the processing speed is slow, resulting in large consumption of computing resources and long response time. In contrast, full page alignment analyzes the overall image of the test paper and processes multiple questions at one time using full page matching. It has higher processing efficiency and can significantly shorten the processing time. However, the full page alignment method is usually susceptible to changes in layout, especially between different versions of test papers. Differences in question layout may lead to poor alignment, thereby reducing its accuracy.

[0087] In view of this, the present disclosure provides a method, device, computer equipment and storage medium for question alignment. The question alignment method can be applied to the scenario of performing question alignment on images such as user's test papers that need to be corrected. Optionally, the question alignment method can be executed by a correction system, and the correction system can be integrated on a terminal device (such as a learning machine). The terminal device can capture images through a built-in image acquisition component, and then the correction system can perform question alignment on the captured images; or, the image to be identified uploaded by the user can be received, so that the correction system can perform question alignment on the received image to be identified.

[0088] Next, the embodiments of the present disclosure are described in detail.

[0089] The first aspect of the present disclosure provides a method for aligning a topic. Figure 1 , which comprises the following steps:

[0090] S101, determining at least two candidate answer pages corresponding to the image to be identified from the answer page library according to the initial similarity between the image to be identified and each answer page in the answer page library.

[0091] The image to be identified refers to an image that has a question alignment requirement; the answer page library includes multiple answer pages, and the initial similarity refers to the similarity determined when determining the candidate answer page corresponding to the image to be identified from the answer page library, which may include the page similarity between the image to be identified and each answer page in the answer page library, that is, the similarity between the entire image to be identified and the entire image of each answer page in the answer page library. The candidate answer page refers to at least two answer pages obtained by coarse screening from the answer page library.

[0092] Optionally, images can be collected through the built-in image acquisition component of the terminal device to obtain images to be identified, or images to be identified uploaded by users can be received; further, the image to be identified can be compared with the image of each answer page in the answer page library for similarity, and the page similarity between the overall image to be identified and the overall image of each answer page in the answer page library can be used as the initial similarity, and then the answer page with an initial similarity greater than a first preset threshold can be used as a candidate answer page.

[0093] S102: Determine a target answer page from at least two candidate answer pages according to the target similarity between the image to be identified and each candidate answer page.

[0094] The target similarity refers to the similarity determined when determining the target answer page corresponding to the image to be identified from the candidate answer pages, and may include the question similarity between the image to be identified and each candidate answer page, that is, the similarity between each question in the image to be identified and each question in the answer page library. The target answer page refers to the answer page corresponding to the image to be identified obtained by finely screening the candidate answer pages.

[0095] Optionally, a similarity comparison can be performed between the image to be identified and the image of each candidate answer page, and the question similarity between each question in the image to be identified and each question on each candidate answer page can be used as the target similarity, and then the candidate answer page with a target similarity greater than a second preset threshold can be used as the target answer page.

[0096] S103, for each question to be identified in the image to be identified, determining a target question corresponding to the question to be identified from the target answer page.

[0097] The to-be-recognized question refers to each question included in the to-be-recognized image, and the target question refers to the question including the answer corresponding to the to-be-recognized question in the target answer page.

[0098] Optionally, after determining the target answer page corresponding to the image to be identified, for each question to be identified in the image to be identified, the image similarity between the image of the question to be identified and the image of each candidate question in the target answer page can be determined, and then the candidate question with the greatest image similarity is taken as the target question corresponding to the question to be identified.

[0099] In the disclosed embodiment, at least two candidate answer pages corresponding to the image to be identified are determined from the answer page library, and the target answer page corresponding to the image to be identified is determined from the at least two candidate answer pages, and further, the target question corresponding to each question to be identified in the identified image is determined from the target answer page. Compared with the prior art which only searches for the target answer page for each overall answer page, or only searches for the corresponding target question for each question to be identified, the present application first searches for the target answer page based on the overall page, and then searches for the target question corresponding to each question to be identified from the target answer page, which can improve the search efficiency while ensuring the accuracy of the target questions found, thereby improving the efficiency and accuracy of correcting the image to be identified.

[0100] Based on the above embodiments, in an exemplary embodiment, Figure 2 As shown, the above S101 is refined, which may specifically include the following steps:

[0101] S201, determining the number of text content lines and the number of character content lines in the image to be recognized.

[0102] The text content includes text content and non-text content, and the non-text content may be illustrations, tables, etc. The number of text content lines refers to the total number of lines including text content and non-text content in the image to be recognized, and the number of text content lines refers to the number of lines including only text content in the image to be recognized.

[0103] Optionally, statistics may be collected on the image to be identified to determine the number of text content lines including text content in the image to be identified, and the total number of lines including text content and non-text content in the image to be identified.

[0104] S202: Determine the initial similarity between the image to be identified and each answer page in the answer page library according to the number of text content lines and the number of text content lines.

[0105] Optionally, after determining the number of text content lines and the number of text content lines of the image to be identified, the number of text content lines can be compared with a preset first line number threshold, and the number of text content lines can be compared with a preset second line number threshold. Based on the comparison results, the initial similarity between the image to be identified and each answer page in the answer page library can be determined.

[0106] For example, in the image to be identified, Figure 3a As shown, when the number of text content lines is greater than or equal to the first line number threshold, and the number of text content lines is greater than or equal to the second line number threshold, the page similarity between the image to be identified and each answer page in the answer library can be determined, and the page similarity can be directly used as the initial similarity.

[0107] In the image to be recognized, Figure 3b As shown, when the number of lines of text content is less than the first line number threshold, or the number of lines of text content is less than the second line number threshold, the page similarity between the image to be identified and each answer page in the answer page library, and the line text similarity between the image to be identified and each answer page in the answer page library are determined, and the initial similarity between the image to be identified and each answer page in the answer page library is determined based on the page similarity and the line text similarity. Specifically, after determining the page similarity between the image to be identified and each answer page in the answer page library, the line text similarity between all the line texts in the image to be identified and each line text in each answer page can be determined, and then for each answer page in the answer page library, the sum of the page similarity and the line text similarity between the image to be identified and the answer page is determined, and then the sum of the page similarity and the line text similarity between the image to be identified and the answer page is used as the initial similarity between the image to be identified and the answer page.

[0108] It can be understood that by judging whether the number of lines of text content is greater than the first line number threshold, and whether the number of lines of text content is greater than the second line number threshold, it is determined whether the initial similarity between the image to be identified and the answer page is determined by the page similarity and the line text similarity, so that the similarity between the image to be identified and the answer page can be captured more accurately during the recognition process. When the number of lines of text content and the number of lines of text content of the image to be identified meet a higher threshold, the judgment can be made directly by page similarity to improve the calculation efficiency; when the number of lines of text content or the number of lines of text content of the image to be identified is small, the page similarity and the line text similarity are combined to improve the accuracy and robustness of determining the initial similarity.

[0109] S203: If the initial similarity between the image to be identified and any answer page in the answer page library is greater than a first similarity threshold, any answer page is used as a candidate answer page.

[0110] The first similarity threshold refers to a pre-set similarity threshold for screening candidate answer pages from the answer page library.

[0111] Optionally, after determining the initial similarity between the image to be identified and each answer page in the answer page library, each initial similarity can be compared with a first similarity threshold, and the answer page with an initial similarity greater than the first similarity threshold is regarded as a candidate answer page corresponding to the image.

[0112] In this embodiment, by determining the number of lines of text content and the number of lines of text content in the image to be identified, the information in the image can be carefully decomposed and analyzed, so that the text structure characteristics of the image can be taken into account during the preliminary screening; further, by calculating the initial similarity between the image to be identified and each answer page in the answer page library, and screening the candidate answer pages according to the set similarity threshold, it is possible to ensure that answer pages that meet certain similarity standards are selected as candidate answer pages, thereby improving the accuracy of subsequent fine screening to determine the target answer page.

[0113] In an exemplary embodiment, Figure 4 As shown, the above S102 is refined, and specifically may include the following steps:

[0114] S401, for each candidate answer page, determining the line text similarity and question similarity between the image to be identified and the candidate answer page.

[0115] Optionally, after determining at least two candidate answer pages, the line text similarity between each line of text in the image to be identified and each line of text in each candidate answer page can be determined, and the maximum similarity among the obtained similarities is used as the similarity of the line text, and then the sum of the similarities of each line of text in the image to be identified is used as the line text similarity between the image to be identified and the candidate answer page. Furthermore, for each question in the image to be identified, the question similarity between the question and each question in each candidate answer page is determined, and the maximum similarity among the obtained similarities is used as the similarity of the question to be identified, and then the sum of the similarities of each question to be identified in the image to be identified is used as the question similarity between the image to be identified and the candidate answer page.

[0116] It can be understood that, by calculating the similarity between each line of text in the image to be identified and each line of text in the candidate answer page for each candidate answer page, and selecting the maximum similarity as the line text similarity of each line of text, and then adding the line text similarities of each line of text, a more accurate line text similarity between the image to be identified and the candidate answer page can be obtained; and then calculating the similarity between each question in the image to be identified and each question in the candidate answer page one by one, and selecting the maximum similarity as the question similarity of each question, and then adding the question similarities of each question, a more accurate question similarity between the image to be identified and the candidate answer page can be obtained, so as to achieve a more accurate effect of determining the target answer page from the candidate answer pages.

[0117] S402, determining a target similarity between the image to be identified and each candidate answer page according to the line text similarity and the question similarity between the image to be identified and each candidate answer page.

[0118] Optionally, after obtaining the line text similarity and question similarity between the image to be identified and each candidate answer page, the sum of the line text similarity and question similarity between the image to be identified and each candidate answer page can be used as the target similarity between the image to be identified and each candidate answer page to improve the accuracy of the determined target similarity.

[0119] S403: taking the candidate answer page with the largest target similarity as the target answer page.

[0120] Optionally, after determining the target similarity between the image to be identified and each candidate answer page, the target similarities may be sorted in descending order, and the candidate answer page with the largest target similarity is used as the target answer page.

[0121] In this embodiment, by comprehensively considering the line text similarity and question similarity between the image to be identified and each candidate answer page, the similarity between the image to be identified and each candidate answer page can be evaluated more comprehensively and accurately. Line text similarity reflects the degree of alignment of the text content in the image to be identified, while question similarity focuses on the similarity recognition of each question itself in the image to be identified. Combining the similarities of these two dimensions to calculate the target similarity helps to comprehensively evaluate the match between the image to be identified and each candidate answer page. Finally, selecting the candidate answer page with the largest target similarity as the target answer page can improve the accuracy of the determined target answer page.

[0122] In an exemplary embodiment, an implementable method is provided for the above S103, which can determine, for each question to be identified in the image to be identified, the line text similarity and the question similarity between the question to be identified and each candidate question in the target answer page, and take the candidate question with the largest sum of the line text similarity and the question similarity in the target answer page as the target question corresponding to the question to be identified.

[0123] Optionally, after determining the target answer page, each question in the image to be identified can be used as a question to be identified, and each question in the target answer page can be used as a candidate question; for each question to be identified in the image to be identified, each line of text in the question to be identified can be compared with each line of text in each candidate question in the target answer page to determine the similarity between each line of text in the question to be identified and each candidate question in the target answer page, and the sum of the similarities between each line of text in the question to be identified and each candidate question in the target answer page can be used as the line text similarity between the question to be identified and each candidate question in the target answer page; further, the question to be identified can be compared as a whole with each question in the target answer page to determine the question similarity between the question to be identified and each candidate question in the target answer page. For each candidate question in the target answer page, the sum of the line text similarity and the question similarity between the question to be identified and the candidate question is determined, and then the candidate question with the largest sum of the line text similarity and the question similarity is used as the target question corresponding to the question to be identified.

[0124] In this embodiment, by combining the line text similarity and the title similarity between the question to be identified and each candidate question in the target answer page, a multi-dimensional matching analysis is performed on each question to be identified and the candidate questions in the target answer page, so that the correct matching answer to the question to be identified can be determined more accurately. Using the sum of the line text similarity and the title similarity as a measurement standard can ensure that the selected target question is highly consistent with the question to be identified in terms of semantics and structure, thereby improving the accuracy of the determined target question.

[0125] On the basis of the above-mentioned embodiment, in an exemplary embodiment, two possible implementation methods are provided for determining the target title corresponding to the title to be identified according to whether the title to be identified contains non-text content.

[0126] In one case, when the question to be identified does not contain non-text content, the candidate question with the largest sum of the line text similarity and the question similarity in the target answer page is used as the target question corresponding to the question to be identified.

[0127] Optionally, if the question to be identified does not contain non-text content (such as illustrations, tables, etc.), the similarity comparison mainly depends on the similarity between the text content and the question as a whole. The line text similarity and the question similarity between the question to be identified and each candidate question in the target answer page can be determined, and then the sum of the line text similarity and the question similarity between the question to be identified and each candidate question in the target answer page can be determined, and the candidate question with the largest sum of line text similarity and question similarity in the target answer page is used as the target question corresponding to the question to be identified.

[0128] Another situation is that when the question to be identified contains non-text content, the feature vector similarity between the non-text content and the non-text content of each candidate question in the target answer page is determined, and based on the line text similarity, the question similarity and the feature vector similarity in the target answer page, the target question corresponding to the question to be identified is determined from each candidate question on the target answer page.

[0129] The feature vector similarity refers to the similarity between the feature vector of the non-text content in the question to be identified and the feature vector of the non-text content in each candidate question in the target answer page.

[0130] Optionally, if the question to be identified contains non-text content (such as illustrations, tables, etc.), the similarity comparison depends not only on the similarity between the text content and the question as a whole, but also on the feature vector of the non-text content. After determining the line text similarity and question similarity between the question to be identified and each candidate question in the target answer page, feature extraction can be performed on the question to be identified and the non-text content in each candidate question in the target answer page to obtain feature vectors in the question to be identified and each candidate question in the target answer page, and feature vector similarity between the feature vector of the non-text content in the question to be identified and the feature vector of the non-text content in each candidate question in the target answer page can be determined; further, the sum of the line text similarity, question similarity and feature vector similarity between the question to be identified and each candidate question in the target answer page can be determined, and the candidate question with the largest sum of line text similarity, question similarity and feature vector similarity in the target answer page is used as the target question corresponding to the question to be identified.

[0131] It can be understood that, when the question to be identified does not include non-text content, by determining the sum of the line text similarity and the question similarity between the question to be identified and each candidate question in the target answer page, the focus is on the consistency of the text content in the question to be identified, which simplifies the complexity of matching, improves efficiency, and at the same time improves the accuracy of determining the target question corresponding to the question to be identified; secondly, when the question to be identified includes non-text content, by calculating the feature vector similarity between the non-text content in the question to be identified and the non-text content in each candidate question in the target answer page, it is ensured that when the question to be identified includes text content and non-text content, both the text content and the non-text content are fully considered, thereby improving the comprehensiveness and accuracy of determining the target question corresponding to the question to be identified.

[0132] In an exemplary embodiment, when there is a title to be identified in the image to be identified for which no corresponding target title is found, Figure 5 As shown, a feasible implementation method is provided for determining the target topic corresponding to the topic to be identified, which specifically includes the following steps:

[0133] S501, determining at least two candidate topics corresponding to the topic to be identified from a topic database.

[0134] The question bank refers to a database that is based on questions and includes multiple questions and answers corresponding to each question.

[0135] Optionally, if some of the questions to be identified fail to find directly matched target questions, a match can be determined between the question to be identified and each question in the question library, and questions with a matching degree greater than a preset matching threshold are used as candidate questions corresponding to the question to be identified.

[0136] S502: Determine a target topic corresponding to the topic to be identified from at least two candidate topics according to the similarity between the topic to be identified and each candidate topic in the topic database.

[0137] Optionally, after determining at least two candidate questions, the overall similarity between the question to be identified and each candidate question in the question library can be determined, and then the candidate question with the greatest similarity between the question to be identified and each candidate question in the question library is used as the target question corresponding to the question to be identified.

[0138] In this embodiment, when a directly matching target title cannot be found in the image to be identified, a pre-set title library is introduced, at least two candidate titles are selected from the title library, and they are screened according to similarity, so as to more accurately and comprehensively determine the target title corresponding to the title to be identified, thereby improving the efficiency and accuracy of correcting the image to be identified.

[0139] Corresponding to the embodiments of the aforementioned method, the present disclosure also provides embodiments of a device and a terminal to which the device is applied.

[0140] Second, as Figure 6 As shown, the present application also provides a title alignment device, comprising:

[0141] A first determination module 601 is used to determine at least two candidate answer pages corresponding to the image to be identified from the answer page library according to the initial similarity between the image to be identified and each answer page in the answer page library, wherein the answer page library includes a plurality of answer pages, and the initial similarity includes the page similarity between the image to be identified and each answer page in the answer page library;

[0142] A second determination module 602 is used to determine a target answer page from at least two candidate answer pages according to a target similarity between the image to be identified and each candidate answer page, wherein the target similarity includes a question similarity between the image to be identified and each candidate answer page;

[0143] The third determination module 603 is used to determine, for each question to be identified in the image to be identified, a target question corresponding to the question to be identified from the target answer page.

[0144] In one embodiment, the first determining module 601 is specifically configured to:

[0145] Determine the number of text content lines and the number of text content lines in the image to be recognized, wherein the text content includes text content and non-text content;

[0146] Determining the initial similarity between the image to be identified and each answer page in the answer page library according to the number of text content lines and the number of text content lines;

[0147] If the initial similarity between the image to be identified and any answer page in the answer page library is greater than a first similarity threshold, any answer page is used as a candidate answer page.

[0148] In one embodiment, the first determining module 601 is specifically configured to:

[0149] When the number of lines of the text content is greater than or equal to the first line number threshold, and the number of lines of the text content is greater than or equal to the second line number threshold, determining the page similarity between the image to be identified and each answer page in the answer library, and using the page similarity as the initial similarity;

[0150] When the number of lines of text content is less than the first line number threshold, or the number of lines of text content is less than the second line number threshold, determine the page similarity between the image to be identified and each answer page in the answer page library, and the line text similarity between the image to be identified and each answer page in the answer page library, and determine the initial similarity between the image to be identified and each answer page in the answer page library based on the page similarity and the line text similarity.

[0151] In one embodiment, the second determining module 602 is specifically configured to:

[0152] For each candidate answer page, determining the line text similarity and question similarity between the image to be identified and the candidate answer page;

[0153] Determine a target similarity between the image to be identified and each candidate answer page according to the line text similarity and the question similarity between the image to be identified and each candidate answer page;

[0154] The candidate answer page with the largest target similarity is used as the target answer page.

[0155] In one embodiment, the second determining module 602 is specifically configured to:

[0156] For each line of text in the image to be recognized, determine the similarity between the line of text and each line of text in the candidate answer page, and use the maximum similarity among the obtained similarities as the similarity of the line of text;

[0157] The sum of the similarities of each line of text in the image to be recognized is taken as the line of text similarity between the image to be recognized and the candidate answer page.

[0158] In one embodiment, the second determining module 602 is specifically configured to:

[0159] For each question to be identified in the image to be identified, determine the similarity between the question to be identified and each question in the candidate answer page, and use the maximum similarity among the obtained similarities as the similarity of the question to be identified;

[0160] The sum of the similarities of each question to be identified in the image to be identified is taken as the question similarity between the image to be identified and the candidate answer page.

[0161] In one embodiment, the second determining module 602 is specifically configured to:

[0162] The sum of the line text similarity and the question similarity between the image to be identified and each candidate answer page is used as the target similarity between the image to be identified and each candidate answer page.

[0163] In one embodiment, the third determining module 603 is specifically used to:

[0164] For each question to be identified in the image to be identified, determining the line text similarity and question similarity between the question to be identified and each candidate question in the target answer page;

[0165] For each question to be identified in the image to be identified, the candidate question with the largest sum of the line text similarity and the question similarity in the target answer page is used as the target question corresponding to the question to be identified.

[0166] In one embodiment, the third determining module 603 is specifically used to:

[0167] In the case that the question to be identified does not contain non-text content, the candidate question having the largest sum of the similarity between the line text and the question in the target answer page is used as the target question corresponding to the question to be identified;

[0168] In the case where the question to be identified contains non-text content, the feature vector similarity between the non-text content and the non-text content of each candidate question in the target answer page is determined, and based on the line text similarity, the question similarity and the feature vector similarity in the target answer page, the target question corresponding to the question to be identified is determined from each candidate question on the target answer page.

[0169] In one embodiment, the title alignment device further comprises:

[0170] A topic determination module is used to determine at least two candidate topics corresponding to the topic to be identified from a topic library when there is a topic to be identified in the image to be identified for which no corresponding target topic is found, wherein the topic library includes multiple topics and answers corresponding to each topic; and determine the target topic corresponding to the topic to be identified from the at least two candidate topics based on the similarity between the topic to be identified and each candidate topic in the topic library.

[0171] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0172] A third aspect of the present disclosure provides a computer program product, including a computer program / instruction, which implements the method described in the first aspect when executed by a processor.

[0173] For the device embodiments and computer program product embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. In addition, the device embodiments described above are merely schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. Those of ordinary skill in the art can understand and implement it without paying creative labor.

[0174] In a fourth aspect, the embodiment of the title alignment device provided by the present disclosure can be applied to a computer device. Figure 7 , which exemplarily shows a hardware schematic diagram of a computer device. For example, device 700 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0175] Device 700 may include one or more of the following components: a processing component 701 , a memory 702 , a power component 703 , a multimedia component 704 , an audio component 705 , an input / output (I / O) interface 706 , a sensor component 707 , and a communication component 708 .

[0176] The processing component 701 generally controls the overall actions of the device 700, such as actions associated with display, phone calls, data communications, camera actions, and recording actions. The processing component 701 may include one or more processors 709 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 701 may include one or more modules to facilitate the interaction between the processing component 701 and other components. For example, the processing component 701 may include a multimedia module to facilitate the interaction between the multimedia component 704 and the processing component 701.

[0177] The memory 702 is configured to store various types of data to support operations on the device 700. Examples of such data include instructions for any application or method operating on the device 700, contact data, phone book data, messages, pictures, videos, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0178] The power component 703 provides power to the various components of the device 700. The power component 703 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 700.

[0179] The multimedia component 704 includes a screen providing an output interface between the device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide action. In some embodiments, the multimedia component 704 includes a front camera and / or a rear camera. When the device 700 is in an action mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0180] The audio component 705 is configured to output and / or input audio signals. For example, the audio component 705 includes a microphone (MIC), and when the device 700 is in an action mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 702 or sent via the communication component 708. In some embodiments, the audio component 705 also includes a speaker for outputting audio signals.

[0181] I / O interface 706 provides an interface between processing component 701 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0182] The sensor assembly 707 includes one or more sensors for providing various aspects of status assessment for the device 700. For example, the sensor assembly 707 can detect the open / closed state of the device 700, the relative positioning of components, such as the display and keypad of the device 700, and the sensor assembly 707 can also detect the position change of the device 700 or a component of the device 700, the presence or absence of user contact with the device 700, the orientation or acceleration / deceleration of the device 700, and the temperature change of the device 700. The sensor assembly 707 can also include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 707 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 707 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0183] The communication component 708 is configured to facilitate wired or wireless communication between the device 700 and other devices. The device 700 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G or 5G or a combination thereof. In an exemplary embodiment, the communication component 708 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 708 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0184] In an exemplary embodiment, the device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-mentioned topic alignment method of the computer device.

[0185] In a fifth aspect, the present disclosure, in an exemplary embodiment, further provides a non-transitory computer-readable storage medium including instructions, such as a memory 702 including instructions, and the instructions can be executed by a processor 709 of a device 700 to complete the above-mentioned topic alignment method of the computer device. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0186] The above describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0187] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the inventions claimed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not claimed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0188] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0189] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. A method for aligning a topic, characterized in that: The method comprises: Determine at least two candidate answer pages corresponding to the image to be identified from the answer page library according to the initial similarity between the image to be identified and each answer page in the answer page library, wherein the answer page library includes a plurality of answer pages, and the initial similarity includes the page similarity between the image to be identified and each answer page in the answer page library; Determining a target answer page from at least two candidate answer pages according to a target similarity between the image to be identified and each candidate answer page, wherein the target similarity includes a question similarity between the image to be identified and each candidate answer page; For each question to be identified in the image to be identified, a target question corresponding to the question to be identified is determined from the target answer page.

2. The method according to claim 1, characterized in that The step of determining at least two candidate answer pages corresponding to the image to be recognized from the answer page library includes: Determine the number of text content lines and the number of text content lines in the image to be recognized, wherein the text content includes text content and non-text content; Determining the initial similarity between the image to be identified and each answer page in the answer page library according to the number of text content lines and the number of text content lines; If the initial similarity between the image to be identified and any answer page in the answer page library is greater than a first similarity threshold, any answer page is used as a candidate answer page.

3. The method according to claim 2, characterized in that The determining, based on the number of lines of the text content and the number of lines of the word content, the initial similarity between the image to be identified and each answer page in the answer page library comprises: When the number of lines of the text content is greater than or equal to the first line number threshold, and the number of lines of the text content is greater than or equal to the second line number threshold, determining the page similarity between the image to be identified and each answer page in the answer library, and using the page similarity as the initial similarity; When the number of lines of text content is less than the first line number threshold, or the number of lines of text content is less than the second line number threshold, determine the page similarity between the image to be identified and each answer page in the answer page library, and the line text similarity between the image to be identified and each answer page in the answer page library, and determine the initial similarity between the image to be identified and each answer page in the answer page library based on the page similarity and the line text similarity.

4. The method according to claim 1, characterized in that: The step of determining a target answer page from at least two candidate answer pages according to the target similarity between the image to be identified and each candidate answer page comprises: For each candidate answer page, determining the line text similarity and question similarity between the image to be identified and the candidate answer page; Determine a target similarity between the image to be identified and each candidate answer page according to the line text similarity and the question similarity between the image to be identified and each candidate answer page; The candidate answer page with the largest target similarity is used as the target answer page.

5. The method according to claim 4, characterized in that The determining of the line text similarity between the image to be recognized and the candidate answer page comprises: For each line of text in the image to be recognized, determine the similarity between the line of text and each line of text in the candidate answer page, and use the maximum similarity among the obtained similarities as the similarity of the line of text; The sum of the similarities of each line of text in the image to be recognized is taken as the line of text similarity between the image to be recognized and the candidate answer page.

6. The method according to claim 4, characterized in that The determining of the question similarity between the image to be identified and each candidate answer page includes: For each question to be identified in the image to be identified, determine the similarity between the question to be identified and each question in the candidate answer page, and use the maximum similarity among the obtained similarities as the similarity of the question to be identified; The sum of the similarities of each question to be identified in the image to be identified is taken as the question similarity between the image to be identified and the candidate answer page.

7. The method according to claim 4, characterized in that Determining the target similarity between the image to be identified and each candidate answer page according to the line text similarity and the question similarity between the image to be identified and each candidate answer page includes: The sum of the line text similarity and the question similarity between the image to be identified and each candidate answer page is used as the target similarity between the image to be identified and each candidate answer page.

8. The method according to claim 1, characterized in that The step of determining, for each question to be identified in the image to be identified, a target question corresponding to the question to be identified from the target answer page includes: For each question to be identified in the image to be identified, determine the line text similarity and question similarity between the question to be identified and each candidate question in the target answer page, and take the candidate question with the largest sum of the line text similarity and the question similarity in the target answer page as the target question corresponding to the question to be identified.

9. The method according to claim 8, characterized in that The step of taking the candidate question having the largest sum of the similarity between the text line in the target answer page and the question similarity as the target question corresponding to the question to be identified comprises: In the case that the question to be identified does not contain non-text content, the candidate question having the largest sum of the similarity between the line text and the question in the target answer page is used as the target question corresponding to the question to be identified; In the case where the question to be identified contains non-text content, the feature vector similarity between the non-text content and the non-text content of each candidate question in the target answer page is determined, and based on the line text similarity, the question similarity and the feature vector similarity in the target answer page, the target question corresponding to the question to be identified is determined from each candidate question on the target answer page.

10. The method according to claim 1, characterized in that The method further comprises: In the case that there is a question to be identified for which no corresponding target question is found in the image to be identified, determining at least two candidate questions corresponding to the question to be identified from a question library, wherein the question library includes a plurality of questions and an answer corresponding to each question; According to the similarity between the to-be-identified topic and each candidate topic in the topic library, a target topic corresponding to the to-be-identified topic is determined from at least two candidate topics.

11. A question alignment device, characterized in that: The device comprises: a first determination module, configured to determine at least two candidate answer pages corresponding to the image to be identified from the answer page library according to an initial similarity between the image to be identified and each answer page in the answer page library, wherein the answer page library includes a plurality of answer pages, and the initial similarity includes a page similarity between the image to be identified and each answer page in the answer page library; a second determination module, configured to determine a target answer page from at least two candidate answer pages according to a target similarity between the image to be identified and each candidate answer page, wherein the target similarity includes a question similarity between the image to be identified and each candidate answer page; The third determination module is used to determine, for each question to be identified in the image to be identified, a target question corresponding to the question to be identified from the target answer page.

12. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 10 is implemented.

13. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 10 is implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images