A method, apparatus, storage medium, and device for solving plane geometry problems.

Abstract

This application discloses a method, apparatus, storage medium, and device for solving plane geometry problems. The method includes: first, acquiring the target plane geometry problem to be solved and its corresponding target image; then, using a preset first deep learning model to predict entity relationships in the problem text to obtain a first neural symbol representation corresponding to the problem text; and using a preset second deep learning model to transform the target image to obtain its corresponding second neural symbol representation; finally, inputting the first and second neural symbol representations into a symbolic reasoning system, and using beam search to perform reasoning to obtain the solution. It is evident that because this application first uses a deep learning model to convert the problem text and the target image of the target plane geometry problem into neural symbol representations, and then uses beam search to perform reasoning through a symbolic reasoning system, the accuracy of the solution to the plane geometry problem can be improved.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a method, apparatus, storage medium and device for solving plane geometry problems. Background Technology

[0002] Plane geometry problems are a major part of the automatic solution of elementary mathematics problems. With the rapid development of artificial intelligence technology, the automatic solution of plane geometry problems has become an active research problem many times. Driven by the combined efforts of technological progress in related fields and the demand for intelligent education, it has once again become a research hotspot in recent years.

[0003] Currently, solving plane geometry problems typically involves first identifying keywords in the problem text, then converting the text into corresponding neural symbols based on these keywords. Next, a Hough transform is used to extract the geometric elements from the image containing the plane geometry problem. OCR technology is then used to recognize the text labels within the image, and distance rules are applied to convert the image into corresponding neural symbols. Finally, these converted neural symbols are used as input to a symbolic reasoning system, which simultaneously parses the target problem and applies geometric axioms and theorems to reason about the symbols. When the existing neural symbolic representation can evaluate the target problem, the system automatically outputs the final answer. However, this method, which relies on keyword identification to convert the problem text into neural symbols, has significant limitations. It fails to accurately understand the semantics of the problem, making it unsuitable for solving various types of plane geometry problems in real-world scenarios. Furthermore, because this method uses Hough transform to extract geometric elements from images, it is only effective for very simple geometric images. For images with many geometric elements or those affected by lighting, camera angles, or paper curvature, the results are often inaccurate. Furthermore, this current solution method cannot solve proof-based plane geometry problems, which constitute a large proportion of all plane geometry problems. Therefore, the results obtained from automated solutions to plane geometry problems are not accurate enough, and for proof-based problems, it cannot even provide a solution. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, storage medium, and device for solving plane geometry problems, which can improve the accuracy of the solution results for plane geometry problems.

[0005] This application provides a method for solving plane geometry problems, including:

[0006] Obtain the target plane geometry problem to be solved, and obtain the target image where the target plane geometry problem is located;

[0007] Using a preset first deep learning model, entity relationship prediction is performed on the question stem text of the target plane geometry problem to obtain the first neural symbol representation corresponding to the question stem text; and using a preset second deep learning model, the target image is transformed to obtain the second neural symbol representation corresponding to the target image.

[0008] The first neural symbol representation and the second neural symbol representation are input into the symbolic reasoning system, and the reasoning solution is obtained by using beam search.

[0009] In one possible implementation, the step of using a preset first deep learning model to predict entity relationships in the text of the target plane geometry problem to obtain a first neural symbol representation corresponding to the text of the problem stem includes:

[0010] Using a pre-defined first deep learning model, the text of the question stem of the target plane geometry problem is encoded to obtain the encoding features of each character in the question stem text;

[0011] The encoded features of each character in the question text are input into a preset classification layer to identify all entities contained in the question text.

[0012] The encoded features of all words in each entity are subjected to average pooling to obtain the feature vector corresponding to each entity;

[0013] The feature vectors corresponding to every two entities are concatenated, and the relationship between every two entities is predicted based on the concatenation result.

[0014] By combining all entities contained in the question text and the relationships between every two entities, the first neural symbolic representation corresponding to the question text is obtained.

[0015] In one possible implementation, the first deep learning model is a BERT model; the entity is a geometric element; and the entity relationship is the relationship between the geometric elements.

[0016] In one possible implementation, the step of using a preset second deep learning model to transform the target image to obtain a second neural symbol representation corresponding to the target image includes:

[0017] Using OCR technology, the target image is identified to obtain text information and symbol information in the target image;

[0018] The target image is encoded using a pre-defined second deep learning model to obtain a feature map corresponding to the target image.

[0019] Each pixel in the feature map is classified to determine the location information of each geometric element in the target image;

[0020] By using distance rules, the text information, symbol information, and position information of each geometric element in the target image are transformed to obtain the second neural symbol representation corresponding to the target image.

[0021] In one possible implementation, the second deep learning model is a ResNet residual network model.

[0022] In one possible implementation, the step of inputting the first neural symbolic representation and the second neural symbolic representation into a symbolic reasoning system, and using beam search to perform reasoning and obtain a solution, includes:

[0023] The first neural symbolic representation and the second neural symbolic representation are input into the symbolic reasoning system. The symbolic reasoning system searches for N geometric axioms and / or theorems that can be used to reason about the first neural symbolic representation and the second neural symbolic representation; where N is a positive integer greater than 0.

[0024] Using the N geometric axioms and / or theorems, reasoning is performed on the first neural symbolic representation and the second neural symbolic representation to generate N reasoning branches;

[0025] Search among the N reasoning branches for the existence of a solution that satisfies the conditions for the target plane geometry problem;

[0026] If so, the reasoning path corresponding to the reasoning branch that satisfies the solution conditions of the target plane geometry problem is output by the symbolic reasoning system as the solution result;

[0027] If not, for each reasoning branch, the process of searching through the symbolic reasoning system for N geometric axioms and / or theorems that can be used to reason about the first neural symbolic representation and the second neural symbolic representation, and subsequent steps, is repeated until the preset maximum search depth is reached to obtain the solution.

[0028] In one possible implementation, the preset maximum search depth is 5.

[0029] This application also provides a device for solving plane geometry problems, including:

[0030] The acquisition unit is used to acquire the target plane geometry problem to be solved, and to acquire the target image where the target plane geometry problem is located;

[0031] The conversion unit is used to predict entity relationships in the text of the target plane geometry problem using a preset first deep learning model to obtain a first neural symbol representation corresponding to the text; and to convert the target image using a preset second deep learning model to obtain a second neural symbol representation corresponding to the target image.

[0032] The solution unit is used to input the first neural symbol representation and the second neural symbol representation into the symbolic reasoning system, and use beam search to perform reasoning and solution to obtain the solution result.

[0033] In one possible implementation, the conversion unit includes:

[0034] The first encoding subunit is used to encode the text of the question stem of the target plane geometry problem using a preset first deep learning model, so as to obtain the encoding features of each character in the question stem text;

[0035] The first identification subunit is used to input the encoding features of each character in the question text into a preset classification layer to identify all entities contained in the question text;

[0036] The pooling subunit is used to perform average pooling on the encoded features of all words in each entity to obtain the feature vector corresponding to each entity.

[0037] The splicing subunit is used to splice the feature vectors corresponding to every two entities and predict the relationship between every two entities based on the splicing result.

[0038] The combination subunit is used to combine all entities contained in the question text and the relationships between every two entities to obtain the first neural symbol representation corresponding to the question text.

[0039] In one possible implementation, the first deep learning model is a BERT model; the entity is a geometric element; and the entity relationship is the relationship between the geometric elements.

[0040] In one possible implementation, the conversion unit includes:

[0041] The second recognition subunit is used to recognize the target image using OCR technology to obtain text information and symbol information in the target image;

[0042] The second encoding and recognition subunit is used to encode the target image using a preset second deep learning model to obtain the feature map corresponding to the target image;

[0043] The classification subunit is used to classify each pixel in the feature map and determine the location information of each geometric element in the target image.

[0044] The transformation subunit is used to transform the text information, symbol information, and position information of each geometric element in the target image using distance rules, so as to obtain the second neural symbol representation corresponding to the target image.

[0045] In one possible implementation, the second deep learning model is a ResNet residual network model.

[0046] In one possible implementation, the solution unit includes:

[0047] The first search subunit is used to input the first neural symbolic representation and the second neural symbolic representation into the symbolic reasoning system, and through the symbolic reasoning system, search for N geometric axioms and / or theorems that can be used to reason about the first neural symbolic representation and the second neural symbolic representation; where N is a positive integer greater than 0.

[0048] The reasoning subunit is used to reason about the first neural symbolic representation and the second neural symbolic representation using the N geometric axioms and / or theorems, generating N reasoning branches.

[0049] The second search subunit is used to search from the N reasoning branches for whether there are any solution conditions that satisfy the target plane geometry problem;

[0050] The first output subunit is used to output the reasoning path corresponding to the reasoning branch that satisfies the solution conditions of the target plane geometry problem through the symbolic reasoning system if a solution condition that satisfies the solution conditions of the target plane geometry problem is found from the N reasoning branches, as the solution result;

[0051] The second output subunit is used to repeatedly call and execute the first search subunit, the reasoning subunit, the second search subunit, and the first output subunit for each reasoning branch if no solution condition for the target plane geometry problem is found from the N reasoning branches, until the preset maximum search depth is reached to obtain the solution result.

[0052] In one possible implementation, the preset maximum search depth is 5.

[0053] This application embodiment also provides a device for solving plane geometry problems, including: a processor, a memory, and a system bus;

[0054] The processor and the memory are connected via the system bus;

[0055] The memory is used to store one or more programs, the one or more programs including instructions, which, when executed by the processor, cause the processor to perform any of the above-described methods for solving the plane geometry problem.

[0056] This application also provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform any of the above-described methods for solving plane geometry problems.

[0057] This application also provides a computer program product, which, when run on a terminal device, causes the terminal device to execute any one of the above-described methods for solving plane geometry problems.

[0058] This application provides a method, apparatus, storage medium, and device for solving plane geometry problems. First, it acquires the target plane geometry problem to be solved and the target image containing the problem. Then, using a preset first deep learning model, it predicts entity relationships in the text of the problem stem to obtain a first neural symbol representation corresponding to the text. Next, using a preset second deep learning model, it transforms the target image to obtain a second neural symbol representation. Finally, it inputs the first and second neural symbol representations into a symbolic reasoning system and uses beam search to perform reasoning to obtain the solution. As can be seen, this application uses deep learning models to convert the text of the target plane geometry problem and its target image into neural symbolic representations, thus avoiding the inaccurate conversion results caused by using keyword searching and Hough transform to process the text and image separately in existing technologies. Furthermore, this application utilizes beam search and a symbolic reasoning system to perform reasoning, thereby solving proof-type plane geometry problems that existing methods cannot solve, and thus improving the accuracy of the solution results for plane geometry problems. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a schematic diagram illustrating the conversion of existing plane geometry problem text and images into neural symbols;

[0061] Figure 2A flowchart illustrating a method for solving a plane geometry problem, provided in an embodiment of this application;

[0062] Figure 3 Example diagrams illustrating the solution process of plane geometry problems provided in embodiments of this application;

[0063] Figure 4 An example diagram provided for embodiments of this application, which classifies each pixel in the feature map to determine the location information of each geometric element in the target image;

[0064] Figure 5 This is a schematic diagram illustrating the composition of a device for solving plane geometry problems, as provided in an embodiment of this application. Detailed Implementation

[0065] Solving plane geometry problems plays a vital role in cultivating students' abstract thinking abilities. Psychologists and educators generally believe that solving plane geometry problems requires advanced thinking skills such as symbolic abstraction and logical reasoning. It is extremely difficult for an algorithm to understand the abstract semantics of the original problem and then reason and calculate the answer based on known geometric knowledge.

[0066] Currently, solving plane geometry problems typically involves two steps: the first step is to convert both the problem statement and the images into neural symbol representations, such as... Figure 1 As shown, this step typically involves first identifying keywords in the question stem text, and then converting the text into corresponding neural symbols based on those keywords. That is, using fixed symbols, such as fixed English words, to represent each set element (including but not limited to line segments, triangles, angles, etc.), as shown in the attached diagram. Figure 1 If the question text states "△BCD is an equilateral triangle", it will be transformed into `Equilateral(Triangle(B,C,D))`, where `Equilateral` and `Triangle` are predefined predicates representing equilateral and triangle respectively. For images, a Hough transform will be used to obtain the geometric elements in the image (e.g., ...). Figure 1 The points and line segments in the image are then identified using Optical Character Recognition (OCR) technology, which identifies the text labels (such as points and line segments) in the image. Figure 1 The image is processed using the letters A, B, C, D, and vertical symbols. Finally, a distance rule (i.e., the nearest matching rule) is used to convert the image into corresponding neural symbols. For example, the point closest to the symbol "A" is point A. Similarly, points B, C, and D can be obtained. For the vertical symbol, the two line segments closest to the vertical symbol are BC and AC. Therefore, the neural symbol expression `Perpendicular(Line(A,C),Line(B,C))` can be obtained. Figure 1As shown. The second step is to use the converted neural symbols as input to the symbolic reasoning system, simultaneously parsing out the target to be solved, and then reasoning about the symbols based on geometric axioms and theorems. When the existing neural symbolic expression can evaluate the target, the system can automatically output the final answer. However, this solution method has the following shortcomings:

[0067] (1) Using keyword search to convert the question stem text into neural symbols has great limitations. The same neural symbol may appear in different question stem texts. For example, Equilateral(Triangle(B,C,D)) can be described as "△BCD is an equilateral triangle" or "△BCD has three equal sides". Relying solely on keywords cannot accurately understand the semantics of the question stem text, which makes it impossible to solve various types of plane geometry problems in real-world scenarios.

[0068] (2) Using Hough transform to obtain geometric elements in an image is only suitable for very simple geometric images. For geometric images with many geometric elements, the accuracy of Hough transform is not high enough, and the expected information may not be detected, which may lead to incorrect judgments and generate a lot of redundant data. Furthermore, if the geometric image is obtained by taking a picture, Hough transform will also be affected by lighting, shooting angle, paper curvature, etc., which will greatly reduce its performance.

[0069] (3) The current solution method cannot solve proof-type plane geometry problems, but proof-type plane geometry problems account for a large proportion of plane geometry problems.

[0070] To address the aforementioned shortcomings and improve the accuracy of solutions to plane geometry problems, this application provides a method for solving plane geometry problems. First, the target plane geometry problem to be solved and its corresponding target image are obtained. Then, a pre-defined first deep learning model is used to predict entity relationships in the text of the target plane geometry problem, resulting in a first neural symbol representation corresponding to the text. A pre-defined second deep learning model is then used to transform the target image, resulting in a second neural symbol representation corresponding to the target image. Next, the first and second neural symbol representations are input into a symbolic reasoning system, and a beam search method is used for reasoning to obtain the solution. As can be seen, this application utilizes deep learning models to convert the text of the target plane geometry problem and its corresponding target image into neural symbol representations, thus avoiding the inaccurate conversion results caused by separately processing the text and image using keyword searching and Hough transform in existing technologies. Furthermore, this application utilizes a beam search method and a symbolic reasoning system for reasoning, thereby solving proof-type plane geometry problems that existing methods cannot solve, and thus improving the accuracy of solutions to plane geometry problems.

[0071] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0072] First Embodiment

[0073] See Figure 2 This is a flowchart illustrating a method for solving a plane geometry problem provided in this embodiment. The method includes the following steps:

[0074] S201: Obtain the target plane geometry problem to be solved, and obtain the target image where the target plane geometry problem is located.

[0075] In this embodiment, any plane geometry problem solved using this embodiment is defined as the target plane geometry problem, and the image containing the target plane geometry problem is defined as the target image. Furthermore, this embodiment does not limit the language type of the target plane geometry problem's text; for example, the target plane geometry problem's text can be Chinese or English. This embodiment also does not limit the source of the target plane geometry problem's text; for example, the target problem text can be the result of speech recognition or plane geometry problem data obtained from a network platform.

[0076] Understandably, the text of the target plane geometry problem can be a sentence or a passage, which is a collection of words or sentences. After obtaining the sentence or passage as the target plane geometry problem to be solved and the target image in which it is located, it can be processed and solved according to the subsequent steps.

[0077] Furthermore, this embodiment does not limit the type of the target image. For example, the target image can be a color image composed of the three primary colors of red (R), green (G), and blue (B), or it can be a grayscale image. Also, this embodiment does not limit the resolution of the target image. For example, the target image can be a 720*480 RGB image or a high-resolution 1920*1280 RGB image. This embodiment also does not limit the method of acquiring the target image; for example, it can be obtained by taking a picture of the document using a mobile phone camera.

[0078] S202: Using a preset first deep learning model, perform entity relationship prediction on the question stem text of the target plane geometry problem to obtain the first neural symbol representation corresponding to the question stem text; and use a preset second deep learning model to perform transformation processing on the target image to obtain the second neural symbol representation corresponding to the target image.

[0079] In this embodiment, after obtaining the target plane geometry problem to be solved and the target image where the target plane geometry problem is located through step S201, in order to solve the target plane geometry problem quickly and accurately, the geometric elements (including but not limited to line segments, triangles, angles, etc.) in the target plane geometry problem can be defined as entities, and the relationships between geometric elements (including but not limited to angle bisectors, perpendicularity, parallelism, etc.) can be defined as relationships between entities. Then, a preset first deep learning model is used to predict entity relationships in the question stem text of the target plane geometry problem to obtain the first neural symbol representation corresponding to the question stem text, so as to more accurately represent the semantic information of the question stem text of the target plane geometry problem. Then, a preset second deep learning model is used to transform the target image to obtain the second neural symbol representation corresponding to the target image, so as to execute the subsequent step S203.

[0080] It should be noted that the specific structures of the first and second deep learning models can be set according to the actual situation. This application does not limit this. One possible implementation is that the first deep learning model can be preset as a BERT model and the second deep learning model can be preset as a Residual Network (ResNet) model.

[0081] Specifically, one possible implementation is that the process of "using a preset first deep learning model to predict entity relationships in the text of the target plane geometry problem and obtain the first neural symbol representation corresponding to the text of the problem stem" in step S202 may include the following steps A1-A5:

[0082] Step A1: Using the preset first deep learning model, encode the text of the target plane geometry problem to obtain the encoding features of each character in the text.

[0083] In this implementation, to more accurately convert the text of the target plane geometry problem into a neural symbolic representation, the text of the target plane geometry problem is first input into a preset first deep learning model (such as the BERT model) to predict the entities and relationships between them in the text of the target plane geometry problem. Specifically, the text of the target plane geometry problem can be represented as (w1, w2, w3, ..., w n ,…), where w i For the i-th character (or word, character, etc.) in the question stem text, the question stem text of the target plane geometry problem is encoded using a first deep learning model (such as the BERT model). The encoded feature representation of each character in the question stem text is as follows:

[0084] E i =Bert(w i )

[0085] Among them, E i The encoded feature representation of the i-th character (or word, character, etc.) in the text of the target plane geometry problem stem; w i This refers to the i-th character (or word, character, etc.) in the text of the target plane geometry problem.

[0086] Step A2: Input the encoding features of each character in the question text into the preset classification layer to identify all entities contained in the question text.

[0087] After obtaining the encoding features of each character in the text of the target plane geometry problem through step A1, the encoding features of each character in the text can be further input into a preset classification layer to determine whether each character is the beginning of an entity, the middle of an entity, the end of an entity, or a non-entity. For example: Figure 3 As shown, assuming Figure 3 The leftmost plane geometry problem is the target plane geometry problem. From the text of the target plane geometry problem, we can determine that "△" is the beginning of the solid triangle, the first letter "C" is the end of the solid triangle, and "as shown in the figure" is a non-solid.

[0088] By analogy, all entities contained in the question text can be identified. Still using... Figure 3For example, all entities in the text of the target plane geometry problem include Triangle(A,B,C), Angle(A,B,C), 90, Line(B,F), Line(A,C), Point(D), etc.

[0089] Step A3: Perform average pooling on the encoded features of all words in each entity to obtain the feature vector corresponding to each entity.

[0090] After identifying all entities contained in the text of the target plane geometry problem through step A2, the encoded features of all characters in each entity can be averaged to obtain the feature vector corresponding to each entity. For example, the feature vector corresponding to the entity "△ABC" can be obtained by average pooling the encoded features of the four characters "△", "A", "B" and "C". Assuming that the encoded features of these four characters are all 768-dimensional vectors, the average of these four 768-dimensional vectors can be calculated to obtain a 768-dimensional average vector, which can be used as the feature vector corresponding to the entity "△ABC".

[0091] By analogy, the feature vector corresponding to each entity contained in the question text can be calculated, which can then be used to execute the subsequent step A4.

[0092] Step A4: Concatenate the feature vectors corresponding to every two entities, and predict the relationship between every two entities based on the concatenation result.

[0093] After calculating the feature vector corresponding to each entity in the question text in step A3, the feature vectors corresponding to every two entities can be concatenated to obtain the concatenated result. This concatenated result is then used to predict the relationship (or lack thereof) between the two entities. Specifically, the obtained concatenated vector can be input into a classification layer for classification, and a multi-dimensional vector can be output through softmax. The value of each dimension of the vector is between 0 and 1, which represents the probability that the relationship between the two entities belongs to various preset entity relationships (such as parallel, perpendicular, etc.) or is unrelated. The preset entity relationship or unrelated relationship corresponding to the highest probability value is taken as the relationship between the two entities.

[0094] For example: Figure 3 The relationship between entities “AB” and “BD” is equal (represented by the neural symbol Equals), the relationship between entities “DE” and “BD” is perpendicular (represented by the neural symbol Perpendicular), and there is no relationship between entities “AB” and “DE”.

[0095] Step A5: Combine all entities contained in the question stem text and the relationships between every two entities to obtain the first neural symbolic representation corresponding to the question stem text.

[0096] After identifying all entities contained in the text of the target plane geometry problem in step A2, and calculating the relationship between every two entities in step A4, the entities contained in the text and the relationships between every two entities can be combined to obtain the first neural symbolic representation corresponding to the text, such as... Figure 3 The content shown in the box above "Neural Symbolic Representation" is as follows.

[0097] In another optional implementation of this application embodiment, the process of "using a preset second deep learning model to perform transformation processing on the target image to obtain the second neural symbol representation corresponding to the target image" in step S202 may specifically include the following steps B1-B4:

[0098] Step B1: Use OCR technology to recognize the target image and obtain the text information and symbol information in the target image.

[0099] In this implementation, to improve the accuracy of solving the target plane geometry problem, it is necessary not only to more accurately convert the problem text into neural symbolic representation, but also to more accurately convert the target image containing the problem into neural symbolic representation. Specifically, firstly, existing or future image recognition methods, such as OCR technology, can be used to recognize the target image to obtain the text and symbolic information in the target image for subsequent step B2. For example, OCR technology can be used to recognize... Figure 3 The leftmost target plane geometry problem is located in the target image with the positions of "A", "B", "C", "D", "E", "F" and the two vertical symbols.

[0100] Step B2: Use a pre-set second deep learning model to encode the target image and obtain the feature map corresponding to the target image.

[0101] After obtaining the target image containing the target planar geometry problem, a pre-set second deep learning model (such as the ResNet model) can be used to encode the target image to generate a feature map corresponding to the target image, which can then be used to execute the subsequent step B3.

[0102] Step B3: Classify each pixel in the feature map to determine the location information of each geometric element in the target image.

[0103] After generating the feature map corresponding to the target image in step B2, each pixel in the feature map can be further classified. For example, the obtained feature map corresponding to the target image can be input into a classification layer to classify each pixel in the feature map, and a multi-dimensional vector is output through softmax, where the value of each dimension of the vector is between 0 and 1, to represent the probability that each pixel belongs to various preset geometric elements (including but not limited to points, line segments, arcs, etc.) or the background, and the preset geometric element or background corresponding to the highest probability value is used as the classification of the corresponding pixel. In this way, the location information of each geometric element in the target image can be determined.

[0104] For example: it is possible to Figure 3 In the feature map of the target image containing the target plane geometry problem, each pixel is classified, such as... Figure 4 As shown, pixels categorized as background are marked in white, pixels categorized as points are marked in dark gray, and pixels categorized as line segments are marked in light gray. This method is used to determine the position of all geometric elements in the target image.

[0105] Step B4: Using distance rules, transform the text information, symbol information, and location information of each geometric element in the target image to obtain the second neural symbol representation corresponding to the target image.

[0106] After identifying the text and symbol information in the large target image using OCR technology in step B1, and determining the location information of each geometric element in the target image in step B3, distance rules can be further used to transform the text and symbol information and the location information of each geometric element in the target image. This transforms the entire target image into a corresponding second neural symbol representation. For example... Figure 4 The point in the top left corner that is closest to the symbol "A" is point A. Figure 4 The vertical symbol in the lower left corner is closest to lines AB and BC, thus yielding the neural symbol expression Perpendicular(Line(A,B),Line(B,C)). Similarly, this leads to... Figure 3 The second neural symbol representation corresponding to the target image is shown in the box below “Neural Symbol Representation” in the middle.

[0107] S203: Input the first neural symbolic representation and the second neural symbolic representation into the symbolic reasoning system, and use the beam search method to reason and solve the problem to obtain the solution result.

[0108] In this embodiment, step S202 utilizes a preset first deep learning model to predict entity relationships in the text of the target plane geometry problem, obtaining a first neural symbolic representation corresponding to the text; and then, using a preset second deep learning model, the target image is transformed to obtain a second neural symbolic representation corresponding to the target image. Furthermore, the first and second neural symbolic representations can be used as initial conditions input into the symbolic reasoning system, such as... Figure 3 As shown in the figure on the right, the beam search method is then used for reasoning to obtain the solution, thereby improving the accuracy of the solution to the target plane geometry problem.

[0109] Specifically, one possible implementation is to first input the first and second neural symbolic representations as initial conditions into the symbolic reasoning system, and then search for N (N is a positive integer greater than 0) geometric axioms and / or theorems that can be used to reason about the first and second neural symbolic representations. The geometric axioms include, but are not limited to, the supplementary angles of the same angle are equal, the complementary angles are also equal, the vertical angles are equal, and the sum of the interior angles of a triangle is 180°. The geometric theorems include, but are not limited to, the similar triangle theorem and the congruent triangle theorem.

[0110] Then, using these N geometric axioms and / or theorems, reasoning is performed on the first and second neural symbolic representations to generate N reasoning branches, thus expanding the initial conditions. It should be noted that if reasoning using the axioms and theorems fails to generate new conditions, that reasoning branch can be discarded.

[0111] Next, the system searches among these N inference branches to see if any conditions satisfying the solution to the target plane geometry problem exist. If so, the symbolic inference system outputs the inference path corresponding to the inference branch satisfying the solution conditions, which is the solution result. If not, for each inference branch, the system repeatedly searches for N geometric axioms and / or theorems that can infer the first and second neural symbolic representations, along with subsequent steps, until the preset maximum search depth is reached, yielding the solution result. In other words, the system searches among these N expanded condition branches to see if the preset conditions to be solved are present. If the preset conditions to be solved are in a certain expanded condition branch, the inference path of that condition branch is output, which is the final solution step. If the preset conditions to be solved cannot be found in any of these N expanded condition branches, the above search and subsequent processing steps are repeated for each expanded condition branch until the preset maximum search depth is reached, yielding the solution result.

[0112] The preset maximum search depth is determined by factors such as problem-solving time and problem-solving rate, and can be set according to actual conditions. This application does not limit this setting. For example, the preset maximum search depth can be set to 5.

[0113] For example: still using the example of... Figure 3 Taking the solution of a target plane geometry problem as an example, the specific solution process is as follows: First, after traversing the N geometric axioms and / or theorems found through the symbolic reasoning system that can infer the first and second neural symbolic representations, based on the perpendicularity axiom, ∠AFB=90° and ∠EDB=90° are deduced. If no proof satisfying the solution conditions of the target plane geometry problem is found under this conditional branch, the first iteration is initiated.

[0114] In the first iteration, based on the geometric axiom that the sum of the interior angles of a triangle is 180°, we can deduce that ∠ABF + ∠CAB = 90°. However, no proof satisfying the solution conditions for the target plane geometry problem was found under this conditional branch, necessitating a second iteration.

[0115] In the second iteration, based on the geometric axiom that complementary angles of the same angle are equal, it can be deduced that ∠CAB=∠EBD. However, no proof satisfying the solution conditions of the target plane geometry problem is found under this conditional branch, necessitating a third iteration.

[0116] In the third iteration, based on the congruent triangle theorem, it is deduced that △ABC≌△BDE. This leads to the conclusion that BC=DE. That is, under this conditional branch, a proof that satisfies the solution conditions of the target plane geometry problem (i.e., the target problem to be solved) can be found. Therefore, the system can output the reasoning path of this conditional branch as the solution result, achieving automatic solution to the target plane geometry problem and improving the accuracy of the solution results.

[0117] In summary, this embodiment provides a method for solving plane geometry problems. First, it obtains the target plane geometry problem to be solved and the target image containing the problem. Then, using a preset first deep learning model, it predicts entity relationships in the text of the problem stem to obtain a first neural symbol representation corresponding to the text. Next, using a preset second deep learning model, it transforms the target image to obtain a second neural symbol representation. Finally, it inputs the first and second neural symbol representations into a symbolic reasoning system and uses beam search to perform reasoning to obtain the solution. As can be seen, this application uses deep learning models to convert the text of the target plane geometry problem and its target image into neural symbolic representations, thus avoiding the inaccurate conversion results caused by using keyword searching and Hough transform to process the text and image separately in existing technologies. Furthermore, this application utilizes beam search and a symbolic reasoning system to perform reasoning, thereby solving proof-type plane geometry problems that existing methods cannot solve, and thus improving the accuracy of the solution results for plane geometry problems.

[0118] Second Embodiment

[0119] This embodiment will introduce a device for solving plane geometry problems; please refer to the above method embodiment for related content.

[0120] See Figure 5 This is a schematic diagram of the composition of a plane geometry problem-solving device provided in this embodiment. The device 500 includes:

[0121] The acquisition unit 501 is used to acquire the target plane geometry problem to be solved, and to acquire the target image where the target plane geometry problem is located;

[0122] The conversion unit 502 is used to use a preset first deep learning model to predict entity relationships in the question stem text of the target plane geometry problem to obtain a first neural symbol representation corresponding to the question stem text; and to use a preset second deep learning model to convert the target image to obtain a second neural symbol representation corresponding to the target image.

[0123] The solution unit 503 is used to input the first neural symbol representation and the second neural symbol representation into the symbol reasoning system, and use beam search to perform reasoning and solution to obtain the solution result.

[0124] In one implementation of this embodiment, the conversion unit 502 includes:

[0125] The first encoding subunit is used to encode the text of the question stem of the target plane geometry problem using a preset first deep learning model, so as to obtain the encoding features of each character in the question stem text;

[0126] The first identification subunit is used to input the encoding features of each character in the question text into a preset classification layer to identify all entities contained in the question text;

[0127] The pooling subunit is used to perform average pooling on the encoded features of all words in each entity to obtain the feature vector corresponding to each entity.

[0128] The splicing subunit is used to splice the feature vectors corresponding to every two entities and predict the relationship between every two entities based on the splicing result.

[0129] The combination subunit is used to combine all entities contained in the question text and the relationships between every two entities to obtain the first neural symbol representation corresponding to the question text.

[0130] In one implementation of this embodiment, the first deep learning model is a BERT model; the entity is a geometric element; and the entity relationship is the relationship between the geometric elements.

[0131] In one implementation of this embodiment, the conversion unit 502 includes:

[0132] The second recognition subunit is used to recognize the target image using OCR technology to obtain text information and symbol information in the target image;

[0133] The second encoding and recognition subunit is used to encode the target image using a preset second deep learning model to obtain the feature map corresponding to the target image;

[0134] The classification subunit is used to classify each pixel in the feature map and determine the location information of each geometric element in the target image.

[0135] The transformation subunit is used to transform the text information, symbol information, and position information of each geometric element in the target image using distance rules, so as to obtain the second neural symbol representation corresponding to the target image.

[0136] In one implementation of this embodiment, the second deep learning model is a ResNet residual network model.

[0137] In one implementation of this embodiment, the solution unit 503 includes:

[0138] The first search subunit is used to input the first neural symbolic representation and the second neural symbolic representation into the symbolic reasoning system, and through the symbolic reasoning system, search for N geometric axioms and / or theorems that can be used to reason about the first neural symbolic representation and the second neural symbolic representation; where N is a positive integer greater than 0.

[0139] The reasoning subunit is used to reason about the first neural symbolic representation and the second neural symbolic representation using the N geometric axioms and / or theorems, generating N reasoning branches.

[0140] The second search subunit is used to search from the N reasoning branches for whether there are any solution conditions that satisfy the target plane geometry problem;

[0141] The first output subunit is used to output the reasoning path corresponding to the reasoning branch that satisfies the solution conditions of the target plane geometry problem through the symbolic reasoning system if a solution condition that satisfies the solution conditions of the target plane geometry problem is found from the N reasoning branches, as the solution result;

[0142] The second output subunit is used to repeatedly call and execute the first search subunit, the reasoning subunit, the second search subunit, and the first output subunit for each reasoning branch if no solution condition for the target plane geometry problem is found from the N reasoning branches, until the preset maximum search depth is reached to obtain the solution result.

[0143] In one implementation of this embodiment, the preset maximum search depth is 5.

[0144] Furthermore, embodiments of this application also provide a device for solving plane geometry problems, including: a processor, a memory, and a system bus;

[0145] The processor and the memory are connected via the system bus;

[0146] The memory is used to store one or more programs, the one or more programs including instructions, which, when executed by the processor, cause the processor to perform any of the above-described implementations of the solution method for the plane geometry problem.

[0147] Furthermore, embodiments of this application also provide a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to execute any of the above-described methods for solving the plane geometry problem.

[0148] Furthermore, this application embodiment also provides a computer program product, which, when run on a terminal device, causes the terminal device to execute any of the above-described methods for solving plane geometry problems.

[0149] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0150] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0151] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0152] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for solving plane geometry problems, characterized in that, include: Obtain the target plane geometry problem to be solved, and obtain the target image where the target plane geometry problem is located; Using a pre-defined first deep learning model, the text of the question stem of the target plane geometry problem is encoded to obtain the encoding features of each character in the question stem text; The encoded features of each character in the question text are input into a preset classification layer to identify all entities contained in the question text; the entities are geometric elements in the target plane geometry problem. The encoded features of all words in each entity are subjected to average pooling to obtain the feature vector corresponding to each entity; The feature vectors corresponding to every two entities are concatenated, and the relationship between every two entities is predicted based on the concatenation result; the relationship between every two entities is the relationship between the geometric elements corresponding to every two entities. The first neural symbol representation corresponding to the question stem text is obtained by combining all entities contained in the question stem text and the relationships between every two entities; and the target image is transformed using a preset second deep learning model to obtain the second neural symbol representation corresponding to the target image. The first neural symbolic representation and the second neural symbolic representation are input into the symbolic reasoning system. The symbolic reasoning system searches for N geometric axioms and / or theorems that can be used to reason about the first neural symbolic representation and the second neural symbolic representation; where N is a positive integer greater than 0. Using the N geometric axioms and / or theorems, reasoning is performed on the first neural symbolic representation and the second neural symbolic representation to generate N reasoning branches; Search among the N reasoning branches for the existence of a solution that satisfies the conditions for the target plane geometry problem; If so, the reasoning path corresponding to the reasoning branch that satisfies the solution conditions of the target plane geometry problem is output by the symbolic reasoning system as the solution result; If not, for each reasoning branch, the process of searching through the symbolic reasoning system for N geometric axioms and / or theorems that can be used to reason about the first neural symbolic representation and the second neural symbolic representation, and subsequent steps, is repeated until the preset maximum search depth is reached to obtain the solution.

2. The method according to claim 1, characterized in that, The first deep learning model is a BERT model; the entity is a geometric element; the entity relationship is the relationship between the geometric elements.

3. The method according to claim 1, characterized in that, The step of using a preset second deep learning model to transform the target image to obtain the second neural symbol representation corresponding to the target image includes: Using OCR technology, the target image is identified to obtain text information and symbol information in the target image; The target image is encoded using a pre-defined second deep learning model to obtain a feature map corresponding to the target image. Each pixel in the feature map is classified to determine the location information of each geometric element in the target image; By using distance rules, the text information, symbol information, and position information of each geometric element in the target image are transformed to obtain the second neural symbol representation corresponding to the target image.

4. The method according to claim 3, characterized in that, The second deep learning model is the ResNet residual network model.

5. The method according to claim 1, characterized in that, The preset maximum search depth is 5.

6. A device for solving plane geometry problems, characterized in that, include: The acquisition unit is used to acquire the target plane geometry problem to be solved, and to acquire the target image where the target plane geometry problem is located; The conversion unit is used to predict entity relationships in the text of the target plane geometry problem using a preset first deep learning model to obtain a first neural symbol representation corresponding to the text; and to convert the target image using a preset second deep learning model to obtain a second neural symbol representation corresponding to the target image. The solution unit is used to input the first neural symbol representation and the second neural symbol representation into the symbolic reasoning system, and use beam search to perform reasoning and solution to obtain the solution result. The conversion unit includes: The first encoding subunit is used to encode the text of the question stem of the target plane geometry problem using a preset first deep learning model, so as to obtain the encoding features of each character in the question stem text; The first identification subunit is used to input the encoding features of each character in the question text into a preset classification layer to identify all entities contained in the question text; the entities are geometric elements in the target plane geometry problem. The pooling subunit is used to perform average pooling on the encoded features of all words in each entity to obtain the feature vector corresponding to each entity. The splicing subunit is used to splice the feature vectors corresponding to every two entities and predict the relationship between every two entities based on the splicing result; the relationship between every two entities is the relationship between the geometric elements corresponding to every two entities. The combination subunit is used to combine all entities contained in the question text and the relationship between every two entities to obtain the first neural symbol representation corresponding to the question text. The solution unit includes: The first search subunit is used to input the first neural symbolic representation and the second neural symbolic representation into the symbolic reasoning system, and through the symbolic reasoning system, search for N geometric axioms and / or theorems that can be used to reason about the first neural symbolic representation and the second neural symbolic representation; where N is a positive integer greater than 0. The reasoning subunit is used to reason about the first neural symbolic representation and the second neural symbolic representation using the N geometric axioms and / or theorems, generating N reasoning branches. The second search subunit is used to search from the N reasoning branches for whether there are any solution conditions that satisfy the target plane geometry problem; The first output subunit is used to output the reasoning path corresponding to the reasoning branch that satisfies the solution conditions of the target plane geometry problem through the symbolic reasoning system if a solution condition that satisfies the solution conditions of the target plane geometry problem is found from the N reasoning branches, as the solution result; The second output subunit is used to repeatedly call and execute the first search subunit, the reasoning subunit, the second search subunit, and the first output subunit for each reasoning branch if no solution condition for the target plane geometry problem is found from the N reasoning branches, until the preset maximum search depth is reached to obtain the solution result.

7. A device for solving plane geometry problems, characterized in that, include: Processor, memory, system bus; The processor and the memory are connected via the system bus; The memory is used to store one or more programs, the one or more programs including instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the method described in any one of claims 1-5.

Images