Geometric large model training method and device, geometric large model application method and device, medium and electronic equipment

Abstract

The invention belongs to the technical field of artificial intelligence, and relates to a geometric large model training method and device, an application method and device, a medium and electronic equipment. The method comprises the following steps: obtaining a geometric image, wherein the geometric image is obtained by reconstructing a geometric form language; a geometric text is determined according to the geometric image, the geometric image and the geometric text are used for training a geometric large model to obtain a geometric large model meeting preset conditions, the geometric text comprises a geometric problem text and a geometric answer text corresponding to the geometric image, and the geometric large model is used for solving the geometric problem. The computing power in the reconstruction process is reduced, the consistency problem of the vertex annotation and the topological structure of the geometric image and the matching degree problem of the reconstructed image and the combined text are solved, the geometric visual perception ability is improved, the two schemes of topological relation perception and geometric axiom alignment are integrated, and the method has the advantages of being high in practicability and easy to popularize. The generalization ability and the reasoning depth of the geometric large model in geometric reasoning and sensing tasks are optimized, and the calculation efficiency and the calculation accuracy are improved.

Description

TECHNICAL FIELD

[0001] The present disclosure belongs to the technical field of artificial intelligence, and particularly relates to a geometric large model training method, a geometric large model application method, a geometric large model training device, a geometric large model application device, a computer readable storage medium and an electronic device. BACKGROUND

[0002] A multi-modal large language model faces great challenges in processing problems related to images.

[0003] When the geometric visual perception ability of the multi-modal large language model is enhanced through an image reconstruction task or a perception question answering method, there are problems of limited topological structure reconstruction ability, insufficient semantic alignment generalization ability, and inability to effectively integrate the two as a whole, making it difficult to comprehensively improve the geometric visual perception ability of the model. SUMMARY

[0004] To overcome the problems in the related art, the present disclosure provides a geometric large model training method, a geometric large model application method, a geometric large model training device, a geometric large model application device, a computer readable storage medium and an electronic device.

[0005] According to a first aspect of an embodiment of the present disclosure, a geometric large model training method is provided, comprising: obtaining a geometric image, the geometric image being obtained by using a geometric form language to reconstruct; determining a geometric text according to the geometric image, and training a geometric large model using the geometric image and the geometric text to obtain the geometric large model satisfying a preset condition, the geometric text including a geometric question text and a geometric answer text corresponding to the geometric image, and the geometric large model being used for solving geometric problems.

[0006] Optionally, the training of the geometric large model using the geometric image and the geometric text to obtain the geometric large model satisfying the preset condition comprises: training the geometric large model using the geometric image, the geometric large model including a large language model, a multi-layer perceptron, and a visual encoder; continuing to train the large language model using the geometric image and the geometric text to obtain the geometric large model satisfying the preset condition.

[0007] Optionally, the obtaining of the geometric image comprises: determining a base image to be drawn and image parameters, and drawing the base image according to the image parameters; obtaining a geometric dictionary, and continuing to edit the base image according to the geometric dictionary to obtain the geometric image.

[0008] Optionally, the geometry dictionary comprises a vertex coordinate dictionary and a connectivity relationship dictionary. The editing the base image according to the geometry dictionary to obtain a geometry image comprises: annotating vertices on the base image according to the vertex coordinate dictionary; connecting the vertices on the base image according to the connectivity relationship dictionary to obtain a geometry image.

[0009] Optionally, the training the geometry large model according to the geometry image comprises: obtaining a prompt word, and inputting the geometry image and the prompt word into a to-be-trained geometry large model, so that the to-be-trained geometry large model outputs a prediction dictionary and a prediction parameter, the prediction dictionary comprising a prediction vertex dictionary and a prediction relationship dictionary; determining a dominant parameter according to the prediction dictionary, the prediction parameter, the geometry dictionary and the image parameter, and determining a first loss function according to the dominant parameter, so as to train the to-be-trained geometry large model by using the first loss function.

[0010] Optionally, the determining the dominant parameter according to the prediction dictionary, the prediction parameter, the geometry dictionary and the image parameter comprises: determining a target distance according to the prediction vertex dictionary and the vertex coordinate dictionary, and determining a first score according to the target distance; determining a second score according to the prediction relationship dictionary and the connectivity relationship dictionary; determining a target score according to the first score and the second score, and determining a dominant parameter according to the target score, the prediction dictionary, the prediction parameter, the geometry dictionary and the image parameter.

[0011] Optionally, the determining the geometry text according to the geometry image comprises: obtaining an image description text when reconstructing the geometry image, and determining a mapping relationship between the image description text and a geometry text; determining the geometry text according to the image description text based on the mapping relationship.

[0012] Optionally, after the determining the geometry text according to the image description text, the method further comprises: establishing a data set composed of the image description text and the geometry text, the data set comprising basic measurement data, relationship reasoning data and geometry analysis data, the geometry analysis data reflecting constraint relationships between geometry elements.

[0013] Optionally, the continuing training of the large language model by using the geometric image and the geometric text to obtain the geometric large model satisfying the preset condition comprises: inputting the geometric image and the geometric question text into the large language model, so that the large language model outputs a predicted answer text; determining a second loss function according to the predicted answer text and the geometric answer text, so that the large language model is trained by using the second loss function to obtain the geometric large model satisfying the preset condition.

[0014] Optionally, the method further comprises: obtaining an inference image and inference text corresponding to the inference image, the inference text comprising inference question text and inference answer text; inputting the inference image and the inference question text into the geometric large model satisfying the preset condition to obtain a predicted answer text, and determining an inference score according to the predicted answer text and the inference answer text; continuing training of the geometric large model according to the inference score.

[0015] According to a second aspect of the embodiments of the present disclosure, a geometric large model application method is provided, comprising: obtaining a to-be-searched image and a to-be-searched question, and searching the to-be-searched image and the to-be-searched question by using the geometric large model satisfying the preset condition to obtain a search answer and a corresponding inference thought chain, the to-be-searched image being a geometric image; and / or obtaining a to-be-searched question, and searching the to-be-searched question by using the geometric large model satisfying the preset condition to obtain a search answer and a corresponding inference thought chain.

[0016] According to a third aspect of the embodiments of the present disclosure, a geometric large model training apparatus is provided, comprising: an image reconstruction module configured to obtain a geometric image, the geometric image being obtained by reconstruction using a geometric form language; a model training module configured to determine a geometric text according to the geometric image, and train a geometric large model by using the geometric image and the geometric text to obtain the geometric large model satisfying the preset condition, the geometric text comprising geometric question text and geometric answer text corresponding to the geometric image, and the geometric large model being used for solving geometric problems.

[0017] Optionally, the model training module is configured to: train a geometric large model by using the geometric image, the geometric large model comprising a large language model, a multi-layer perceptron, and a visual encoder; Continue training the large language model using the geometric image and the geometric text to obtain the geometric large model satisfying a preset condition.

[0018] Optionally, the image reconstruction module is configured to: determine a base image to be drawn and image parameters, and draw the base image according to the image parameters; obtain a geometric dictionary, and continue to edit the base image according to the geometric dictionary to obtain a geometric image.

[0019] Optionally, the geometric dictionary includes a vertex coordinate dictionary and a connectivity relationship dictionary. The image reconstruction module is configured to: label vertices on the base image according to the vertex coordinate dictionary; connect the vertices on the base image according to the connectivity relationship dictionary to obtain a geometric image.

[0020] Optionally, the model training module is configured to: obtain a prompt word, and input the geometric image and the prompt word into a geometric large model to be trained, so that the geometric large model to be trained outputs a prediction dictionary and prediction parameters, the prediction dictionary including a prediction vertex dictionary and a prediction relationship dictionary; determine a dominant parameter according to the prediction dictionary, the prediction parameters, the geometric dictionary and the image parameters, and determine a first loss function using the dominant parameter, so as to train the geometric large model to be trained using the first loss function.

[0021] Optionally, the model training module is configured to: obtain an image description text when reconstructing a geometric image, and determine a mapping relationship between the image description text and a geometric text; determine the geometric text according to the image description text based on the mapping relationship.

[0022] According to a fourth aspect of the embodiments of the present disclosure, a geometric large model application device is provided, including: an image search module configured to obtain an image to be searched and a question to be searched, and search the image to be searched and the question to be searched using a geometric large model satisfying a preset condition to obtain a search answer and a corresponding reasoning thought chain, the image to be searched being a geometric image; and / or a text search module configured to obtain a question to be searched, and search the question to be searched using a geometric large model satisfying a preset condition to obtain a search answer and a corresponding reasoning thought chain.

[0023] According to a fifth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, which stores computer program instructions. The computer program instructions are executed by a processor to implement the steps of the geometric large model training method of any one of the first aspect of the present disclosure or the steps of the geometric large model application method of any one of the second aspect of the present disclosure.

[0024] According to a sixth aspect of the embodiments of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions to implement the steps of the geometric large model training method of any one of the first aspect of the present disclosure or the steps of the geometric large model application method of any one of the second aspect of the present disclosure.

[0025] The technical solutions provided by the embodiments of the present disclosure can include the following beneficial effects: In the method and device provided by the exemplary embodiments of the present disclosure, the geometric image is reconstructed in the form of geometric language, which reduces the number of characters and the reconstruction computing power required in the reconstruction process, solves the consistency problem of vertex annotation and topological structure of the geometric image, improves the efficiency and accuracy of image reconstruction, and determines the geometric text according to the geometric image, solves the matching problem of the reconstructed image and the combined text, improves the geometric visual perception ability, and provides support for the geometric large model to enhance the understanding of the relationship attributes and numerical attributes of the geometric image and improve the generalization performance of the geometric large model. On this basis, the geometric large model is trained using the geometric image and the geometric text, effectively integrating the topological relationship perception and geometric axiom alignment in two dimensions, so that the geometric large model exhibits stronger generalization ability and reasoning depth in geometric reasoning and perception tasks, improving the computing efficiency and accuracy of the geometric large model, and providing technical support and application solutions for complex geometric reasoning tasks.

[0026] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0027] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure.

[0028] Figure 1 The flowchart schematically shows a geometric large model training method in the exemplary embodiments of the present disclosure; Figure 2 The flowchart schematically shows a method for obtaining a geometric image in the exemplary embodiments of the present disclosure; Figure 3 Fig. 7 schematically illustrates a flowchart of a method for continuing to edit a base image in some embodiments of the present disclosure; Figure 4 Fig. 8 schematically illustrates a flowchart of a method for determining geometric text in some embodiments of the present disclosure; Figure 5 Fig. 9 schematically illustrates a flowchart of a method for training a geometric large model in some embodiments of the present disclosure; Figure 6 Fig. 10 schematically illustrates a flowchart of a method for training a geometric large model as a whole in some embodiments of the present disclosure; Figure 7 Fig. 11 schematically illustrates a flowchart of a method for determining advantage parameters in some embodiments of the present disclosure; Figure 8 Fig. 12 schematically illustrates a flowchart of a method for continuing to train a large language model in some embodiments of the present disclosure; Figure 9 Fig. 13 schematically illustrates a flowchart of a method for further training a geometric large model in some embodiments of the present disclosure; Figure 10 Fig. 14 schematically illustrates a flowchart of a method for applying a geometric large model in some embodiments of the present disclosure; Figure 11 Fig. 15 schematically illustrates a flowchart of a training method of a geometric large model in an application scenario in some embodiments of the present disclosure; Figure 12 Fig. 16 schematically illustrates a structure diagram of a geometric large model training device in some embodiments of the present disclosure; Figure 13 Fig. 17 schematically illustrates a structure diagram of a geometric large model application device in some embodiments of the present disclosure; Figure 14 Fig. 18 schematically illustrates a structure diagram of another geometric large model training device in some embodiments of the present disclosure; Figure 15 Fig. 19 schematically illustrates a structure diagram of still another geometric large model training device in some embodiments of the present disclosure. DETAILED DESCRIPTION

[0029] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The following description is only exemplary and is not intended to represent all aspects of the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0030] It is specified that all the actions of acquiring signals, information or data in this disclosure are carried out in compliance with the corresponding data protection regulations and policies of the country where the device is located, and with the authorization given by the corresponding device owner.

[0031] With the rapid development of artificial intelligence technology, Large Language Models (LLMs) have shown superior reasoning capabilities, especially when dealing with math problems in textual form. However, Multimodal Large Language Models (MLLMs) face greater challenges when dealing with problems involving images. These models not only need to interpret textual information, but also need to understand mathematical diagrams and identify details crucial to solving problems.

[0032] While MLLMs show significant effectiveness in general visual question answering, their training relies mainly on natural scene image datasets. When these models are applied to mathematical diagrams, this reliance creates a large domain gap, resulting in poor performance.

[0033] Geometry Problem Solving (GPS) is a challenging task that requires the integration of visual perception and symbolic reasoning, making it an effective benchmark for evaluating the reasoning capabilities of MLLMs. Current research attempts to improve the symbolic reasoning performance of LMMs by collecting or synthesizing geometry reasoning datasets. Symbolic reasoning training is effective, but it ignores the limitations of inaccurate visual perception on GPS capabilities.

[0034] Recent research on enhancing the geometric visual perception of LMMs can be broadly divided into two categories. The first category is the image reconstruction task, which emphasizes the topological visual features of geometric images. Pixel-based reconstruction methods prioritize the accurate positioning of pixel positions and color values, but often ignore the consistency of vertex annotations and topological structures. In contrast, code-based reconstruction methods better maintain the topological structure, but their high complexity requires the reconstruction of 2500 characters for each image.

[0035] The second category is the perception question answering method, which aims to achieve semantic alignment between geometric structures and mathematical axioms. Data synthesis methods can efficiently create large-scale perception question answering datasets. However, the generated images usually only contain simple geometric shapes, which leads to a gap in complexity compared to reasoning images and limits the generalization performance. On the contrary, using MLLMs for dataset expansion can guarantee image complexity, but the inherent hallucination problem in MLLMs often leads to mismatches between captions and images, ultimately reducing perception capabilities.

[0036] Therefore, the current methods for improving the geometric perception ability of MLLMs still have obvious deficiencies, mainly in the following three aspects: Limited topological structure reconstruction capability: Although the pixel-based reconstruction method can accurately locate the pixel position and color value, it ignores the consistency of vertex annotation and topological structure; while the encoding-based method better maintains the topological structure, but its high complexity significantly increases the computational cost, and a large number of characters need to be reconstructed for each image.

[0037] Insufficient semantic alignment generalization capability: The perception question answering method generates geometric question answering data sets through data synthesis, but the generated images are too simple, which has a gap with the complex images in the reasoning task, limiting the generalization performance of the model. Although MLLMs expand the data set can improve the image complexity, but the illusion problem leads to the mismatch between the subtitles and the images, which weakens the perception ability.

[0038] Insufficient method integration: The existing methods only focus on a single aspect of topological relationship perception or geometric axiom alignment, and fail to effectively combine the two from the whole, making it difficult to comprehensively improve the geometric visual perception ability of the model.

[0039] In summary, the existing geometric visual perception enhancement methods have dual limitations of topological structure perception and geometric axiom alignment.

[0040] In view of the problems in the related art, the present disclosure provides a geometric large model training method, Figure 1 is a flowchart of a geometric large model training method according to an exemplary embodiment, as Figure 1 shown, the method can at least include the following steps: Step S110. Obtain a geometric image, the geometric image is obtained by reconstructing a geometric form language.

[0041] Step S120. Determine a geometric text according to the geometric image, and train a geometric large model using the geometric image and the geometric text to obtain a geometric large model satisfying a preset condition, the geometric text includes a geometric question text and a geometric answer text corresponding to the geometric image, and the geometric large model is used for solving geometric problems.

[0042] In the example embodiments of the present disclosure, the consistency of vertex annotation and topology of the geometric image is solved by reconstructing the geometric image in the form of geometric language, the efficiency and accuracy of image reconstruction are improved, the number of characters and reconstruction computing power required in the reconstruction process are reduced, the geometric text is determined according to the geometric image, the matching degree of the reconstructed image and the combined text is solved, the geometric visual perception ability is improved, and the understanding of the geometric large model on the relationship attribute and the numerical attribute of the geometric image is enhanced, and the generalization performance of the geometric large model is improved. On this basis, the geometric large model is trained by using the geometric image and the geometric text, which effectively integrates the topological relationship perception and the geometric axiom alignment in two dimensions, so that the geometric large model exhibits stronger generalization ability and reasoning depth in geometric reasoning and perception tasks, improves the calculation efficiency and accuracy of the geometric large model, and provides technical support and application scheme for complex geometric reasoning tasks.

[0043] The steps of the geometric large model training method will be described in detail below.

[0044] In step S110, a geometric image is obtained, which is reconstructed by using a geometric form language.

[0045] In the example embodiments of the present disclosure, in order to improve the understanding of the model on the topology by training a multi-modal large model for image reconstruction, the geometric image reconstructed can be obtained first.

[0046] In the optional embodiments, Figure 2 A flowchart of a method for obtaining a geometric image is shown, as shown in Figure 2 The method can at least include the following steps: in step S210, a base image to be drawn and image parameters are determined, and the base image is drawn according to the image parameters.

[0047] The base image can be a regular image, such as a circle, a triangle, etc., which is not specially limited in the example embodiments. The image parameter can be a circle radius variable. It should be noted that when the base image is a circle, the image parameter can be the corresponding radius; when the base image is a non-circular shape, the circle radius variable can be set to 0.

[0048] In step S220, a geometric dictionary is obtained, and the base image is further edited according to the geometric dictionary to obtain a geometric image.

[0049] In the optional embodiments, the geometric dictionary includes: a vertex coordinate dictionary and a connected relationship dictionary; Figure 3 A flowchart of a method for further editing a base image is shown, as shown in Figure 3As shown, the method can at least include the following steps: in step S310, annotating the vertices on the base image according to the vertex coordinate dictionary.

[0050] Since the structure information of the geometric body can be decoupled into the set of vertices and edges, a highly decoupled geometric form language VertexLang (vertex-based geometric form language) for image reconstruction is developed.

[0051] VertexLang consists of 3 core components: a circle radius variable, a vertex coordinate dictionary, and a connected relationship dictionary. In the vertex coordinate dictionary, the vertex annotation is used as the key, and the corresponding vertex coordinates (rounded to two decimal places) are stored as numerical values. The vertex annotation is information such as vertex A, A and B in vertex B. Further, all coordinates can be normalized to emphasize the relative position relationship between vertices rather than the specific edge length.

[0052] Therefore, after obtaining the vertex coordinate dictionary, all vertices can be annotated one by one on the image according to the vertices recorded in the vertex coordinate.

[0053] In step S320, the vertices are connected on the base image according to the connected relationship dictionary to obtain a geometric image.

[0054] In the connected relationship dictionary, the vertex annotation is used as the key, and the list of vertices connected by a straight line is used as the value.

[0055] Therefore, after obtaining the connected relationship dictionary, the annotated vertices can be connected according to the straight line connection relationship recorded in the connected relationship dictionary to use the base image as a geometric image together.

[0056] In step S120, a geometric text is determined according to the geometric image, and a geometric large model is trained using the geometric image and the geometric text to obtain a geometric large model satisfying a preset condition, the geometric text includes a geometric question text and a geometric answer text corresponding to the geometric image, and the geometric large model is used for solving geometric problems.

[0057] In an exemplary embodiment of the present disclosure, the geometric text can be further determined to enter the training of the geometric large model.

[0058] In an optional embodiment, Figure 4 A flowchart of a method for determining a geometric text is shown as Figure 4 As shown, the method can at least include the following steps: in step S410, obtaining an image description text when reconstructing a geometric image, and determining a mapping relationship between the image description text and the geometric text.

[0059] When reconstructing a geometric image, some editing tools can be used to obtain an image description text of the geometric image.

[0060] Since the mapping relationship between the image description text and the geometric text is established in advance during the development stage, the mapping relationship can be obtained as the basis for determining the geometric text. The geometric text can include two parts of geometric question text and geometric answer text, and the geometric answer text includes incorrect answer text and correct answer text.

[0061] In addition, the geometric question text can be classified into two types of relationship question text and numerical question text. The relationship question text includes relative spatial positioning, etc., and the numerical question text includes relative line segment length, angle, and area, etc.

[0062] In step S420, the geometric text is determined according to the image description text based on the mapping relationship.

[0063] After determining the mapping relationship between the image description text and the geometric text, the corresponding geometric text can be determined according to the image description text and the mapping relationship.

[0064] In an optional embodiment, a data set composed of image description text and geometric text is established, the data set includes basic measurement data, relationship reasoning data, and geometric analysis data, and the geometric analysis data reflects the constraint relationship between geometric elements.

[0065] The basic measurement data focuses on estimating and comparing geometric quantities, emphasizing the perception of numerical values and the understanding of basic geometric concepts; the relationship reasoning data focuses on identifying the structural relationship between geometric elements, and these tasks emphasize the ability of the model to identify and explain the relationship between geometric objects; the geometric analysis data involves the combination constraints and logical dependencies between elements, and these tasks require the model to perform advanced geometric analysis based on evidence reasoning similar to humans, which is the core dependency in geometric reasoning. The geometric analysis data is an important part of the established data set.

[0066] Further, the way of training the geometric large model using the geometric image and the geometric text can include two rounds of training. The first round of training is to train the entire geometric large model using the geometric image, and the second round of training can be to continue training the large language model in the geometric large model using the geometric image and the geometric text.

[0067] In an optional embodiment, Figure 5 The flowchart of the method for training the geometric large model is shown as Figure 5 The method can at least include the following steps: in step S510, the geometric large model is trained using the geometric image, and the geometric large model includes a large language model, a multilayer perceptron, and a visual encoder.

[0068] In an optional embodiment, Figure 6A flowchart of a method of training a whole geometric large model is shown in FIG. 6. Figure 6 As shown in FIG. 6, the method can include the following steps: in step S610, a prompt is obtained, and the geometric image and the prompt are input into the geometric large model to be trained, so that the geometric large model to be trained outputs a predicted dictionary and a predicted parameter, the predicted dictionary including a predicted vertex dictionary and a predicted relationship dictionary.

[0069] The prompt can be "please rebuild the geometric image" or the like, or other prompts can be set according to actual conditions, which are not specially limited in the present exemplary embodiment.

[0070] Further, the geometric image and the prompt are input into the geometric large model to be trained for training, and the content output by the geometric large model to be trained includes the predicted dictionary and the predicted parameter, which is the same as the content of the graph parameter and can be a predicted circle radius variable.

[0071] In step S620, the advantage parameter is determined according to the predicted dictionary, the predicted parameter, the geometric dictionary and the image parameter, and the first loss function is determined using the advantage parameter, so that the geometric large model to be trained is trained using the first loss function.

[0072] In an optional embodiment, Figure 7 A flowchart of a method of determining an advantage parameter is shown in FIG. 7. Figure 7 As shown in FIG. 7, the method can include the following steps: in step S710, a target distance is determined according to the predicted vertex dictionary and the vertex coordinate dictionary, and a first score is determined according to the target distance.

[0073] After the geometric large model outputs the predicted vertex dictionary, the vertex coordinate accuracy can be evaluated according to the Euclidean distance between the predicted vertex dictionary and the vertex coordinate dictionary, i.e., the target distance. Specifically, the first score can be determined according to formula (1) based on the target distance and two predefined thresholds and (1) It is worth noting that the final vertex coordinate accuracy first score can be set as the average of all vertex scores, or other determination methods can be set according to actual conditions, which are not specially limited in the present exemplary embodiment.

[0074] In step S720, a second score is determined according to the predicted relationship dictionary and the connectivity relationship dictionary.

[0075] After the geometric large model outputs the predicted relationship dictionary, the connectivity relationship accuracy can be measured by comparing the F1 score of the predicted relationship dictionary with the connectivity relationship dictionary, and the F1 score is the second score.​

[0076] In step S730, a target score is determined according to the first score and the second score, and a dominant parameter is determined according to the target score, a prediction dictionary, a prediction parameter, a geometry dictionary and an image parameter.

[0077] After the first score and the second score are determined, the first score and the second score can be weighted and summed to determine the target score. The weights of the first score and the second score can be set to 0.5, or other weight values can be set according to actual conditions or other ways to determine the target score, which is not specially limited in this example embodiment.

[0078] When the dominant parameter is determined according to the target score, the prediction dictionary, the prediction parameter, the geometry dictionary and the image parameter, it is considered that the Group Relative Policy Optimization (GRPO) reinforcement learning algorithm has less influence on the distribution of the model. However, since the initial performance of the model in image reconstruction is weak, it is difficult for GRPO to effectively identify the optimization direction. In order to improve the stability of training and accelerate early convergence, DynamicGT-GRPO (Dynamic Ground Truth Guided Group Relative Policy Optimization Algorithm) is proposed, which can dynamically replace the model generated strategy with the ground truth to guide the model to develop towards the correct optimization direction. Specifically, the dominant parameter can be determined in the manner of formula (2): (2) wherein, is the dominant parameter, is the target score, is the prediction dictionary and the prediction parameter, is the geometry dictionary and the image parameter, is a random number between 0 and 1, represents the probability of replacement with the ground truth. In other words, is a single strategy, represents a group of strategies.

[0079] When the model cannot independently generate a correct solution, the ground truth strategy provides greater advantage, effectively guiding the optimization direction of the model. On the contrary, as the model learns to generate correct solutions, the advantage of the ground truth strategy decreases, allowing the model to shift its focus back to its own strategy output without being excessively affected by the ground truth.

[0080] Therefore, after the dominant parameter is determined, the dominant parameter can be brought into the formula of GRPO to determine a first loss function in the case of maximizing the target function, and the geometric large model to be trained is trained by using the first loss function.

[0081] Since the geometric large model comprises not only a large language model but also a multi-layer perceptron and a visual encoder, the parameter training of the multi-layer perceptron and the visual encoder can be determined to be completed at this time.

[0082] In step S520, the large language model is continuously trained using the geometric image and the geometric text to obtain a geometric large model satisfying a preset condition.

[0083] After the overall training of the geometric large model using the geometric image, the next step of training can be entered.

[0084] In an optional embodiment, Figure 8 A flowchart of a method for continuously training a large language model is shown, as shown in Figure 8 The method can at least include the following steps: in step S810, inputting a geometric image and geometric question text into a large language model to make the large language model output a predicted answer text.

[0085] After the multi-layer perceptron and the visual encoder in the geometric large model are trained using the geometric image, the large language model in the geometric large model can be further trained using the geometric image and the geometric text.

[0086] Specifically, the geometric image and the geometric answer text can be input into the large language model, and the large language model can output a predicted answer text.

[0087] In step S820, a second loss function is determined according to the predicted answer text and the geometric answer text, and the large language model is trained using the second loss function to obtain a geometric large model satisfying a preset condition.

[0088] Considering the diversity of the format of the predicted answer text, a Direct Preference Optimization (DPO) algorithm can be used for training to make the predicted answer text and the geometric answer text determine a second loss function to be minimum, so that the training of the large language model is completed through the second loss function to obtain a geometric large model satisfying a preset condition. The preset condition satisfied by the geometric large model can be that the second loss function reaches a minimum. The size of the loss function can reflect the pros and cons of the performance of the model, and the goal of the training process is to continuously reduce the value of the loss function, so that the loss function can be minimized by adjusting the model parameters.

[0089] After two rounds of training of the geometric large model, the geometric large model satisfying the preset condition can be put into the training of the geometric large model.

[0090] Furthermore, the geometric large model satisfying the preset condition can be continuously trained using a geometric reasoning dataset to further improve the geometric reasoning capability of the geometric large model.

[0091] In optional embodiments, Figure 9 A flowchart illustrating a method of further training a geometric large model is shown as Figure 9 As shown, the method can at least include the following steps: in step S910, an inference image and an inference text corresponding to the inference image are obtained, the inference text including an inference question text and an inference answer text.

[0092] The inference image and the inference text are used to represent a geometric problem that requires certain calculations and inferences to solve the answer.

[0093] In step S920, the inference image and the inference question text are input into a geometric large model that meets the preset conditions to obtain a predicted answer text, and a reasoning score is determined according to the predicted answer text and the inference answer text.

[0094] The inference image and the inference question text are input into a geometric large model that meets the preset conditions, which can output a predicted answer text.

[0095] Further, a reward mechanism is set for the predicted answer text of the inference, which rewards the inference accuracy. When the predicted answer text is consistent with the inference answer text, the reasoning score is 1; when the predicted answer text is inconsistent with the inference answer text, the reasoning score is 0.

[0096] In step S930, the geometric large model is further trained according to the reasoning score.

[0097] After determining the reasoning score according to the predicted answer text and the inference answer text, the reasoning score can be used to continue fine-tuning the geometric large model that meets the preset conditions to further improve the symbolic reasoning ability of the geometric large model.

[0098] The present disclosure also provides a geometric large model application method, Figure 10 A flowchart illustrating a geometric large model application method according to an exemplary embodiment is shown as Figure 10 As shown, the method can at least include the following steps: in step S1010, an image to be searched and a question to be searched are obtained, and a geometric large model that meets the preset conditions is used to search the image to be searched and the question to be searched to obtain a search answer and a corresponding reasoning thought chain, the image to be searched being a geometric image.

[0099] When searching using a geometric large model that meets the preset conditions, the input content can include the image to be searched and the question to be searched, so that the geometric large model outputs a search answer and a corresponding reasoning thought chain. The reasoning thought chain is used to represent the reasoning process and basis of the search answer.

[0100] The to-be-searched image and the to-be-searched question can be obtained by photographing using a terminal device such as a mobile phone, or can be obtained by other manners, and the exemplary embodiment does not specially limit this. When the geometric large model training method is applied to a terminal device or a server such as a cloud, after the to-be-searched image and the to-be-searched question are obtained by photographing, the geometric large model deployed on the terminal device or the server such as the cloud can be called to search to obtain a search answer determined by an inference thought chain.

[0101] In step S1020, the to-be-searched question is obtained, and a search answer and a corresponding inference thought chain are obtained by searching the to-be-searched question using the geometric large model satisfying the preset condition.

[0102] When searching using the geometric large model satisfying the preset condition, the input content can include the to-be-searched question, so that the geometric large model outputs a search answer and a corresponding inference thought chain. The inference thought chain is used to represent the reasoning process and basis of the search answer.

[0103] The to-be-searched question can be obtained by photographing using a terminal device such as a mobile phone, or can be obtained by other manners, and the exemplary embodiment does not specially limit this. When the geometric large model training method is applied to a terminal device or a server such as a cloud, after the to-be-searched question is obtained by photographing, the geometric large model deployed on the terminal device or the server such as the cloud can be called to search to obtain a search answer determined by an inference thought chain.

[0104] The geometric large model training method in the embodiment of the disclosure will be described in detail below in combination with an application scenario.

[0105] To solve the problems of limited topology reconstruction ability, insufficient semantic alignment generalization ability, and insufficient method integration of the method for improving the geometric perception ability of MLLMs, a multi-modal perception method GeoRPR (Geometry Reconstruction Perception Reasoning) based on double-dimension progressive enhancement is proposed. The method can effectively improve the geometric visual perception ability of LMMs without significantly increasing the computational complexity. Specifically, it includes the following three core stages: Simple image reconstruction paradigm based on VertexLang: The method proposes an image reconstruction method based on a carefully designed simple form language VertexLang in the first stage. By simplifying the geometric image reconstruction to a task that requires only 0.3k characters on average, the reconstruction complexity is significantly reduced, and the model's perception ability of topology is enhanced, laying a solid foundation for subsequent geometric learning.

[0106] Geo Vision Percepter perception engine: In the second stage, the Geo Vision Percepter engine is designed and introduced to generate complex perception question and answer data with 100% accuracy. These data are trained by combining topological relationships with geometric axioms to improve the ability of the model from topological perception to geometric semantic alignment.

[0107] Global optimization based on high-quality reasoning data: In the third stage, the model is trained using 16.5k high-quality reasoning data to further strengthen the model's understanding of topological relationships and geometric axioms, and to achieve a comprehensive conversion from attribute perception to GPS capability.

[0108] Through the synergistic effect of the above three stages, GeoRPR significantly improves the geometric visual perception ability of LMMs while considering the computational efficiency and generalization performance of the model, providing stronger technical support and application potential for complex geometric reasoning tasks.

[0109] To overcome the limitations of multi-modal large models in solving geometric problems due to insufficient perception ability, the GeoRPR framework is proposed, which significantly enhances the ability to solve geometric problems through two core perception enhancement components: topological structure perception and structure axiom alignment.

[0110] Prior to this, multi-modal large models were trained for image reconstruction to improve the model's understanding of topological structures. To simplify the reconstruction process and emphasize topological structures, a new reconstruction method based on the formal language VertexLang is introduced. In addition, the Dynamic GT-GRPO training algorithm is proposed to help the model efficiently learn reconstruction skills while maintaining its original reasoning functions.

[0111] The structural information of geometric bodies can be decoupled into the concepts of vertex and edge sets, and a highly decoupled geometric formal language VertexLang for image reconstruction is developed. VertexLang consists of three core components: a circle radius variable, a vertex coordinate dictionary, and a connectivity relationship dictionary.

[0112] If there is a circle in the geometric figure, the radius variable stores the radius of the circle; otherwise, its value is 0. In the vertex coordinate dictionary, the vertex annotation serves as the key, and the corresponding vertex coordinates (rounded to two decimal places) are stored as numerical values. All coordinates are normalized to emphasize the relative positional relationships between vertices rather than specific edge lengths. In the connectivity relationship dictionary, the vertex annotation is used as the key, and the list of vertices connected by straight lines is used as the value.

[0113] In the image rendering phase, three variables are input into a predefined image rendering function to generate a unique geometric image. Such a design not only ensures the consistency of geometric topology, but also improves the efficiency of geometric reconstruction. In the analysis of 2500 samples, the number of comment characters reconstructed using VertexLang is reduced by 88% compared to code-based reconstruction. In the training and use of multi-modal large models, the more characters, the longer the time-consuming, and the more computing power required, so this geometric formal language can well improve the efficiency of geometric solving and reduce the solving computing power.

[0114] In order to construct high-quality training data suitable for VertexLang image reconstruction, a GeoVisionReconstructor (reconstruction data generation engine based on Vertex symbolic language) data generation engine is designed. In this engine, a method of base selection combined with special line addition is used to generate images.

[0115] In the base selection process, it is first determined whether a circular base is included to establish a radius value. In addition, the base can be at least 20 pre-set regular images, and at least one of them can be randomly selected as the base image when generating the base image.

[0116] Since geometric elements are added in order, their vertex coordinates and connection relationships are stored according to the syntax requirements of VertexLang, and are managed in the vertex coordinate storage module and the connection relationship storage module, respectively, to form a vertex coordinate dictionary and a connection relationship dictionary. Finally, the radius and the two dictionaries are input into the code rendering module, which uses a pre-designed algorithm to produce a unique geometric image. The pre-designed algorithm is the image rendering function, which can realize the geometric image reconstruction as shown in Figure 2 .

[0117] After the enhanced model perceives the geometric topology, perception question and answer training is performed to link topology perception with geometric axioms.

[0118] In order to increase the complexity of perception question and answer data while ensuring accuracy, GeoVisionPerceptor (geometric vision perception data generation engine) is proposed, which is a novel framework designed to generate geometric perception data pairs that closely match geometric axioms.

[0119] Specifically, the corresponding image description and geometric properties are obtained step by step in the image generation process. Then they are processed by the description conversion module and the attribute conversion module respectively, and the carefully designed conversion templates rooted in geometric axioms are used to generate question and answer pairs. The description conversion module generates relational questions, such as relative spatial positioning, to ensure consistency with axioms related to association and order. The attribute conversion module focuses on numerical problems, such as relative line segment length, angle and area, to ensure consistency with axioms that govern congruence and proportion. GeoVision Perceptor enhances the model's understanding of the relational and numerical properties of geometric figures, thereby bridging the gap between perception and reasoning.

[0120] Further, the GeoPercept (axiom-based geometric perception dataset) dataset is constructed by GeoVision Perceptor, which contains 80k geometric perception question and answer pairs. Generally, GeoPercept contains 13 types of questions, which are divided into 3 levels according to the requirements of geometric cognition, namely basic measurement level, relational reasoning level and high-level verification level.

[0121] Among them, the basic measurement category includes AngleValue (angle value measurement), LengthCompare (length comparison), AreaCompare (area comparison) and other tasks, focusing on estimating and comparing geometric quantities. These tasks emphasize the perception of numerical values and the understanding of basic geometric concepts.

[0122] The relational reasoning category includes ParallelCheck (parallel relationship judgment), BiectorCheck (angle bisector judgment) and PerpCheck (perpendicular line judgment) and other tasks, which focus on identifying the structural relationship between geometric elements and emphasize the model's ability to identify and explain the relationship between geometric objects.

[0123] Advanced verification is unique to GeoPercept. This category includes Collinear (collinear judgment), PerpFootCheck (perpendicular foot judgment), OnCircle (point circle position relationship judgment) and other tasks, involving combined constraints and logical dependencies between elements. These tasks require the model to perform advanced geometric analysis similar to human evidence-based reasoning, which is a core dependency in geometric reasoning.

[0124] Figure 11 The flowchart of the training method of the geometric large model in the application scenario is shown as follows. Figure 11 The first stage reflects the training of geometric topological structure perception enhancement.

[0125] Considering that human brain perceives spatial relationship as a basic ability to solve geometric problems, the first stage focuses on improving the model's ability to perceive topology. Specifically, VertexLang image reconstruction data is used for training, and the DynamicGT-GRPO method is introduced to improve the stability at the beginning of training.

[0126] During the training process, the gap between the VertexLang image reconstruction task and the pre-training task of the model poses a challenge. Applying supervised fine-tuning (SFT) for knowledge injection will destroy the distribution of the model, leading to a decline in inference ability. To alleviate this problem, a group relative policy optimization reinforcement learning algorithm is explored, which has less impact on the distribution of the model. However, due to the weak initial performance of the model in image reconstruction, GRPO has difficulty in effectively identifying the optimization direction. To improve the stability of training and accelerate early convergence, DynamicGT-GRPO is proposed, which dynamically replaces the model-generated strategy with ground truth (GT) to guide the model to develop in the correct optimization direction. GT will affect advantage calculation in the manner shown in equation (2).

[0127] A ReCon (reconstruction) reward is carefully designed for the VertexLang image reconstruction task, including two components: vertex coordinate accuracy and connected relationship accuracy. Vertex coordinate accuracy is evaluated by calculating the Euclidean distance between the model's output vertex coordinates and the corresponding GT coordinates. Based on the distance and a pre-defined threshold and , the scoring rule for each vertex is defined as shown in equation (1). Wherein in equation (2) corresponds to in equation (1).

[0128] The final vertex coordinate accuracy score is the average of all vertex scores. The accuracy of connected relationship is measured by comparing the model's output edge set and the true edge set using F1 score. The final reward score is determined by the vertex coordinate accuracy score and the connected relationship accuracy score, such as weighted sum or other methods.

[0129] The second stage reflects the structure axiom alignment training.

[0130] The second stage aims to link topological awareness with geometric axioms. Considering the diversity of answer formats, a direct preference optimization algorithm is employed for training. DPO does not strictly limit the content of the model's response; instead, the DPO algorithm optimizes the model as long as the output gradually approaches the correct answer and deviates from incorrect answers. This sparse supervision avoids excessive focus on non-unique labeled expressions, effectively enhancing the model's understanding of the relationship between structure and axioms. Strong generalization performance on real data provides a solid knowledge base for subsequent symbolic reasoning.

[0131] The third stage reflects the fusion of perception and reasoning training.

[0132] After enhancing the perception ability in two dimensions, the focus shifts to improving the symbolic reasoning ability of the model. Due to the lack of high-quality geometric reasoning datasets, the GRPO algorithm is used to optimize the model. GRPO improves data utilization efficiency by promoting deeper reasoning processes. With only 16.5k data samples, the model's symbolic reasoning ability is greatly improved, and its performance in solving mathematical reasoning and graph reasoning tasks is also significantly improved.

[0133] In the application of the geometric large model, the use of the trained model is consistent with the base model.

[0134] The GeoRPR method can be widely applied to the training and optimization of multi-modal large language models in geometric reasoning and cross-modal perception tasks, and is particularly suitable for the following typical application scenarios: Geometric reasoning and spatial cognition field: After training with the GeoRPR method, multi-modal large language models can have stronger geometric reasoning and spatial cognition capabilities. Facing complex problems containing geometric images and text descriptions, the model can complete precise spatial relationship analysis and multi-step logical reasoning based on topological structure, geometric axioms, and language instructions, and finally generate answers or conclusions consistent with geometric logic. This capability can be widely applied in intelligent education, such as geometric teaching aids, virtual reality scene construction, intelligent design assistance, etc., and has high practical value and technical potential.

[0135] Graph reconstruction and design optimization field: Models trained with the GeoRPR method perform better in graph reconstruction and design optimization. For geometric graph inputs containing complex topological structures, the model can quickly identify vertex relationships and structural features, generate high-quality reconstruction results based on minimal form language, and support optimization and improvement of design schemes. This technology can be applied to industrial design assistance, building structure analysis, automated graph generation, etc., to improve design efficiency and intelligence level.

[0136] Through the step-by-step enhanced training of GeoRPR, the multi-modal large language model exhibits stronger generalization capability and deeper reasoning in geometric reasoning and perception tasks, providing innovative technical support and application solutions for solving complex geometric problems.

[0137] In the application process, the geometric large model training method can exhibit higher image reconstruction accuracy and significantly improve reasoning performance, and has certain degree of advancement on multiple mainstream benchmarks, and also brings superior out-of-domain generalization performance.

[0138] In the example embodiments of the present disclosure, the geometric image is reconstructed by means of geometric form language, reducing the number of characters and reconstruction computing power required in the reconstruction process, solving the vertex annotation and topological structure consistency problem of the geometric image, improving the efficiency and accuracy of image reconstruction, and determining the geometric text according to the geometric image, solving the matching problem of the reconstructed image and the combined text, improving the geometric visual perception ability, providing support for the geometric large model to enhance the understanding of the relationship attributes and numerical attributes of the geometric image and improve the generalization performance of the geometric large model. On this basis, the geometric large model is trained using the geometric image and the geometric text, effectively integrating the topological relationship perception and geometric axiom alignment dimensions, so that the geometric large model exhibits stronger generalization capability and deeper reasoning in geometric reasoning and perception tasks, improving the computational efficiency and accuracy of the geometric large model, and providing technical support and application solutions for complex geometric reasoning tasks.

[0139] In addition, in the example embodiments of the present disclosure, a geometric large model training device is also provided, Figure 12 The structure diagram of the geometric large model training device is shown as Figure 12 As shown, the geometric large model training device 1200 can include an image reconstruction module 1210 and a model training module 1220. Wherein: The image reconstruction module 1210 is configured to obtain a geometric image, the geometric image being reconstructed using geometric form language; The model training module 1220 is configured to determine a geometric text according to the geometric image, and train a geometric large model using the geometric image and the geometric text to obtain the geometric large model satisfying a preset condition, the geometric text including a geometric problem text and a geometric answer text corresponding to the geometric image, and the geometric large model being used for solving geometric problems.

[0140] In some embodiments of the present disclosure, the model training module 1220 is configured to: Train a geometric large model using the geometric image, the geometric large model including a large language model, a multi-layer perceptron, and a visual encoder; Continue training the large language model using the geometric image and the geometric text to obtain the geometric large model satisfying a preset condition.

[0141] In some embodiments of the present disclosure, the image reconstruction module 1210 is configured to: determine a base image to be drawn and image parameters, and draw the base image according to the image parameters; obtain a geometric dictionary, and continue to edit the base image according to the geometric dictionary to obtain a geometric image.

[0142] In some embodiments of the present disclosure, the geometric dictionary includes a vertex coordinate dictionary and a connectivity relationship dictionary. The image reconstruction module 1210 is configured to: label vertices on the base image according to the vertex coordinate dictionary; connect the vertices on the base image according to the connectivity relationship dictionary to obtain a geometric image.

[0143] In some embodiments of the present disclosure, the model training module 1220 is configured to: obtain a prompt word, and input the geometric image and the prompt word into a geometric large model to be trained, so that the geometric large model to be trained outputs a prediction dictionary and a prediction parameter, the prediction dictionary including a prediction vertex dictionary and a prediction relationship dictionary; determine a dominant parameter according to the prediction dictionary, the prediction parameter, the geometric dictionary and the image parameter, and determine a first loss function using the dominant parameter, so as to train the geometric large model to be trained using the first loss function.

[0144] In some embodiments of the present disclosure, the model training module 1220 is configured to: determine a target distance according to the prediction vertex dictionary and the vertex coordinate dictionary, and determine a first score according to the target distance; determine a second score according to the prediction relationship dictionary and the connectivity relationship dictionary; determine a target score according to the first score and the second score, and determine a dominant parameter according to the target score, the prediction dictionary, the prediction parameter, the geometric dictionary and the image parameter.

[0145] In some embodiments of the present disclosure, the model training module 1220 is configured to: obtain an image description text when reconstructing a geometric image, and determine a mapping relationship between the image description text and a geometric text; determine the geometric text according to the image description text based on the mapping relationship.

[0146] In some embodiments of the present disclosure, the geometric large model training apparatus 1200 is further configured to: establish a data set composed of the image description text and the geometric text, the data set including basic measurement data, relationship reasoning data, and geometric analysis data reflecting constraint relationships between geometric elements.

[0147] In some embodiments of the present disclosure, the model training module 1220 is configured to: input the geometric image and the geometric question text into the large language model to make the large language model output a predicted answer text; determine a second loss function according to the predicted answer text and the geometric answer text, to train the large language model to obtain the geometric large model satisfying a preset condition by using the second loss function.

[0148] In some embodiments of the present disclosure, the geometric large model training apparatus 1200 is further configured to: obtain an inference image and an inference text corresponding to the inference image, the inference text including an inference question text and an inference answer text; input the inference image and the inference question text into the geometric large model satisfying a preset condition to obtain a predicted answer text, and determine an inference score according to the predicted answer text and the inference answer text; continue to train the geometric large model according to the inference score.

[0149] In some embodiments of the present disclosure, a geometric large model application apparatus is further provided, Figure 13 a structural schematic diagram of the geometric large model application apparatus is shown, as shown in Figure 13 the geometric large model application apparatus 1300 can include: an image search module 1310 configured to obtain a to-be-searched image and a to-be-searched question, and search the to-be-searched image and the to-be-searched question by using the geometric large model satisfying a preset condition to obtain a search answer and a corresponding inference thought chain; and / or a text search module 1320 configured to obtain a to-be-searched question, and search the to-be-searched question by using the geometric large model satisfying a preset condition to obtain a search answer and a corresponding inference thought chain.

[0150] As to the apparatus in the above-mentioned embodiments, the specific manners in which various modules perform operations have been described in details in the embodiments of the method, and will not be described in details here.

[0151] The present disclosure also provides a computer readable storage medium, having stored thereon computer program instructions, which when executed by a processor implement the steps of the geometric large model training method provided by the present disclosure.

[0152] Figure 14 is another block diagram of a geometric large model training apparatus 1400 according to an exemplary embodiment. For example, the apparatus 1400 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like.

[0153] Referring to Figure 14 The apparatus 1400 can include one or more of the following components: a processing component 1402, a memory 1404, a power supply component 1406, a multimedia component 1408, an audio component 1410, an input / output (I / O) interface 1412, a sensor component 1414, and a communication component 1416.

[0154] The processing component 1402 usually controls overall operations of the apparatus 1400, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1402 can include one or more processors 1420 to execute instructions to complete all or part of steps of the above methods. Further, the processing component 1402 can include one or more modules to facilitate interaction between the processing component 1402 and other components. For example, the processing component 1402 can include a multimedia module to facilitate the interaction between the multimedia component 1408 and the processing component 1402.

[0155] The memory 1404 is configured to store various types of data to support operations of the apparatus 1400. Examples of these data include instructions for any application or methods operating on the apparatus 1400, contact data, phonebook data, messages, pictures, videos, and the like. The memory 1404 can be realized by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0156] The power supply component 1406 supplies electrical power for various components of the apparatus 1400. The power supply component 1406 can include a power supply management system, one or more power supplies, and other components associated with generating, managing and distributing power for the apparatus 1400.

[0157] The multimedia component 1408 includes a display for the device 1400 to display output to a user. In some embodiments, the display is touch screen. If the display is a touch screen, the display can be implemented as a touch screen to receive input from the user. The touch screen includes one or more touch sensors to sense touch, swipe, and gestures on the touch screen. The touch sensors can sense not only the boundary of a touch or swipe action, but also can sense duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 1408 includes a front-facing camera and / or a rear-facing camera. When the device 1400 is in operation, the front-facing camera and / or the rear-facing camera can be used to capture still and / or moving pictures.

[0158] The audio component 1410 is configured to output and / or input audio signals. For example, the audio component 1410 includes a microphone (MIC) that is configured to receive an external audio signal when the device 1400 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 1404 or transmitted via the communication component 1416. In some embodiments, the audio component 1410 also includes a speaker for outputting audio signals.

[0159] The input / output interface 1412 provides an interface between the processing component 1402 and peripheral interface modules, which can include a keypad, click wheel, buttons, and so on. These buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0160] The sensor component 1414 includes one or more sensors for providing status assessments of various aspects of the device 1400. For example, the sensor component 1414 can detect an open / closed position of the device 1400, relative positioning of components, such as a display and a keypad of the device 1400, a change of position of the device 1400 or a component of the device 1400, presence or absence of user contact with the device 1400, an orientation or acceleration / deceleration of the device 1400, and a temperature change of the device 1400. The sensor component 1414 can include a proximity sensor configured to detect presence of a nearby object without any physical touch. The sensor component 1414 can further include a light sensor (e.g., a CMOS or CCD image sensor) configured to function as an image capture device in a camera application. In some embodiments, the sensor component 1414 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0161] The communication component 1416 is configured to facilitate wired or wireless communication between the device 1400 and other devices. The device 1400 can access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1416 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1416 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) techniques, infrared data association (IrDA) techniques, ultra-wideband (UWB) techniques, Bluetooth (BT) techniques, and other techniques.

[0162] In an exemplary embodiment, the device 1400 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements, for performing the above-described methods.

[0163] In an exemplary embodiment, a non-transitory computer readable storage medium including instructions, such as the memory 1404 including instructions, is also provided, which can be executed by the processor 1420 of the device 1400 to complete the above-described methods. For example, the non-transitory computer readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc.

[0164] The apparatus described above can be an independent electronic device, but also can be a part of an independent electronic device, for example, in an embodiment, the apparatus can be an integrated circuit (IC) or a chip, wherein the integrated circuit can be an IC or a collection of multiple ICs; the chip can include but is not limited to the following types: GPU (Graphics Processing Unit), CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), SOC (System on Chip, SoC, System on Chip or System on Chip), etc. The integrated circuit or chip described above can be used to execute executable instructions (or code) to implement the geometric large model training method described above. Among them, the executable instructions can be stored in the integrated circuit or chip, or obtained from other devices or equipment, for example, the integrated circuit or chip includes a processor, a memory, and an interface for communicating with other devices. The executable instructions can be stored in the memory, and when the executable instructions are executed by the processor, the geometric large model training method described above is implemented; or the integrated circuit or chip can receive executable instructions through the interface and transmit them to the processor for execution to implement the above method.

[0165] In another exemplary embodiment, a computer program product is also provided, which includes a computer program capable of being executed by a programmable device, and the computer program has a code portion for executing the geometric large model training method described above when executed by the programmable device.

[0166] Figure 15 is another block diagram of a geometric large model training apparatus 1500 according to an exemplary embodiment. For example, the apparatus 1500 can be provided as a server. Referring to Figure 15 , the apparatus 1500 includes a processing component 1522, which further includes one or more processors, and a memory resource represented by a memory 1532, for storing instructions executable by the processing component 1522, such as an application program. The application program stored in the memory 1532 can include one or more than one module each corresponding to a set of instructions. In addition, the processing component 1522 is configured to execute instructions to perform the geometric large model training method described above.

[0167] The apparatus 1500 can also include a power supply component 1526 configured to supply power to the apparatus 1500, a wired or wireless network interface 1550 configured to connect the apparatus 1500 to a network, and an input / output interface 1558. The apparatus 1500 can operate based on an operating system stored in the memory 1532.

[0168] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure. Variations and modifications of the embodiments disclosed herein can be made based on the description set forth herein, without departing from the scope and spirit of the disclosure. The specification and examples given are intended as illustrative only and not in a limiting sense. For the avoidance of doubt, the disclosure is to be construed as embracing all alternatives, modifications and equivalents within the spirit and scope of the disclosure as defined by the appended claims.

[0169] It is to be understood that the disclosure is not limited to the precise construction described above and shown in the attached drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the disclosure is limited only by the claims that follow.

Claims

1. A geometric large model training method, characterized in that, The method comprises the following steps: acquiring a geometry image, wherein the geometry image is reconstructed by using a geometry form language; determining a geometry text according to the geometry image, and training a geometry large model by using the geometry image and the geometry text to obtain the geometry large model meeting preset conditions, wherein the geometry text comprises geometry question text and geometry answer text corresponding to the geometry image, and the geometry large model is used for solving geometry questions. 2.The geometric large model training method of claim 1, wherein, The training of the geometry large model by using the geometry image and the geometry text to obtain the geometry large model meeting preset conditions comprises the following steps: training a geometry large model by using the geometry image, wherein the geometry large model comprises a large language model, a multilayer perceptron and a visual encoder; continuing to train the large language model by using the geometry image and the geometry text to obtain the geometry large model meeting preset conditions. 3.The geometric large model training method of claim 2, wherein, The acquisition of the geometry image comprises the following steps: determining a base image to be drawn and image parameters, and drawing the base image according to the image parameters; acquiring a geometry dictionary, and continuing to edit the base image according to the geometry dictionary to obtain a geometry image. 4.The geometric large model training method of claim 3, wherein, The geometry dictionary comprises a vertex coordinate dictionary and a connectivity relationship dictionary. The editing of the base image according to the geometry dictionary to obtain a geometry image comprises the following steps: labeling vertices on the base image according to the vertex coordinate dictionary; connecting the vertices on the base image according to the connectivity relationship dictionary to obtain a geometry image.

5. The geometric large model training method according to claim 4, characterized in that, The training of the geometry large model by using the geometry image comprises the following steps: acquiring a prompt word, and inputting the geometry image and the prompt word into a geometry large model to be trained, so that the geometry large model to be trained outputs a prediction dictionary and a prediction parameter, wherein the prediction dictionary comprises a prediction vertex dictionary and a prediction relationship dictionary; determining a dominant parameter according to the prediction dictionary, the prediction parameter, the geometry dictionary and the image parameter, and determining a first loss function by using the dominant parameter, so as to train the geometry large model to be trained by using the first loss function. 6.The geometric large model training method of claim 5, wherein, The determination of the dominant parameter according to the prediction dictionary, the prediction parameter, the geometry dictionary and the image parameter comprises the following steps: determining a target distance according to the prediction vertex dictionary and the vertex coordinate dictionary, and determining a first score according to the target distance; determining a second score according to the prediction relationship dictionary and the connectivity relationship dictionary; determining a target score according to the first score and the second score, and determining a dominant parameter according to the target score, the prediction dictionary, the prediction parameter, the geometry dictionary and the image parameter. 7.The geometric large model training method of claim 2, wherein, The determination of the geometry text according to the geometry image comprises the following steps: acquiring an image description text when the geometry image is reconstructed, and determining a mapping relationship between the image description text and a geometry text; determining the geometry text according to the image description text based on the mapping relationship. 8.The geometric large model training method of claim 7, wherein, After the determination of the geometry text according to the image description text, the method further comprises the following steps: establish a dataset composed of the image description text and the geometry text, the dataset including basic measurement data, relationship reasoning data and geometry analysis data, the geometry analysis data reflecting constraint relationships between geometry elements. 9.The geometric large model training method of claim 8, wherein, The continuing training of the large language model by using the geometry image and the geometry text to obtain the geometry large model satisfying the preset condition comprises: inputting the geometry image and the geometry question text into the large language model to make the large language model output a predicted answer text; determining a second loss function according to the predicted answer text and the geometry answer text to train the large language model by using the second loss function to obtain the geometry large model satisfying the preset condition. 10.The geometric large model training method of claim 1, wherein, The method further comprises: obtaining a reasoning image and a reasoning text corresponding to the reasoning image, the reasoning text including a reasoning question text and a reasoning answer text; inputting the reasoning image and the reasoning question text into the geometry large model satisfying the preset condition to obtain a predicted answer text, and determining a reasoning score according to the predicted answer text and the reasoning answer text; continuing to train the geometry large model according to the reasoning score.

11. A geometric macro-model application method, characterized by, comprises: obtaining a to-be-searched image and a to-be-searched question, and searching the to-be-searched image and the to-be-searched question by using the geometry large model satisfying the preset condition to obtain a search answer and a corresponding reasoning thought chain, the to-be-searched image being a geometry image; and / or obtaining a to-be-searched question, and searching the to-be-searched question by using the geometry large model satisfying the preset condition to obtain a search answer and a corresponding reasoning thought chain.

12. A geometry large model training apparatus, comprising: comprises: an image reconstruction module configured to obtain a geometry image, the geometry image being reconstructed by using a geometry form language; a model training module configured to determine a geometry text according to the geometry image, and train a geometry large model by using the geometry image and the geometry text to obtain the geometry large model satisfying the preset condition, the geometry text including a geometry question text and a geometry answer text corresponding to the geometry image, the geometry large model being used for solving a geometry question.

13. The geometry large model training apparatus according to claim 12, wherein The model training module is configured to: train a geometry large model by using the geometry image, the geometry large model including a large language model, a multilayer perceptron and a visual encoder; continue to train the large language model by using the geometry image and the geometry text to obtain the geometry large model satisfying the preset condition.

14. The geometry large model training apparatus according to claim 13, wherein The image reconstruction module is configured to: determine a to-be-drawn base image and image parameters, and draw the base image according to the image parameters; obtain a geometry dictionary, and continue to edit the base image according to the geometry dictionary to obtain a geometry image.

15. The geometry large model training apparatus according to claim 14, wherein The geometry dictionary includes a vertex coordinate dictionary and a connectivity relationship dictionary. The image reconstruction module is configured to: label a vertex on the base image according to the vertex coordinate dictionary; connect the vertex on the base image according to the connectivity relationship dictionary to obtain a geometry image.

16. The geometry large model training apparatus according to claim 15, wherein The model training module is configured to: The prompt word is obtained, and the geometric image and the prompt word are input into a to-be-trained geometric large model to make the to-be-trained geometric large model output a predicted dictionary and a predicted parameter, and the predicted dictionary includes a predicted vertex dictionary and a predicted relationship dictionary; Advantage parameters are determined according to the predicted dictionary, the predicted parameter, the geometric dictionary and the image parameter, and a first loss function is determined by using the advantage parameters, so as to train the to-be-trained geometric large model by using the first loss function.

17. The geometric large model training apparatus of claim 13, wherein, The model training module is configured to: Obtain image description text when reconstructing a geometric image, and determine a mapping relationship between the image description text and geometric text; Determine the geometric text according to the image description text based on the mapping relationship.

18. A large geometry model application device, characterized by comprising: Comprise: An image search module is configured to obtain a to-be-searched image and a to-be-searched question, and search the to-be-searched image and the to-be-searched question by using a geometric large model meeting a preset condition to obtain a search answer and a corresponding reasoning thought chain, and the to-be-searched image is a geometric image; And / or A text search module is configured to obtain a to-be-searched question, and search the to-be-searched question by using a geometric large model meeting a preset condition to obtain a search answer and a corresponding reasoning thought chain.

19. A computer-readable storage medium having stored thereon computer program instructions, wherein, The program instructions are executed by the processor to implement the steps of the method of any one of claims 1-11.

20. An electronic device, comprising: Comprise: A memory having a computer program stored thereon; A processor configured to execute the computer program in the memory to implement the steps of the method of any one of claims 1-11.

Images