Visual language model training method and system

Abstract

The invention relates to a visual language model training method and system, and the method comprises the steps: providing a multi-modal geometric training data set which comprises a plurality of data units, each data unit comprises digital image data for encoding a geometric figure, a section of text data for describing a question to the geometric figure and a standardized answer encoded in a markup language format supporting machine analysis; generating one or more candidate answers according to the text data and the digital image data in the geometric training data set by applying a reinforcement learning framework and taking the visual language model as a strategy; for each generated candidate answer, a reward function module is applied to allocate a reward value. According to the method, generalized space intelligent improvement is realized, the effectiveness and robustness of training are improved, and the model can be encouraged to explore different problem solving paths and find a correct reasoning chain.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of artificial intelligence, and particularly relates to a visual language model training method and system. BACKGROUND

[0002] In recent years, multi-modal large models (VLM) represented by GPT-4, Gemini, etc. have made breakthrough progress in natural language understanding, image description, common sense question answering, etc. However, these models often perform poorly when dealing with tasks that require accurate spatial perception, a phenomenon known as the "Molodtsov Paradox" in the field of artificial intelligence, which means that machines perform well in advanced logical reasoning, but struggle in basic perception and motor skills inherent to humans.

[0003] Specifically for the spatial intelligence of VLM, the existing technology mainly faces many bottlenecks. First, the basic spatial ability is weak and has poor generalization, and the existing method for improving the spatial ability of VLM usually fine-tunes some data sets for specific spatial relationships. However, the model trained in this way often only "remembers" the patterns and biases of the specific data set, and does not learn the universal spatial relationship principles. When faced with a slightly changed scene or an unseen spatial task type, the performance of the model will drop sharply, showing poor generalization ability.

[0004] Secondly, the quality of training data is uneven and lacks systematization, although there are some visual question answering (VQA) data sets, they are usually general and contain a large number of non-spatial class questions, resulting in sparse training signals. Even the data sets that focus on spatial ability often have scattered content and lack systematic geometric constraints, which cannot force the model to learn the underlying Euclidean space axioms.

[0005] In addition, there are defects in model evaluation and reward mechanism. During the training process, how to accurately evaluate the answers generated by the model and give correct feedback is a great challenge. For mathematical expression type answers: the traditional string exact matching method cannot handle equivalent but different form mathematical expressions. If simply judged wrong, the correct reasoning of the model will be punished, hindering its learning. For pure numerical type answers, computers inherently have precision errors when performing floating point operations. If the model is required to output numerical values exactly the same as the standard answer, it will be judged wrong due to small calculation errors. Conversely, if the tolerance is too high, it cannot distinguish whether the model is based on correct reasoning to obtain an approximate solution, or is simply a numerical guess. SUMMARY

[0006] To solve the above problems in the prior art, the present application provides a visual language model training method and system.

[0007] According to one aspect of the present invention, a method for training a visual language model is provided, the method comprising the following steps: providing a multimodal geometric training dataset, the geometric training dataset containing multiple data units, each data unit containing digital image data encoding a geometric figure, text data describing a question about the geometric figure, and a standardized answer encoded in a markup language format that supports machine parsing; applying a reinforcement learning framework, using the visual language model as a strategy, to generate one or more candidate answers based on the text data and digital image data in the geometric training dataset; and applying a reward function module to assign a reward value to each generated candidate answer.

[0008] The application of the reward function module to allocate reward values ​​further includes: parsing the standardized answer of the data unit to determine whether it is a mathematical expression type containing symbolic variables or a pure numerical type without symbolic variables; when it is determined to be a mathematical expression type, parsing the candidate answer and the standardized answer into their respective abstract syntax trees or equivalent symbolic representations, normalizing the symbolic representations, confirming that the normalized representations are consistent, and allocating a preset positive reward value; or, when it is determined to be a pure numerical type, converting the candidate answer and the standardized answer into floating-point numbers, calculating the relative error between the two floating-point numbers, comparing the absolute value of the relative error with a preset numerical error threshold, and allocating the preset positive reward value when the absolute value is less than or equal to the threshold.

[0009] The reinforcement learning framework of the application is a population relative policy optimization framework, which specifically includes: in a training iteration, sampling from a population containing multiple policies to generate multiple candidate answers, and calculating and updating the parameters of the policy based on the performance of the reward value obtained by each policy relative to a benchmark policy in the population.

[0010] The baseline strategy is dynamically determined based on the standardization of the reward values ​​of all strategies in the group.

[0011] The normalization process includes at least one of the following operations: sorting the items in the symbolic representation alphabetically, merging like items, or expanding the product.

[0012] The preset numerical error threshold is 0.01; the markup language format is LaTeX.

[0013] The multimodal geometry training dataset processing includes: aggregating initial geometry problem data from at least one digitized textbook, online education platform, or mathematics competition question bank; applying a deduplication module to identify and remove duplicate or highly similar problems based on perceptual hashing or feature vector similarity of image content and semantic similarity of text content; and applying a filtering module to remove data units containing non-Euclidean geometry, poorly defined problems, or images with quality below a preset standard.

[0014] According to another aspect of the present invention, a training system for a visual language model is provided, the system comprising: a data supply module configured to access and provide a multimodal geometric training dataset, each data unit of the dataset containing a digital image, a text question, and a standardized answer encoded in a machine-resolvable format; a model fine-tuning module configured to load the visual language model and apply a reinforcement learning framework to generate candidate answers based on the data provided by the data supply module; and a reward function module configured to receive the candidate answers and the corresponding standardized answers, and assign a reward value to them.

[0015] The reward function module includes: an answer type determiner, used to determine whether the standardized answer is a mathematical expression or a pure numerical type; a symbolic validator, activated when the answer type is a mathematical expression, used to parse, normalize, and compare candidate answers with standardized answers; and a numerical comparator, activated when the answer type is a pure numerical type, used to calculate the relative error between candidate answers and standardized answers and compare it with a preset threshold. The model fine-tuning module is configured to implement a population relative policy optimization algorithm, which performs updates by comparing the performance of a policy with a baseline performance derived from the policy population.

[0016] The symbol verifier implements its parsing, normalization, and comparison functions by calling a computer algebra system library.

[0017] According to another aspect of the present invention, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, characterized in that, when the computer program is executed by one or more processors, the one or more processors cause the one or more processors to perform a training method for a visual language model as described in any of the preceding claims.

[0018] Compared to existing technologies, this invention achieves generalizable spatial intelligence enhancement. It innovatively uses formalized geometry training as a proxy task. Geometry contains rich, systematic, and universal Euclidean spatial constraints, such as angles, proportions, symmetry, and transformations. By training on this task, the model is forced to learn these underlying spatial relationship principles, rather than the surface features specific to the task. Therefore, the trained model can generalize its learned spatial reasoning abilities to other unseen spatial tasks, fundamentally solving the problem of poor generalization ability in existing methods.

[0019] Compared with existing technologies, this invention improves the effectiveness and robustness of training. This invention adopts a reinforcement learning framework, especially the GRPO algorithm, which, compared with the traditional supervised fine-tuning SFT, encourages the model to explore different problem-solving paths and find the correct inference chain, rather than simply imitating the training samples.

[0020] Compared with existing technologies, this invention achieves accurate and fair automated evaluation. The dual-track reward function designed in this invention solves the evaluation problem in the background technology. For mathematical expressions, symbolic equivalence verification ensures consistent positive feedback for all mathematically correct but differently written answers. For pure numerical values, bounded relative error tolerates reasonable floating-point calculation errors while effectively filtering out numerical guesses without reasoning basis with a very small threshold, thus guaranteeing the accuracy of the reward signal.

[0021] Compared with existing technologies, this invention guarantees high-quality training input. The systematically refined multimodal geometric dataset upon which this invention relies, through steps such as aggregation, deduplication, and filtering, ensures the high quality, systematicity, and focus of the training data, providing a solid foundation for achieving the aforementioned beneficial effects. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is a system structure block diagram for training a visual language model according to an embodiment of the present invention.

[0024] Figure 2 This is a flowchart of a method for training a visual language model according to an embodiment of the present invention.

[0025] Figure 3This is a schematic diagram of the internal logic of the reward function module according to an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0027] First, let's explain the key terms used. Among them, the Vision-Language Model (VLM) is an artificial intelligence model that integrates computer vision and natural language processing (NLP) functions. As a multimodal AI system, VLM takes text and images or videos as input and produces text as output, usually in the form of image or video descriptions, to answer questions about images or identify certain parts of images or objects in videos.

[0028] Reinforcement learning with verifiable rewards (RLVR) uses automatically verifiable objective metrics (such as correctness of answers and code executability) as reward signals for reinforcement learning to optimize model performance.

[0029] Groupwise Relative Policy Optimization (GRPO) is a type of RLVR designed to improve the performance of large language models in complex tasks such as mathematical reasoning and programming. Its core idea is to optimize the policy through relative comparisons of samples within a group, rather than relying on absolute reward values, thereby reducing computational costs and improving training stability.

[0030] Spatial intelligence refers to the ability of VLM models to perceive, reason, and act in three-dimensional space and time. It involves the perception and mental manipulation of spatial relationships, covering a variety of tasks such as quantity estimation, orientation judgment, and geometric structure understanding, aiming to enable machines to understand and manipulate the physical world like humans.

[0031] Please see Figure 1This diagram illustrates a structural block diagram of a computing system 100 according to an embodiment of the present invention. The system can be a single server, a distributed computing cluster, or any hardware entity with sufficient computing power. System 100 typically includes at least one processor 110 and a memory 120 communicatively connected to the processor 110. The memory 120 stores computer program instructions that, when executed by the processor 110, cause system 100 to implement the methods of the present invention.

[0032] In one embodiment, system 100 can be functionally considered as comprising multiple interconnected logical modules: a data supply module 121, responsible for storing, managing, and providing the multimodal geometric training dataset required for training; a model fine-tuning module 122, the core training execution unit, which loads the visual language model (VLM) to be trained and executes the reinforcement learning algorithm; and a reward function module 123, a key module for evaluating the model output and providing a reward signal. This module can be further subdivided into an answer type determiner 123a, a symbol validator 123b, and a numerical comparator 123c.

[0033] Now combine Figure 2 The flowchart 200 shown illustrates the training method of the present invention in detail.

[0034] 201. Provide a multimodal geometric training dataset. This step corresponds to the function of the data supply module 121 in the system. One of the core ideas of this invention is that a high-quality, systematic dataset is the foundation for successful training. Generating a geometric training dataset requires steps such as aggregation, refinement, deduplication, filtering, and standardization, as described below.

[0035] In one embodiment, the method includes an aggregation step that extensively collects raw data containing geometric figures and problems from multiple authoritative sources, such as digitized geometry textbooks, online education platforms like Khan Academy, open-source datasets like GeoQA, and competition problem sets like the International Mathematical Olympiad.

[0036] In one embodiment, a refining step is included to rigorously process the aggregated raw data.

[0037] One embodiment includes a deduplication step to prevent the model from overfitting to duplicate samples. For images, perceptual hashing (pHash) or extracting image feature vectors from models such as CLIP can be used, and the similarity of image content can be determined by calculating hash distance or vector cosine similarity. For text, techniques such as TF-IDF or Sentence-BERT can be used to calculate the semantic similarity of the text. When the similarity between images and text both exceed a preset threshold, they are identified as duplicate data and removed.

[0038] One embodiment includes a filtering step to filter out data that does not meet the requirements in order to ensure the focus and effectiveness of the training task. This includes removing data that contains non-Euclidean geometry (such as hyperbolic geometry or spherical geometry), data with ambiguous problem descriptions or lacking necessary conditions, and data with blurry images or containing too much irrelevant noise.

[0039] In one embodiment, a standardization step is included to organize the refined data into a unified structured format, i.e., data units. Each data unit contains: digital image data (such as JPEG or PNG format); text data (UTF-8 encoded strings); and standardized answers. Crucially, the standardized answers are encoded using a machine-readable markup language, preferably LaTeX in this embodiment. For example, the answer "The circumference of a circle is 2πr" is encoded as the string 2πr, which lays the foundation for subsequent automated and accurate verification.

[0040] In one embodiment, the final generated dataset (e.g., a dataset called Euclid25K) systematically covers core concepts in plane and solid geometry, such as angle calculation, line segment ratio, area / volume solving, and symmetry transformation, becoming a proxy task for training the VLM to learn the underlying spatial rules.

[0041] 202. Apply a reinforcement learning framework, using the visual language model as a strategy, to generate one or more candidate answers based on the text data and digital image data in the geometric training dataset.

[0042] This method includes fine-tuning using a reinforcement learning framework, which corresponds to the function of model fine-tuning module 122. This module loads a pre-trained VLM (such as LLaVA, MiniGPT-4, etc.) as the policy network to be trained.

[0043] Unlike traditional supervised fine-tuning (SFT), which merely allows the model to mimic the standard answer, this invention employs a reinforcement learning (RL) framework. In this embodiment, an advanced RL algorithm, Group Relative Policy Optimization (GRPO), is preferably used. The core process of GRPO is as follows.

[0044] Here, a policy population P = {π1, π2, ..., πn} is maintained, where each π is an instance of VLM or a variant thereof. In a training iteration, for a given data unit (image, question), k policies are sampled from the population P. Each sampled policy πᵢ generates a candidate answer aᵢ.

[0045] 203. For each generated candidate answer, apply the reward function module to assign a reward value.

[0046] Each answer aᵢ is fed into the reward function module to calculate the reward value rᵢ. A baseline performance of the population is calculated, which could be, for example, the normalization of the rewards obtained by these k strategies.

[0047] In this approach, for each policy πᵢ, the advantage function used for parameter updates is no longer its absolute reward rᵢ, but rather its relative reward relative to the baseline (advantageᵢ = rᵢ - r_baseline). Using this advantage function, the parameters of each policy πᵢ are updated according to policy gradient algorithms such as Proximal Policy Optimization (PPO).

[0048] The advantage of using GRPO compared to simple RL is that it effectively reduces the variance of reward values ​​by introducing relative performance comparisons within the population, making the training process more stable, the convergence speed faster, and the policy space better explored.

[0049] Finally, this method includes applying a reward function module to allocate rewards, corresponding to the functionality of reward function module 123 and its sub-modules. This module is invoked after model fine-tuning module 122 generates a candidate answer to provide precise feedback signals.

[0050] In one embodiment, the module receives candidate answers and standardized answers. See also Figure 3 The answer type determination is performed by the answer type determiner 123a. First, the standardized answer string is parsed, and it is determined whether it is a mathematical expression type or a pure numeric type by checking whether the string contains predefined symbolic variables (such as 'a', 'b', 'x', 'r', 'h', etc.).

[0051] In one embodiment, for mathematical expression type processing, symbolic equivalence verification is performed by symbolic verifier 123b. If the expression is determined to be a mathematical expression type, this verifier is activated; this verifier cannot use simple string comparison.

[0052] First, the candidate answer (e.g., the model might generate (r*2)*pi) and the standardized answer (2 \pi r) are passed to a computer algebra system (CAS), such as the SymPy library. CAS parses these two strings into their respective abstract syntax trees (ASTs) or internal symbolic representations.

[0053] Next, these two symbolic representations are normalized. Normalization operations may include: reordering the multiplication terms alphabetically, merging like terms, and automatically performing simple algebraic simplifications. After normalization, (r*2)*pi and 2 \pi r will be converted into the same unique internal representation.

[0054] Finally, compare the two normalized representations. If they are completely identical, the candidate answer is determined to be correct.

[0055] In one embodiment, a positive reward is assigned. If the verification passes, a preset positive reward value, such as +1.0, is assigned; otherwise, the reward is 0 or a small negative value.

[0056] In one embodiment, for pure numeric type processing, numerical verification is performed by numerical comparator 123c; if it is determined to be a pure numeric type, this comparator is activated.

[0057] This process involves converting both candidate and standardized answers to high-precision floating-point numbers; then calculating the relative error between them: relative_error = |candidate_value - true_value| / |true_value|. The use of relative error instead of absolute error aims to fairly handle answers of different orders of magnitude.

[0058] In one embodiment, a comparison is made with a threshold; the calculated relative error is compared with a preset, extremely small numerical error threshold. In this embodiment, the threshold is preferably 0.01, or 1%.

[0059] In one embodiment, a positive reward is assigned. The answer is deemed correct and a preset positive reward value, such as +1.0, is assigned only if the absolute value of the relative error is less than or equal to the threshold. This strict threshold design accommodates the inherent floating-point precision issues of computers while effectively preventing the model from "cheating" for rewards by randomly guessing a value within a certain range.

[0060] Through the above methods, this invention constructs a complete closed loop from high-quality data input to efficient exploratory training and then to precise and robust feedback, which can systematically improve the underlying spatial intelligence and generalization ability of VLM in an unprecedented way.

[0061] Specifically, in one embodiment, the present invention uses the Euclid25K multimodal geometry dataset to achieve this goal by training the model's ability in visual spatial imagination and symbolic mathematical reasoning. Euclid25K brings together open-source datasets such as Geometry3K, MMK12, and SolidGeo, as well as K-12 textbooks and intermediate and advanced mathematics competition problems, totaling 24,840 geometry problems.

[0062] This process involves manual deduplication and normalization, and removal of samples with missing images or incomplete fields. Furthermore, manual review is conducted to eliminate non-Euclidean geometry problems as much as possible.

[0063] Organize each question into a format like... The triplet, in which, It can be a single image or multiple images. (Answer) This is the normalized LaTeX expression. Regarding the validation set, considering the complexity of solid geometry in three-dimensional space, it's necessary to test the model's ability to handle viewpoint invariance, polyhedral cutting, and volume-surface area reasoning. Therefore, approximately 1000 solid geometry problems were extracted from Euclid25K to form an independent validation set, with the remainder used as the training set.

[0064] To minimize confounding factors, the model was trained solely on Euclid25K using reinforcement learning (RLVR) with verifiable rewards, and a uniform cue template was employed on downstream spatial intelligence benchmarks without using any additional tricks, to ensure that the performance gains truly stemmed from geometric knowledge rather than other heuristics.

[0065] In one specific embodiment, following the standard training process of GRPO, the geometric solution capabilities of the Qwen2.5VL (3B, 7B, 32B, 72B) and RoboBrain2.0 (7B, 32B) models are enhanced.

[0066] During training, the input problem is first set up. From the old strategy Medium sampling answer Calculate the objective function : Among them, there are and and the advantage function for Then maximize the following loss function in, The parameters represent the counting loss of the VLM model; min(A, B) means taking the minimum value between A and B; γ represents the sum of the lengths of all sampled answers; clip indicates that... The value is clipped to limit it to 1.2 / 0.8 (here). =0.2); mean and std are functions used to count the mean and the difference between the labels, and KL represents the KL divergence. G is the size of the training set, |o i | represents the number of tokens the model used to answer a question in the training set; β is a balance coefficient that controls the weight between the objective function J(θ) and the KL divergence, which measures the current policy π. θ With reference strategy π ref Differences; For question q and answer o i Find the expected value under the distribution; R i It is the reward for the i-th answer.

[0067] Furthermore, within the verifiable reward framework, in addition to a unified formatted reward, separate reward functions are designed for different tasks. If the answer is a LaTeX mathematical expression containing variables, MATHVERIFY is called to perform a sign equivalence check, ensuring that equivalent forms such as 2πr, 2πr, and (2r)π receive the same reward. For purely numerical answers, to prevent reward hacking, the traditional average relative error is abandoned; instead, a reward is only given when the predicted value is within ±1% of the true value. In this case, the reward calculation formula is as follows: r ans Reward value: 1 when the predicted value meets the condition, and 0 otherwise; p: The predicted value generated by the model; t: The true value of the problem.

[0068] This strategy filters out rough answers that are irrelevant to the geometric conclusions, while tolerating rounding or floating-point truncation errors during the generation process.

[0069] In the specific training, the default configurations of VeRL and EasyR1 were used to train the Qwen2.5VL series (3B, 7B, 32B, 72B) and RoboBrain2.0 series (7B, 32B). The training epochs and optimizer were as follows: 15 epochs were trained using the Adam optimizer with a learning rate of 1×10⁻. 6 The weight decay was set to 1×10⁻². 128 NVIDIA H100 80GB HBM3 graphics cards were used for the 32B and 72B models; 32 of the same graphics cards were used for the 3B and 7B models to meet the large memory requirements of reinforcement learning training.

[0070] In the GRPO strategy, each problem was rolled out 5 times, and the sampling temperature was set to 1. The KL divergence coefficient β was set to 1 × 10⁻². During the testing phase, to ensure the reproducibility of the results, the sampling temperature was fixed at 0 according to the standards of VSIBench and MindCube. The specific benefits are shown in Tables 1-3 below.

[0071] Table 1 shows the benefits of using the method of this invention on the SuperClevr and Omni3DBench datasets. Table 2 shows the benefits on the VSI-Bench dataset. Table 3 shows the benefits the model gains using the spatial intelligence dataset and the constructed Euclidean geometry dataset. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A training method for a visual language model, characterized in that, Includes the following steps: A multimodal geometry training dataset is provided, which contains multiple data units, each containing digital image data of an encoded geometry, a text data describing a question about the geometry, and a standardized answer encoded in a markup language format that supports machine parsing. By applying a reinforcement learning framework and using the visual language model as a strategy, one or more candidate answers are generated based on the text data and digital image data in the geometric training dataset. For each generated candidate answer, a reward function module is applied to assign a reward value.

2. The method according to claim 1, characterized in that, The application reward function module for allocating reward values ​​further includes: The standardized answer to the data unit is parsed and determined to be either a mathematical expression type containing symbolic variables or a pure numeric type without symbolic variables; When the answer is determined to be a mathematical expression, the candidate answer and the standardized answer are parsed into their respective abstract syntax trees or equivalent symbolic representations. The symbolic representations are then normalized. Once the normalized representations are confirmed to be consistent, a preset positive reward value is assigned. Alternatively... When the answer is determined to be a pure numerical type, the candidate answer and the standardized answer are converted into floating-point numbers, the relative error between the two floating-point numbers is calculated, the absolute value of the relative error is compared with a preset numerical error threshold, and if the absolute value is less than or equal to the threshold, the preset positive reward value is assigned.

3. The method according to claim 1, characterized in that, The reinforcement learning framework described is a population-relative policy optimization framework, specifically including: In a training iteration, multiple candidate answers are generated by sampling from a population containing multiple policies, and the parameters of the policy are calculated and updated based on the performance of each policy relative to a benchmark policy in the population.

4. The method according to claim 3, characterized in that, The baseline strategy is dynamically determined based on the standardization of the reward values ​​of all strategies in the group.

5. The method according to claim 2, characterized in that, The normalization process includes: Perform at least one of the following operations on the items in the symbolic representation: alphabetical sorting, merging like items, or expanding the product.

6. The method according to claim 2, characterized in that, The preset numerical error threshold is 0.01; the markup language format is LaTeX.

7. The method according to claim 1, characterized in that, The processing of the multimodal geometric training dataset includes: Aggregate initial geometry problem data from at least one digital textbook, online education platform, or math competition question bank; The application deduplication module identifies and removes duplicate or highly similar problems based on the perceptual hash or feature vector similarity of image content and the semantic similarity of text content. Apply a filtering module to remove data units that contain non-Euclidean geometry, have poorly defined problems, or have image quality below a preset standard.

8. A training system for a visual language model, characterized in that, include: The data supply module is configured to access and provide a multimodal geometric training dataset, each data unit of which contains a digital image, a text question, and a standardized answer encoded in a machine-resolvable format. The model fine-tuning module is configured to load the visual language model and apply a reinforcement learning framework to generate candidate answers based on the data provided by the data supply module. The reward function module is configured to receive the candidate answer and the corresponding standardized answer, and assign a reward value to them.

9. The system according to claim 8, characterized in that, The reward function module includes: Answer type determiner, used to determine whether the answer type of a standardized answer is a mathematical expression type or a pure numeric type; The symbol validator, activated when the answer type is a mathematical expression, is used to parse, normalize, and compare candidate answers with standardized answers; The numerical comparator, activated when the answer type is purely numerical, is used to calculate the relative error between the candidate answer and the standardized answer and compare it with a preset threshold. The model fine-tuning module is configured to implement a population-relative policy optimization algorithm, which performs updates by comparing the performance of a policy with a baseline performance derived from the policy population.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by one or more processors, it causes the one or more processors to perform the training method for the visual language model as described in any one of claims 1 to 7.

Images