Remote sensing visual language large model training method and device based on unified reinforcement learning

Abstract

The invention relates to a remote sensing visual language large model training method and device based on unified reinforcement learning, and belongs to the technical field of artificial intelligence and remote sensing image processing, and the method comprises the steps: carrying out the preprocessing of an input remote sensing image and a text instruction, and extracting visual features and text features; performing modal alignment, and inputting the modal alignment result into a pre-trained large language model for supervised instruction fine adjustment to obtain a basic model; constructing a multi-dimensional deterministic unified reward module based on a truth value; and performing enhanced fine tuning on the basic model, calculating a reward value output in the group by using a unified reward module, and updating model parameters based on relative advantages to obtain an optimized remote sensing visual language large model. According to the method, a deterministic reward module is adopted, a value network is abandoned through group relative strategy optimization, the calculation cost is reduced, task-level indexes are directly optimized through a multi-dimensional unified reward function, and the precision and output normalization of remote sensing image interpretation are improved.

Description

Technical Field

[0001] This invention relates to a method and apparatus for training a large remote sensing visual language model based on unified reinforcement learning, belonging to the fields of artificial intelligence and remote sensing image processing technology. Background Technology

[0002] With the rapid development of remote sensing Earth observation technology, massive amounts of remote sensing image data urgently require efficient and intelligent interpretation methods. In recent years, Multimodal Large Language Models (MLLMs) based on the Transformer architecture have made significant progress in general vision-language tasks, providing new solutions for tasks such as remote sensing image description, visual question answering, and object detection and counting. However, in the process of realizing this invention, the inventors discovered at least the following problems in the existing technology: Existing remote sensing multimodal models mainly rely on supervised fine-tuning (SFT). While this approach enables models to have basic dialogue and recognition capabilities, it still has limitations when dealing with complex and specific tasks in the field of remote sensing: (1) Illusion problem and format error: The model is prone to generating descriptions that do not match the image content, or the structured format of the output (such as bounding box coordinates) does not conform to the parsing specifications, resulting in downstream applications being unable to read it. (2) Difficulty in task alignment: Remote sensing tasks are diverse (such as counting, detection, and yes / no judgment), and the traditional cross-entropy loss function is difficult to directly optimize the evaluation indicators (such as intersection-over-union ratio, counting error, etc.) of these specific tasks. (3) Lack of fine-grained feedback: Relying solely on text matching of standard answers cannot provide a fine-grained evaluation of the "quality" of the model output (for example, a detection box that is slightly off but correct overall should be penalized differently from one that is completely wrong).

[0003] To address these issues, existing research has begun to explore the introduction of reinforcement learning (RL), such as human feedback-based reinforcement learning (RLHF). However, traditional RLHF methods typically require training an additional neural network reward model to evaluate the generation quality. This approach faces two major challenges in the field of remote sensing: firstly, it significantly increases memory usage and computational resource consumption; secondly, the reward model fitted by the neural network is prone to reward hacking, making it difficult to accurately reflect the rigid constraints on geometric coordinates and numerical values ​​in remote sensing tasks, unlike mathematical rules.

[0004] It is evident that traditional supervised learning paradigms, as well as traditional reinforcement learning paradigms that rely on independent value networks, are both inadequate for efficiently and accurately addressing the "modal alignment" and "rigid metric optimization" problems in remote sensing multimodal tasks. Therefore, constructing a feedback mechanism that eliminates the need for additional value network training and directly transforms the multidimensional task evaluation metrics required in the remote sensing field into model optimization dynamics is crucial for achieving deep alignment between semantic understanding and spatiotemporal geometric features of remote sensing images. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes a training method and apparatus for a large remote sensing visual language model based on unified reinforcement learning. This method can solve the problems of inaccurate model localization, large counting errors, non-standard output formats, high training costs, and difficulty in reward fitting in existing technologies.

[0006] The technical solution adopted by this invention to solve its technical problem is as follows: In a first aspect, embodiments of the present invention provide a method for training a large remote sensing visual language model based on unified reinforcement learning, comprising the following steps: Step S1: Preprocess the input remote sensing images and text commands, and extract visual features and text features respectively; Step S2: Modal alignment of the visual features and text features is performed and input into a pre-trained large language model for supervised fine-tuning to obtain the basic model; Step S3: Construct a multidimensional deterministic unified reward module based on truth values. The unified reward module includes at least two of the following: quantity reward, detection reward, binary reward, and format reward, and is used to evaluate the quality of the model output. Step S4: The basic model is enhanced and fine-tuned based on the Group Relative Policy Optimization (GRPO) algorithm. The reward value of the output within the group is calculated using the unified reward module. The model parameters are updated based on the relative advantage to obtain the optimized remote sensing visual language large model.

[0007] Secondly, embodiments of the present invention provide a training device for a large remote sensing visual language model based on unified reinforcement learning, comprising: The preprocessing module is used to preprocess the input remote sensing images and text commands, extracting visual features and text features respectively. The instruction fine-tuning module is used to perform modal alignment of the visual features and text features, input them into a pre-trained large language model for supervised instruction fine-tuning, and obtain the base model. A reward calculation module is used to construct a multidimensional deterministic unified reward module based on truth values. The unified reward module includes at least two of the following: quantity reward, detection reward, binary reward, and format reward, and is used to evaluate the quality of the model output. The enhancement and fine-tuning module is used to enhance and fine-tune the basic model based on the group relative strategy optimization algorithm. It uses the unified reward module to calculate the reward value of the output within the group, updates the model parameters based on the relative advantage, and obtains the optimized remote sensing visual language large model.

[0008] Thirdly, an electronic device provided by an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the remote sensing visual language large model training method based on unified reinforcement learning as described above.

[0009] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the remote sensing visual language large model training method based on unified reinforcement learning as described above.

[0010] One of the above technical solutions has the following advantages or beneficial effects: 1. The embodiments of the present invention introduce a multidimensional deterministic unified reward module based on truth values, which directly uses real labels to calculate accurate reward values, replacing the traditional unstable neural network reward model and solving the problems of difficulty in reward fitting and high computational resource consumption.

[0011] 2. The embodiments of the present invention adopt a group relative strategy optimization algorithm, which generates multiple outputs for the same input and calculates the relative advantage, thereby eliminating the dependence on independent value networks and realizing efficient reinforcement learning without value networks, significantly reducing training complexity.

[0012] 3. The embodiments of the present invention integrate multi-dimensional evaluation indicators such as quantity, detection, binary and format into a unified reward mechanism, which can directly optimize task-level indicators (such as IoU and counting error), successfully solve the technical problem that the traditional cross-entropy loss function is difficult to directly optimize non-differentiable indicators, and realize the deep alignment of semantic understanding and geometric positioning in remote sensing image interpretation.

[0013] 4. By introducing format rewards, this embodiment of the invention dynamically constrains and standardizes the model generation strategy, effectively improving the structure of the model output results and fundamentally eliminating parsing failures caused by non-standard format outputs, thus meeting the engineering needs of downstream automation applications. Attached Figure Description

[0014] Figure 1This is a flowchart illustrating a training method for a large remote sensing visual language model based on unified reinforcement learning, according to an exemplary embodiment. Figure 2 This is a schematic diagram of a training device for a large remote sensing visual language model based on unified reinforcement learning, according to an exemplary embodiment. Detailed Implementation

[0015] To more clearly illustrate the technical features of the present invention, the present invention will be described in detail below through specific embodiments and in conjunction with the accompanying drawings.

[0016] like Figure 1 As shown in the figure, an embodiment of the present invention provides a method for training a large remote sensing visual language model based on unified reinforcement learning, which includes the following steps: Step S1: Preprocess the input remote sensing images and text commands, and extract visual features and text features respectively; Step S2: Modal alignment of the visual features and text features is performed and input into a pre-trained large language model for supervised fine-tuning to obtain the basic model; Step S3: Construct a multidimensional deterministic unified reward module based on truth values. The unified reward module includes at least two of the following: quantity reward, detection reward, binary reward, and format reward, and is used to evaluate the quality of the model output. Step S4: The basic model is enhanced and fine-tuned based on the group relative strategy optimization algorithm. The reward value of the output within the group is calculated using the unified reward module. The model parameters are updated based on the relative advantage to obtain the optimized remote sensing visual language large model.

[0017] As one possible implementation of this embodiment, step S1 includes the following steps: Step S11: Encode the input remote sensing image using a pre-trained visual encoder, extract its deep visual features, and convert them into image embedding vectors. Step S12: The input text instructions are encoded by a word segmenter to convert the text sequence into a text embedding vector.

[0018] As one possible implementation of this embodiment, the visual encoder is a visual backbone network based on the Transformer architecture, including any one of CLIP-ViT, Swin Transformer, or ViT.

[0019] As one possible implementation of this embodiment, the visual backbone network is a CLIP-ViT model, whose output is a series of visual feature blocks, denoted as... ,in This represents the number of feature blocks.

[0020] As one possible implementation of this embodiment, the text instructions include task instructions for description, detection, counting, or judgment. After processing by a word segmenter, a text embedding vector is obtained, denoted as... ,in The length of the text sequence.

[0021] As one possible implementation of this embodiment, step S2 includes the following steps: Step S21: The image embedding vector is mapped to the same feature space as the text embedding vector through a multimodal projection layer to obtain aligned visual features; Step S22: The aligned visual features and the text embedding vector are concatenated in the sequence dimension to form a joint input sequence; Step S23: Input the joint input sequence into the pre-trained large language model, and use the cross-entropy loss function for supervised instruction fine-tuning to obtain the base model.

[0022] As one possible implementation of this embodiment, the multimodal projection layer is composed of a multilayer perceptron or a Q-Former structure.

[0023] As one possible implementation of this embodiment, in step S22, the concatenated joint input sequence is denoted as... ,in For the mapped visual features, This is a text embedding vector.

[0024] As one possible implementation of this embodiment, the large language model is a Transformer decoder-based model that uses cross-entropy loss during the instruction fine-tuning stage. Optimize.

[0025] As one possible implementation of this embodiment, step S3 includes the following steps: Step S31: Calculate the quantity reward based on the difference between the predicted quantity and the actual quantity output by the model. Step S32: Calculate the detection reward based on the intersection-union ratio of the predicted bounding box and the ground truth bounding box output by the model. Step S33: Calculate the binary reward based on the consistency between the predicted category and the true category output by the model. Step S34: Calculate the format reward based on whether the structured format output by the model conforms to the predefined specifications; Step S35: At least two of the quantity reward, detection reward, binary reward and format reward are weighted and merged to obtain a unified reward value.

[0026] As one possible implementation of this embodiment, the quantity reward The calculation formula is: , in, For the number of predictions by the model, For the actual quantity, This is the temperature coefficient.

[0027] As one possible implementation of this embodiment, the detection reward The calculation formula is: , in, For the set of true bounding boxes, For the first A true bounding box, No. One predicted bounding box.

[0028] As one possible implementation of this embodiment, the binary reward is defined as: If the model predicts the category and the true label, then ; If the model predicts a category that is inconsistent with the true label, then .

[0029] As one possible implementation of this embodiment, the format reward The determination is made based on whether the model output contains complete structured labels and conforms to the numerical range: .

[0030] As one possible implementation of this embodiment, the unified reward The calculation formula is: , in, These are the weighting coefficients for each reward component.

[0031] As one possible implementation of this embodiment, step S4 includes the following steps: Step S41, query the given multimodal input. Generate a set using the current policy model Different output sequences ; Step S42: Calculate the reward value for each output sequence using the unified reward module to obtain the reward set. ; Step S43: Calculate the relative advantage value for each output based on the mean and standard deviation of the reward values ​​within the group; Step S44: Construct an objective function based on the relative advantage value, and update the parameters of the current policy model by combining the KL divergence constraint of the basic model, so that the model tends to generate high reward output.

[0032] As one possible implementation of this embodiment, in step S41... It is an integer greater than 1.

[0033] As one possible implementation of this embodiment, the relative advantage value The calculation formula is: , in For the first The reward for each output, and These are the mean and standard deviation of the group rewards, respectively. It is a very small constant.

[0034] As one possible implementation of this embodiment, the objective function is: , in, For the current strategy model, For the aforementioned basic model, To enhance the probability ratio between the old and new strategies during fine-tuning training, For the strategy model in the sampling phase, Let be the relative advantage value of the i-th output. The cutoff factor is , The KL divergence weights constrain the current model. Without deviating from the basic model , for The weighting coefficients.

[0035] As one possible implementation of this embodiment, the basic model The parameters remain frozen during the fine-tuning phase.

[0036] This invention replaces the neural network reward model with a multidimensional deterministic unified reward module and eliminates the value network through group relative strategy optimization. While reducing computational costs, it directly optimizes task-level indicators, effectively improving the accuracy and output standardization of remote sensing image interpretation.

[0037] like Figure 2 As shown in the figure, an embodiment of the present invention provides a training device for a large remote sensing visual language model based on unified reinforcement learning, comprising: The preprocessing module is used to preprocess the input remote sensing images and text commands, extracting visual features and text features respectively. The instruction fine-tuning module is used to perform modal alignment of the visual features and text features, input them into a pre-trained large language model for supervised instruction fine-tuning, and obtain the base model. A reward calculation module is used to construct a multidimensional deterministic unified reward module based on truth values. The unified reward module includes at least two of the following: quantity reward, detection reward, binary reward, and format reward, and is used to evaluate the quality of the model output. The enhancement and fine-tuning module is used to enhance and fine-tune the basic model based on the group relative strategy optimization algorithm. It uses the unified reward module to calculate the reward value of the output within the group, updates the model parameters based on the relative advantage, and obtains the optimized remote sensing visual language large model.

[0038] It should be noted that the specific implementation of each module / unit in the device embodiment can be referred to the relevant description in the foregoing method embodiment, and will not be repeated here.

[0039] Using one of the above technical solutions to detect changing regions within a public dataset, the specific implementation steps for training a large remote sensing visual language model based on unified reinforcement learning are as follows.

[0040] Step 1: Construct a multimodal input and preprocessing module: In this embodiment, the input data includes high-resolution remote sensing images and corresponding text instructions.

[0041] (1) For the input remote sensing images The image is processed using a pre-trained visual encoder. Preferably, this embodiment uses CLIP-ViT as the visual backbone network. After processing by the encoder, the image outputs a series of visual feature blocks (i.e., image embeddings), denoted as... ,in This represents the number of feature blocks.

[0042] (2) For the input text instructions (i.e., "describe the changing areas between two images" or "detect the aircraft in the image"), the text sequence is converted into corresponding text embeddings using a tokenizer, denoted as . ,in The length of the text sequence.

[0043] Step 2: Multimodal instruction fine-tuning training: This step aims to enable the model to establish basic visual-language alignment capabilities.

[0044] (1) To address the inconsistency between the dimensions of visual features and text features, this embodiment of the invention designs a multimodal projection layer. This layer is composed of a multilayer perceptron (MLP) or a Q-Former, which projects visual features... Mapping to text features The same feature space yields mapped visual features. .

[0045] (2) Map the visual features Text features The sequences are concatenated along the sequence dimension to form a joint input sequence. .

[0046] (3) Combine the sequence The input is fed into a large language model. This model contains... The Transformer decoder consists of stacked layers (Layer 1 to Layer N), each containing a multi-head self-attention mechanism and a feedforward neural network (FFN). At this stage, the cross-entropy loss function is utilized. The model is then subjected to supervised fine-tuning so that it can generate a base response that conforms to the instructions.

[0047] Step 3: Construct a unified reward model based on multi-dimensional indicators: To avoid the errors and resource consumption caused by training additional reward models, this embodiment of the invention directly utilizes the real labels in the training set during the reinforcement learning phase to construct a rule-based unified deterministic reward function. This function consists of a weighted average of four specific sub-rewards: quantity reward, detection reward, binary reward, and format reward. A unified reward is then constructed based on these four specific sub-rewards.

[0048] (1) For counting tasks. Let the target number predicted by the model be... The number of real tags is To apply a soft penalty to errors, this embodiment defines the quantity reward as follows: , in, This is a temperature coefficient used to adjust the tolerance for counting errors. The reward is designed to encourage the model's output values ​​to approximate the true values.

[0049] (2) For target detection and localization tasks. Let the set of bounding boxes output by the model be... The set of true bounding boxes is This embodiment uses the maximum matching strategy to calculate the intersection-over-union (IoU) between the predicted bounding box and the ground truth bounding box. The detection reward is defined as: , in, For the first A true bounding box, No. One predicted bounding box.

[0050] (3) For yes / no judgment or classification tasks. If the model predicts the category that matches the true label, then ,otherwise .

[0051] (4) To ensure that the output of the large model can be parsed by downstream programs, this embodiment of the invention introduces a format reward. The text output by the model strictly follows a predefined structure. In this example, the predefined structure is set as follows: <box> and< / box> Tags, when outputting structured formats, must contain complete... <box> and< / box> Labels with coordinate values ​​between 0 and 1000, and correct format and values, will receive a positive reward; otherwise, they will receive a strong negative penalty. The specific calculation formula is as follows: .

[0052] (5) The above reward components are linearly weighted and merged to obtain the final unified reward: , In this embodiment, after repeated parameter adjustments and optimizations, the following settings are implemented: , , , The weighting coefficients mentioned above are for illustrative purposes only and can be adjusted according to different task requirements.

[0053] Step 4: Fine-tuning the model based on group-relative strategy optimization: Based on the SFT model obtained in step 2, reinforcement learning training is performed using the unified reward model constructed in step 3 to further improve the model's performance on professional tasks.

[0054] (1) For a given multimodal input query (Includes images) and text ,Model Random sampling is performed based on the current strategy to generate a set of... Different output sequences .

[0055] As a preferred embodiment, the group size is set. This means generating eight different answer instances for the same question at once.

[0056] (2) Using the unified reward function defined in step 3, combined with the input The corresponding ground truth labels are as follows: Calculate the reward value for each output sequence to obtain the reward set. .

[0057] (3) To measure the relative quality of each output within the current group, calculate the mean of the reward within the group. and standard deviation : ， , Then calculate the first The advantage value of each output : , in, To prevent the minimum value where the denominator is zero.

[0058] (4) Based on the calculated advantage value To update the model parameters, minimize the following loss function: , in, For the current training model, The base model (a copy of the model fine-tuned in step 2, with parameters frozen during the reinforcement learning phase). To enhance the probability ratio between the old and new strategies during fine-tuning training, The probability distribution of the old strategy model used in the sampling phase is used to calculate the importance sampling ratio; This is a truncation operation used to ensure training stability. The cutoff factor is set to 0.2 in this embodiment as a preferred option. Meanwhile, The KL divergence term constrains the current model. Without deviating from the basic model (i.e., the model after SFT) is too far away to prevent catastrophic forgetting or language ability degradation during reinforcement learning. Specifically, for each generated token, the difference in the output probability distribution of the two models is calculated using the following formula: , This term exists as a penalty to ensure that the output style of the trained model does not deviate from the scope of human natural language. Through the above training mechanism, the model can automatically... The differences in the quality of group sampling results enhance the probability of generating high-scoring responses and suppress the generation of low-scoring responses (such as format errors and positioning deviations). Furthermore, the entire process requires no additional value model training, significantly reducing memory usage and improving training efficiency. In this embodiment, the following settings are configured... .

[0059] This invention, by introducing a unified reward model based on multidimensional indicators and a reinforcement learning fine-tuning strategy, effectively alleviates the problems of instruction alignment difficulties, output format errors, and low numerical positioning accuracy in general multimodal large models for specific remote sensing tasks. Furthermore, the training method of this invention is a rare innovative scheme that introduces reinforcement learning feedback mechanisms into the construction of remote sensing multimodal large models. Through innovative design and collaborative optimization of multidimensional reward signals such as quantity, detection, binary representation, and format, it effectively suppresses model illusions and achieves accurate understanding and structured output of object-level information from remote sensing images, possessing significant value for future research.

[0060] An electronic device provided in this invention includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of any of the above-described methods for training large remote sensing visual language models based on unified reinforcement learning.

[0061] Specifically, the aforementioned memory and processor can be general-purpose memory and processor, without any specific limitations. When the processor runs the computer program stored in the memory, it can execute the aforementioned training method for a large remote sensing visual language model based on unified reinforcement learning.

[0062] Corresponding to the above application startup method, this embodiment of the invention also provides a computer-readable storage medium storing a computer program, which, when run by a processor, executes the steps of any of the above-described methods for training large remote sensing visual language models based on unified reinforcement learning.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A training method for a large-scale remote sensing visual language model based on unified reinforcement learning, characterized in that, Includes the following steps: Step S1: Preprocess the input remote sensing images and text commands, and extract visual features and text features respectively; Step S2: Modal alignment of the visual features and text features is performed and input into a pre-trained large language model for supervised fine-tuning to obtain the basic model; Step S3: Construct a multidimensional deterministic unified reward module based on truth values. The unified reward module includes at least two of the following: quantity reward, detection reward, binary reward, and format reward, and is used to evaluate the quality of the model output. Step S4: The basic model is enhanced and fine-tuned based on the group relative strategy optimization algorithm. The reward value of the output within the group is calculated using the unified reward module. The model parameters are updated based on the relative advantage to obtain the optimized remote sensing visual language large model.

2. The method for training a large remote sensing visual language model based on unified reinforcement learning according to claim 1, characterized in that, Step S1 includes the following steps: Step S11: Encode the input remote sensing image using a pre-trained visual encoder, extract its deep visual features, and convert them into image embedding vectors. Step S12: The input text instructions are encoded by a word segmenter to convert the text sequence into a text embedding vector.

3. The method for training a large remote sensing visual language model based on unified reinforcement learning according to claim 2, characterized in that, The visual encoder is a visual backbone network based on the Transformer architecture, including any one of CLIP-ViT, SwinTransformer, or ViT.

4. The method for training a large remote sensing visual language model based on unified reinforcement learning according to claim 2, characterized in that, Step S2 includes the following steps: Step S21: The image embedding vector is mapped to the same feature space as the text embedding vector through a multimodal projection layer to obtain aligned visual features; Step S22: The aligned visual features and the text embedding vector are concatenated in the sequence dimension to form a joint input sequence; Step S23: Input the joint input sequence into the pre-trained large language model, and use the cross-entropy loss function for supervised instruction fine-tuning to obtain the base model.

5. The method for training a large remote sensing visual language model based on unified reinforcement learning according to claim 1, characterized in that, Step S3 includes the following steps: Step S31: Calculate the quantity reward based on the difference between the predicted quantity and the actual quantity output by the model. Step S32: Calculate the detection reward based on the intersection-union ratio of the predicted bounding box and the ground truth bounding box output by the model. Step S33: Calculate the binary reward based on the consistency between the predicted category and the true category output by the model. Step S34: Calculate the format reward based on whether the structured format output by the model conforms to the predefined specifications; Step S35: At least two of the quantity reward, detection reward, binary reward and format reward are weighted and merged to obtain a unified reward value.

6. The method for training a large remote sensing visual language model based on unified reinforcement learning according to any one of claims 1-5, characterized in that, Step S4 includes the following steps: Step S41, query the given multimodal input. Generate a set using the current policy model Different output sequences ; Step S42: Calculate the reward value for each output sequence using the unified reward module to obtain the reward set. ; Step S43: Calculate the relative advantage value for each output based on the mean and standard deviation of the reward values ​​within the group; Step S44: Construct an objective function based on the relative advantage value, and update the parameters of the current policy model by combining the KL divergence constraint of the basic model, so that the model tends to generate high reward output.

7. The method for training a large remote sensing visual language model based on unified reinforcement learning according to claim 6, characterized in that, The objective function is: ， in, For the current strategy model, For the aforementioned basic model, To enhance the probability ratio between the old and new strategies during fine-tuning training, For the strategy model in the sampling phase, Let be the relative advantage value of the i-th output. The cutoff factor is , For KL divergence weights, for The weighting coefficients.

8. A training device for a large-scale remote sensing visual language model based on unified reinforcement learning, characterized in that, include: The preprocessing module is used to preprocess the input remote sensing images and text commands, extracting visual features and text features respectively. The instruction fine-tuning module is used to perform modal alignment of the visual features and text features, input them into a pre-trained large language model for supervised instruction fine-tuning, and obtain the base model. A reward calculation module is used to construct a multidimensional deterministic unified reward module based on truth values. The unified reward module includes at least two of the following: quantity reward, detection reward, binary reward, and format reward, and is used to evaluate the quality of the model output. The enhancement and fine-tuning module is used to enhance and fine-tune the basic model based on the group relative strategy optimization algorithm. It uses the unified reward module to calculate the reward value of the output within the group, updates the model parameters based on the relative advantage, and obtains the optimized remote sensing visual language large model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the remote sensing visual language large model training method based on unified reinforcement learning as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the remote sensing visual language large model training method based on unified reinforcement learning as described in any one of claims 1-7.

Images