Three-dimensional space refers to reasoning method, device, electronic equipment and storage medium

Abstract

The application provides a three-dimensional space reference reasoning method and device, electronic equipment and storage medium, and relates to the technical field of data processing. The method comprises the following steps: acquiring RGB-D image data of a target scene and a natural language instruction containing a space constraint; wherein the RGB-D image data is multi-modal image data containing color visual information and depth information; inputting the RGB-D image data and the natural language instruction into a pre-trained visual language large model to output a text containing an explicit reasoning process and target point coordinates conforming to the space constraint; wherein the visual language large model is obtained through two-stage supervised learning fine-tuning enhanced by depth alignment and space understanding and reinforcement learning fine-tuning based on the explicit reasoning process; and the visual language large model comprises an independent depth encoder used for processing depth information. Through the method provided by the application, the comprehensive performance in a complex space reference task is improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a three-dimensional spatial referential reasoning method, apparatus, electronic device, and storage medium. Background Technology

[0002] In recent years, with the expanding applications of robots and intelligent systems in daily life and industry, spatial intelligence in open-world environments has become a research hotspot. To enable robots to interact accurately in complex scenarios, a thorough understanding and reasoning of the three-dimensional space within the environment is necessary, with "spatial referring" being one of the core issues. Spatial referring refers to the ability of a system to accurately predict target points that satisfy specific spatial relationships, given multimodal sensory information (such as RGB or RGB-D images) and text instructions with spatial constraints, thus providing a basis for downstream functions (such as navigation, grasping, and placement).

[0003] However, existing technologies have significant shortcomings in single-step spatial perception, multi-step complex reasoning, and the fusion of depth and RGB modalities, making it difficult to simultaneously achieve high-precision recognition and reasoning of complex three-dimensional spatial relationships. Furthermore, the lack of support for explicit reasoning processes limits the model's adaptability in open-world environments.

[0004] How to enhance the three-dimensional spatial understanding and multi-step reasoning ability of multimodal large models for complex instructions, thereby effectively improving their three-dimensional spatial referential reasoning ability, is a technical problem that needs to be solved. Summary of the Invention

[0005] This invention provides a three-dimensional spatial referential reasoning method, device, electronic device, and storage medium to address the deficiencies in the prior art.

[0006] This invention provides a three-dimensional spatial referential reasoning method, comprising the following steps:

[0007] Acquire RGB-D image data of the target scene and natural language instructions containing spatial constraints; wherein, the RGB-D image data is: multimodal image data containing color visual information and depth information;

[0008] The RGB-D image data and the natural language instructions are input into a pre-trained visual language large model, and the output includes text containing explicit reasoning process and target point coordinates that conform to spatial constraints.

[0009] The visual language large model is obtained through joint training of two-stage supervised learning fine-tuning enhanced by depth alignment and spatial understanding and reinforcement learning fine-tuning based on a display reasoning process; the visual language large model comprises an independent depth encoder, and the depth encoder is used for processing the depth information.

[0010] According to the three-dimensional space reference reasoning method provided in the application, the visual language large model further comprises a picture encoder, a multi-modal projection layer and a large language model; the multi-modal projection layer comprises a picture projection layer and a depth projection layer.

[0011] The RGB-D image data and the natural language instruction are input into the pre-trained visual language large model, and text containing an explicit reasoning process and target point coordinates meeting spatial constraints are output, which comprises the following steps:

[0012] The image features of the RGB-D image data are extracted through the picture encoder, and the depth features of the RGB-D image data are extracted through the depth encoder;

[0013] The image features and the depth features are respectively mapped to a feature space matched with the large language model through the picture projection layer and the depth projection layer;

[0014] Based on the projected image features, the projected depth features and text features extracted according to the natural language instruction, the large language model outputs text containing an explicit reasoning process and target point coordinates meeting spatial constraints.

[0015] According to the three-dimensional space reference reasoning method provided in the application, the process of the two-stage supervised learning fine-tuning comprises the following steps:

[0016] A training data set with RGB-D annotation is obtained, and the depth projection layer parameters are updated based on the training data set to realize depth alignment;

[0017] A data set containing three-dimensional annotations and an explicit reasoning process and a general instruction following data set are obtained, and the picture encoder parameters, the depth encoder parameters, the projection layer parameters and the large language model parameters are jointly updated based on the data set containing three-dimensional annotations and an explicit reasoning process and the general instruction following data set to realize spatial understanding enhancement.

[0018] According to the three-dimensional space reference reasoning method provided in the application, the process of the reinforcement learning fine-tuning comprises the following steps:

[0019] acquire historical RGB-D image data and historical natural language instructions, and input the historical RGB-D image data and the historical natural language instructions into a supervised learning fine-tuned visual large language model to generate a plurality of candidate reasoning paths; wherein each candidate reasoning path in the plurality of candidate reasoning paths comprises text containing an explicit reasoning process and target point coordinates conforming to spatial constraints;

[0020] For each candidate reasoning path in the plurality of candidate reasoning paths, a pre-designed reward function is used for evaluation to obtain a reward value;

[0021] Based on the reward value of each candidate reasoning path, a group relative policy optimization (GRPO) algorithm is used to optimize the supervised learning fine-tuned visual large language model to obtain a trained visual language large model.

[0022] According to the three-dimensional space reference reasoning method provided by the application, the reward function comprises a result type reward function and a process type reward function;

[0023] For each candidate reasoning path in the plurality of candidate reasoning paths, a pre-designed reward function is used for evaluation to obtain a reward value, comprising:

[0024] Based on each candidate reasoning path in the plurality of candidate reasoning paths, the result type reward function is used to obtain a format reward and a coordinate prediction accuracy reward; wherein the format reward and the coordinate prediction accuracy reward are used to evaluate the compliance of the output format and the closeness of the target point coordinates to the real target point;

[0025] Based on each candidate reasoning path in the plurality of candidate reasoning paths, the process type reward function is used to obtain an explicit step format and a key step accuracy; wherein the explicit step format and the key step accuracy are used to evaluate the correctness of the display reasoning process;

[0026] Based on the format reward and the coordinate prediction accuracy reward, and the explicit step format and the key step accuracy, the reward value is obtained.

[0027] According to the three-dimensional space reference reasoning method provided by the application, the reward value of each candidate reasoning path is normalized to calculate the relative advantage value of each candidate reasoning path;

[0028] For each candidate reasoning path in the plurality of candidate reasoning paths, a pre-designed reward function is used for evaluation to obtain a reward value;

[0029] Based on the relative advantage value of each candidate reasoning path, a policy update is performed, and a KL divergence regularization term is added in the policy update process to limit the offset degree between the new policy and the reference policy, so as to obtain the trained visual language large model.

[0030] The application further provides a three-dimensional space reference reasoning device, comprising the following modules.

[0031] An acquisition module is configured to acquire RGB-D image data of a target scene and a natural language instruction containing a space constraint.

[0032] A reasoning module is configured to input the RGB-D image data and the natural language instruction into a pre-trained visual language large model, and output text containing an explicit reasoning process and target point coordinates meeting the space constraint.

[0033] The visual language large model is obtained through two-stage supervised learning fine-tuning enhanced by depth alignment and space understanding and reinforcement learning fine-tuning based on the explicit reasoning process.

[0034] According to the three-dimensional space reference reasoning device provided by the application, the visual language large model further comprises a picture encoder, a multi-modal projection layer and a large language model.

[0035] The reasoning module is specifically configured to:

[0036] The picture encoder is configured to extract image features of the RGB-D image data, and the depth encoder is configured to extract depth features of the RGB-D image data.

[0037] The image features and the depth features are respectively mapped to a feature space matched with the large language model through the picture projection layer and the depth projection layer.

[0038] Based on the projected image features, the projected depth features and text features extracted according to the natural language instruction, the large language model outputs text containing an explicit reasoning process and target point coordinates meeting the space constraint.

[0039] The application further provides an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the three-dimensional space reference reasoning method according to any one of the above embodiments when executing the computer program.

[0040] The application further provides a non-transitory computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the three-dimensional space reference reasoning method according to any one of the above.

[0041] The application further provides a computer program product comprising a computer program, which, when executed by a processor, implements the three-dimensional space reference reasoning method according to any one of the above.

[0042] The application provides a three-dimensional space reference reasoning method, device, electronic equipment and storage medium. The method comprises the following steps: obtaining RGB-D image data of a target scene and a natural language instruction containing a space constraint; wherein the RGB-D image data is multi-modal image data containing color visual information and depth information; inputting the RGB-D image data and the natural language instruction into a pre-trained visual language large model to output text containing an explicit reasoning process and target point coordinates conforming to the space constraint; wherein the visual language large model is obtained through two-stage supervised learning fine-tuning enhanced by depth alignment and space understanding and reinforcement learning fine-tuning based on the explicit reasoning process; and the visual language large model comprises an independent depth encoder for processing the depth information. Therefore, the three-dimensional perception accuracy is enhanced by introducing an independent depth encoder in the single-step space understanding stage, and multi-step space reasoning is realized by using the reinforcement learning fine-tuning strategy based on the explicit reasoning process, thereby greatly improving the comprehensive performance in complex space reference tasks. Therefore, the position recognition and reasoning accuracy of the model in an open environment can be significantly improved, and the needs of various downstream robot application requirements such as navigation, grasping and placing can be effectively met. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the application or prior art, the following will briefly introduce the drawings needed in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0044] Figure 1 is a flowchart of the three-dimensional space reference reasoning method provided by the application.

[0045] Figure 2 is a complete flowchart of the three-dimensional space reference reasoning method provided by the application.

[0046] Figure 3 is a structural schematic diagram of the three-dimensional space reference reasoning device provided by the application.

[0047] Figure 4Fig. 1 is a structural schematic diagram of an electronic device provided by the present application. DETAILED DESCRIPTION

[0048] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in conjunction with the drawings in the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work belong to the protection scope of the present application.

[0049] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the present application. Figures 1-4 A three-dimensional space reference reasoning method and device, an electronic device and a storage medium are described.

[0050] It should be noted that the prior art still has the following deficiencies, which directly affect the effect and scalability of space indication in practical application:

[0051] 1. Only single-step space understanding is focused on, and multi-step reasoning ability is lacking.

[0052] At present, most researches are limited to the identification of single or simple spatial relationships in a scene, and less in-depth exploration of multi-step reasoning involved in complex instructions. In other words, the research focus is more on matching the spatial position information or basic orientation features of objects with text information, but it is often difficult to support multi-stage combined thinking. In an open world scene, spatial descriptions often contain multiple constraints, such as "first locate the nearest dinner plate, then find the soy sauce dish next to it", and existing methods lack complete reasoning of such multi-level instructions, making it difficult for the system to accurately obtain target coordinates.

[0053] 2. The use of depth information is insufficient or inappropriate, which can easily cause modal interference.

[0054] Space reference relies on accurate single-step space understanding. Although existing methods enhance this ability by fusing 3D modalities, some methods require expensive three-dimensional reconstruction of multi-view images, resulting in high computational and data acquisition costs. In addition, some methods use RGB-D data to directly treat depth as an equivalent input of RGB and use the same image encoder for processing, but this can cause mutual interference between RGB and depth modalities, reducing the overall performance of the encoder. At the same time, in order to alleviate this interference, additional training data is often needed, which will further increase the cost in practice and limit the adaptability of the model to complex scenes.

[0055] 3. Lack of explicit reasoning process, which can easily cause insufficient model generalization ability.

[0056] Currently, most multi-modal large models rely on full-supervised fine-tuning (SFT) to learn implicit reasoning processes when performing spatial reference tasks. During training, the model is prone to "remembering" the answers to training samples, making it difficult to make reliable predictions for unseen scenarios or more complex instructions. In a broadcast or invisible scenario, without explicit reasoning steps, errors in the reasoning process are difficult to detect and correct, which also limits the model's generalization performance and accuracy in real-world applications.

[0057] Therefore, the prior art has obvious deficiencies in single-step spatial perception, multi-step complex reasoning, and the way of fusing depth and RGB modalities, making it difficult to balance high-precision identification and reasoning of complex three-dimensional spatial relationships. At the same time, the lack of support for explicit reasoning processes also limits the model's ability to adapt to open-world environments. Based on this, the present application provides a three-dimensional spatial reference reasoning method to solve at least one of the above problems.

[0058] Figure 1 is a flowchart of the three-dimensional spatial reference reasoning method provided by the present application, as Figure 1 shown, the method comprises the following steps:

[0059] Step 100, obtaining RGB-D image data of a target scene and a natural language instruction containing spatial constraints; wherein the RGB-D image data is multi-modal image data containing color vision information and depth information.

[0060] It should be noted that the core idea of the present embodiment is to solve the problem that existing spatial reference reasoning cannot balance accurate three-dimensional perception and multi-step complex reasoning. A visual language large model (VLM) for three-dimensional scenes with an explicit reasoning mechanism, named RoboRefer, is proposed. By introducing an independent depth encoder in the single-step spatial understanding stage to enhance three-dimensional perception accuracy, and using reinforcement learning fine-tuning (RFT) based on explicit reasoning process to realize multi-step spatial reasoning, the comprehensive performance in complex spatial reference tasks is greatly improved. Thus, the position recognition and reasoning accuracy of the model in an open environment can be significantly improved, effectively meeting the needs of various downstream robot applications such as navigation, grasping, and placing.

[0061] Step 200, inputting the RGB-D image data and the natural language instruction into a pre-trained visual language large model to output text containing an explicit reasoning process and target point coordinates conforming to spatial constraints;

[0062] The visual language large model is obtained through joint training of two-stage supervised learning fine-tuning enhanced by depth alignment and space understanding and reinforcement learning fine-tuning based on display reasoning process; the visual language large model comprises an independent depth encoder, and the depth encoder is used for processing the depth information.

[0063] It should be noted that the visual language large model further comprises a picture encoder, a multi-modal projection layer and a large language model; the multi-modal projection layer comprises a picture projection layer and a depth projection layer.

[0064] The step 200 specifically comprises:

[0065] The step 210 comprises extracting image features of the RGB-D image data through the picture encoder and extracting depth features of the RGB-D image data through the depth encoder.

[0066] The step 220 comprises mapping the image features and the depth features to a feature space matched with the large language model through the picture projection layer and the depth projection layer respectively.

[0067] The step 230 comprises outputting, through the large language model, text containing an explicit reasoning process and target point coordinates conforming to a space constraint based on the projected image features, the projected depth features and text features extracted according to the natural language instruction.

[0068] In one embodiment, a visual language large model (VLM) facing a three-dimensional scene and having an explicit reasoning mechanism is proposed, named RoboRefer, which can realize single-step accurate three-dimensional space perception and multi-step complex reasoning at the same time. The model comprises the following key modules:

[0069] 1. Picture encoder: used for extracting features from color images (RGB), retaining the representation ability of existing pre-trained image encoders.

[0070] 2. Depth encoder: independent of the RGB encoder, used for processing depth (Depth) information, capable of explicitly capturing three-dimensional information such as distance, orientation and size change in the scene, avoiding interference with RGB features.

[0071] 3. Multi-modal projection layer (picture projection layer and depth projection layer): projecting the features output by the picture and depth encoders to a feature space matched with the large language model, so that the large language model can realize consistent alignment of semantics and vision when receiving multi-modal input.

[0072] 4. Large Language Model (LLM): As the core understanding and reasoning engine, it comprehensively understands the scene and instructions through interaction with multi-modal features. Based on this language model, the model can output location predictions (coordinate points) or question and answer results, or generate explicit textual reasoning steps during the reasoning process.

[0073] After receiving the "instruction with spatial constraints" and the RGB-D input of the scene, the model first encodes the depth information independently, and then multi-modal fusion with text features. The subsequent output can contain an explicit process of step-by-step reasoning, and can also finally obtain a target point prediction that meets the spatial relationship constraints, thereby meeting the demand for spatial reference tasks in complex three-dimensional scenes to achieve diversified downstream tasks such as navigation, grasping, and placing.

[0074] To achieve accurate three-dimensional perception and multi-step reasoning, this embodiment adopts a two-stage training strategy: supervised fine-tuning (SFT) and reinforcement learning fine-tuning (RFT). These two stages are used together to integrate three-dimensional information, explicit reasoning process, and high-accuracy prediction capability while ensuring good generalization and robustness in open environments. The training process of the visual language large model provided in this embodiment is described in detail below.

[0075] The process of the two-stage supervised learning fine-tuning includes:

[0076] Step S210, obtain a training data set with RGB-D annotation, and update the depth projection layer parameters based on the training data set to realize depth alignment.

[0077] Step S220, obtain a data set containing three-dimensional annotations and explicit reasoning processes, and a general instruction following data set, and jointly update the picture encoder parameters, the depth encoder parameters, the projection layer parameters, and the large language model parameters based on the data set containing three-dimensional annotations and explicit reasoning processes and the general instruction following data set to realize spatial understanding enhancement.

[0078] Supervised fine-tuning (SFT) includes the following two steps:

[0079] 1. Depth alignment.

[0080] For the newly added depth encoder, the text space and depth feature space need to be aligned. For this purpose, the present invention selects a training data set with RGB-D annotation (for example, RefSpatial), and projects the depth features to a semantic space matching the language model by only updating the "depth projection layer".

[0081] In this process, the picture encoder and language model parameters do not change, thereby avoiding interference of depth information with the existing pre-trained encoder.

[0082] 2. Enhanced spatial understanding ability.

[0083] After the completion of the depth alignment, the model is further fine-tuned using the combined picture and depth input, including:

[0084] The model's understanding of spatial dimensions is reinforced using data containing detailed three-dimensional annotations and explicit multi-step reasoning processes, such as the multi-step spatial reference examples in RefSpatial.

[0085] An additional instruction-following dataset is introduced to enable the model to have more general instruction understanding capabilities.

[0086] Through the above training steps, the model retains its original language understanding and visual question answering (VQA) capabilities while further acquiring accurate perception of three-dimensional attributes such as distance, direction, and location. It also has a preliminary implicit grasp of multi-step spatial reasoning, although there may still be issues with memory generalization. Therefore, this stage of SFT lays the foundation for subsequent explicit reasoning enhancement.

[0087] The reinforcement learning fine-tuning process includes:

[0088] Step S230, historical RGB-D image data and historical natural language instructions are obtained, and the historical RGB-D image data and the historical natural language instructions are input into the supervised learning fine-tuned visual large language model to generate a plurality of candidate reasoning paths; wherein each candidate reasoning path in the plurality of candidate reasoning paths includes: text containing an explicit reasoning process and target point coordinates conforming to spatial constraints.

[0089] Step S240, for each candidate reasoning path in the plurality of candidate reasoning paths, a pre-designed reward function is used for evaluation to obtain a reward value.

[0090] It should be noted that the reward function includes a result class reward function and a process class reward function.

[0091] Step S240 specifically includes:

[0092] Step S241, based on each candidate reasoning path in the plurality of candidate reasoning paths, a format reward and a coordinate prediction accuracy reward are obtained through the result class reward function; wherein the format reward and the coordinate prediction accuracy reward are used to evaluate the compliance of the output format and the closeness of the target point coordinates to the true target point.

[0093] Step S242, based on each candidate reasoning path in the plurality of candidate reasoning paths, obtaining an explicit step format and a key step accuracy through the process class reward function; wherein the explicit step format and the key step accuracy are used to evaluate the correctness of the display reasoning process.

[0094] Step S243, based on the format reward and the coordinate prediction accuracy reward, and the explicit step format and the key step accuracy, obtaining the reward value.

[0095] Step S250, based on the reward value of each candidate reasoning path, optimizing the supervised learning fine-tuned visual large language model through a grouping relative strategy optimization (GRPO) algorithm to obtain a trained visual language large model.

[0096] Step S250 specifically comprises:

[0097] Step S251, normalizing the reward value of each candidate reasoning path to calculate a relative advantage value of each candidate reasoning path.

[0098] Step S252, updating the strategy based on the relative advantage value of each candidate reasoning path, and adding a KL divergence regularization term in the strategy updating process to limit the offset degree between the new strategy and the reference strategy, to obtain the trained visual language large model.

[0099] After the SFT is completed, the model has a certain multi-step reasoning ability, but still tends to rely on a "memory" mode and is difficult to accurately reason about novel or more complex spatial constraints. Therefore, the present application further designs a reinforcement learning fine-tuning (RFT) based on reinforcement learning to optimize the explicit reasoning process. The main process includes:

[0100] 1. Multi-step output of the explicit reasoning process.

[0101] For complex spatial reference tasks, the complete reasoning process is divided into multiple key steps; each step can output structured text (such as "[perception type] [target object]:..."), which contains judgment information about a specific target or location, such as Figure 1 the "thinking process" of.

[0102] This facilitates the measurement and reward of "process-level correctness" in training, avoiding reliance on only the final prediction result.

[0103] 2. Design of reward function.

[0104] Outcome reward:

[0105] including structured reasoning output format reward (ROF) and coordinate prediction accuracy reward (RP) to measure the format compliance of the final output and the proximity of the coordinates to the true target points.

[0106] Process Reward:

[0107] including explicit step format (RPF) and key step accuracy (RAcc) to measure the correctness of the reasoning process and achieve fine guidance for intermediate reasoning steps.

[0108] By combining result reward and process reward, the model is encouraged to make accurate spatial judgments at each step and give the correct answer in the final prediction.

[0109] 3. Reinforcement learning update based on GRPO.

[0110] The model is optimized using Group Relative Policy Optimization (GRPO). Specifically, several reasoning processes and prediction results are sampled (i.e. multi-thread exploration), the corresponding reward values are calculated by comparing the reward functions, and the relative advantage is used for policy update.

[0111] At the same time, in order to ensure the stability of the model and avoid deviating too far from the initial SFT strategy, a KL divergence regularization term is added to the loss function. This not only preserves the good prior learned in the SFT stage, but also obtains further enhancement of high-precision spatial reasoning through exploratory attempts.

[0112] Through the above training steps, the obtained model can not only obtain accurate three-dimensional perception based on independent deep encoder, but also use explicit process supervision and reinforcement learning for adaptive correction and iteration in multi-step reasoning process, so as to have higher spatial reference accuracy and better complex scene generalization ability in open scene.

[0113] The above is a step-by-step description of the three-dimensional space reference reasoning method provided by the present application. As can be seen from the description of the above steps, according to the three-dimensional space reference reasoning method provided by the present application, the RGB-D image data of the target scene and the natural language instruction containing the space constraint are obtained; wherein the RGB-D image data is multi-modal image data containing color visual information and depth information; the RGB-D image data and the natural language instruction are input into a pre-trained visual language large model, and text containing an explicit reasoning process and target point coordinates conforming to the space constraint are output; wherein the visual language large model is obtained through two-stage supervised learning fine-tuning enhanced by depth alignment and space understanding, and reinforcement learning fine-tuning based on the explicit reasoning process; the visual language large model includes an independent depth encoder, and the depth encoder is used to process the depth information. Therefore, the present application introduces an independent depth encoder in the single-step space understanding stage to enhance the three-dimensional perception accuracy, and on this basis, adopts a reinforcement learning fine-tuning strategy based on the explicit reasoning process to realize multi-step space reasoning, which greatly improves the comprehensive performance in complex space reference tasks. Therefore, the position recognition and reasoning accuracy of the model in an open environment can be significantly improved, and the needs of various downstream robot applications such as navigation, grasping and placing can be effectively met.

[0114] The core technical scheme of the present application mainly focuses on the fusion of vision and language in three-dimensional space. In view of the problem that the existing visual language model relying only on two-dimensional image mode cannot obtain accurate three-dimensional perception, a dedicated depth encoder is added in the network structure, and a targeted reinforcement fine-tuning (RFT) strategy is combined to realize a high-precision model for multi-step complex space reasoning. Figure 2 is the complete flowchart of the three-dimensional space reference reasoning method provided by the present application. The three-dimensional space reference reasoning method provided by the present application will be described in detail below. Figure 2

[0115] VLM architecture: First, the present application proposes a VLM architecture for three-dimensional scenes. Referring to Figure 2 It can be seen that the model uses an independent picture encoder and a depth encoder to extract image and depth features respectively in the input stage, and then connects through a projection layer (projector) and a large language model (LLM) to realize functions such as visual question answering (VQA) or coordinate prediction.

[0116] ​The reason for separately setting the depth encoder is that three-dimensional information is crucial for spatial understanding. If only picture images are used for VLM pre-training, there will be a lack of accurate capture of information such as depth, distance or distance relationship. Although some previous methods simply regard depth as a special image modality for joint encoding, this will cause modal interference, and additional picture data is needed for collaborative training to make up for the performance loss. In contrast, the independent depth encoder adopted in the present application is updated independently during training as follows: when starting joint fine-tuning of pictures and picture-depth, the image encoder is not disturbed by the depth input, and the depth encoder can be updated independently, thereby maximizing the retention of the model's adaptability to general visual question answering (VQA) and fully strengthening the understanding of depth information.

[0117] SFT training: on the basis of this architecture, the present application further proposes a two-stage supervised fine-tuning (SFT) scheme to enhance the model's multi-step complex reasoning ability in three-dimensional space. Let the training data set be wherein represents an RGB or RGB-D image observation, represents a natural language question, represents the corresponding answer. The model optimization objective is as follows:

[0118]

[0119] wherein, represents the output distribution under the current model parameters, represents the tth output word.

[0120] This full supervised fine-tuning SFT includes two steps of depth alignment phase and spatial understanding enhancement phase:

[0121] Depth alignment phase:

[0122] In this phase, only the mapping module of the depth feature (i.e. the depth projector) is updated, and the goal is to align the newly added depth modality with the language modality. During training, the spatial reference dataset (such as RefSpatial) with RGB-D annotation is used to realize modal alignment by minimizing the difference between the predicted result and the true answer. In specific implementation, the maximum learning rate is set to , the weight decay coefficient is 0, and the warm-up ratio is 0.03. The model with a parameter magnitude of 2B uses a batch size of 7 on each GPU, and the 8B model uses a batch size of 3, and a training cycle.

[0123] Spatial understanding enhancement phase:

[0124] At this stage, all model parameters participate in training, including the RGB encoder, depth encoder, language model, and its projection module. The training data includes not only single-step reference tasks, but also multi-step spatial reasoning tasks with explicit reasoning processes, as well as multiple instruction-following datasets (including the LLaVA-1.5 dataset, the LRV dataset, and the RefCOCO / + / g dataset). This stage aims to enhance the model's spatial understanding and explicit reasoning capabilities through a composite supervision signal, providing a good initialization for the next step of reinforcement learning fine-tuning (RFT). The training parameters for this stage are: the maximum learning rate , the batch size per GPU for the 2B model is 6, and for the 8B model is 2. Other hyperparameters remain the same as in the first stage.

[0125] RFT training: Although SFT introduces a large amount of data, even including reasoning data for multi-step thinking processes, it tends to remember the answers and is helpless in the face of more novel combinations of spatial constraints. Therefore, the present invention provides a spatial reference reasoning method based on reinforcement fine-tuning (Reinforcement Fine-Tuning, RFT), aiming to improve the model's generalization ability and reasoning accuracy in complex spatial relationship understanding and accurate point prediction. This method introduces a group relative policy optimization (Group Relative Policy Optimization, GRPO) mechanism after the supervised fine-tuning (Supervised Fine-Tuning, SFT) and combines multiple reward function designs, thereby more effectively guiding the model to generate structured and verifiable reasoning processes and high-precision prediction results.

[0126] Specifically, it includes the following steps:

[0127] 1. Reinforcement learning policy optimization framework.

[0128] First, use the initial policy model trained by SFT as the initial policy in the RFT stage . In each training iteration, the system generates N candidate actions (answers) according to the input state , where represents the visual encoding (RGB or RGB-D) of the input image, represents the encoding of the text question, which is sampled by the current policy :

[0129]

[0130] This method ensures the diversity of the sampling results to encourage policy exploration and avoid early convergence.

[0131] 2. Reward function design.

[0132] To achieve a comprehensive evaluation of the inference results and processes, the present embodiment designs a multi-reward mechanism including result-based reward functions and process-based reward functions.

[0133] (1) Result-based reward function.

[0134] Result format reward : If the model output meets the following format specifications, the reward is 1, otherwise 0:

[0135] <think>... reasoning process...< / think>

[0136] <answer>... final answer...< / answer>

[0137] Point accuracy reward : If the L1 distance between the model output point prediction result and the labeled point is less than 50 pixels, it is considered accurate, and the reward is 1, otherwise 0:

[0138]

[0139] (2) Process-based reward function.

[0140] To overcome the shortcomings of traditional process rewards relying on large language or multi-modal models (such as LLM / VLM), the present embodiment innovatively designs a rule-based process reward mechanism, directly using the key step annotation information provided in the data set, avoiding the uncertainty of external evaluators.

[0141] Process format reward : If the model output intermediate inference step meets the following structured format, the reward is 1:

[0142] [Type] [Target Object]: [Value]

[0143] Where "Type" is limited to "Position", "Orientation", or "Size"; "Target Object" is a uniquely identifiable entity; and "Value" depends on the type:

[0144] If Position: Format is [(x, y)], where x, y ∈ [0, 1]

[0145] If Orientation: Format is a three-dimensional unit vector (x, y, z)

[0146] If Size: Value is a scalar in meters

[0147] Examples include:

[0148] [Position] [Second largest cup]: [(0.245, 0.147)]

[0149] [Orientation] [Cup handle]: (1.000, 0.000, 0.000)

[0150] Precision reward : Only evaluate the perception output in the key step. If the predicted target matches the annotation and the error between the predicted value and the true value is within the allowable range, a reward is given. For example:

[0151] For "position" class perception, if the L1 distance < 50 pixels, the reward is 1;

[0152] For "orientation" class, if the cosine similarity > 0.8, the reward is 1;

[0153] For "size" class, if the predicted value is within the true value ± 15% range, the reward is 1.

[0154] Defined as follows:

[0155]

[0156]

[0157]

[0158] 3. Reward normalization and policy update.

[0159] The total reward of each group of sampled actions is composed of four parts:

[0160]

[0161] Among them, the process class reward is multiplied by a coefficient of 0.25 to prevent it from accumulating too high in multi-step reasoning.

[0162] Then the reward inside each group is normalized to calculate the relative advantage value:

[0163]

[0164] Finally, the relative advantage value is used for policy update to enhance the probability of high-quality actions and suppress the generation of low-quality actions. At the same time, to ensure the stability of the training process, a KL divergence regularization term is added in the policy update to limit the offset degree between the new policy and the reference policy.

[0165] Notably, the SFT initialization stage provides strong prior knowledge for the model, enabling it to quickly adapt to the output format and effectively support accurate step-by-step spatial reasoning by leveraging the spatial understanding learned in the SFT stage. As shown in Figure 2, the model trained through RFT demonstrates good generalization ability in tasks such as four-step spatial reference, enabling it to handle complex spatial relationships step by step and generate precise point predictions.

[0166] The three-dimensional spatial reference reasoning method provided by the present application has the following beneficial effects:

[0167] 1. Significantly improved three-dimensional spatial perception accuracy.

[0168] Advantages: The present application addresses the three-dimensional spatial perception needs in the robot interaction process, and effectively improves the recognition accuracy of spatial attributes such as target position, orientation, and depth by using an independent depth encoder. Compared with some existing solutions that simply treat depth data as RGB input or require high-cost multi-view three-dimensional reconstruction, the present application significantly reduces the interference and learning difficulty between modalities through a special depth branch and image branch architecture that does not interfere with each other and cooperates with each other.

[0169] Reasons:

[0170] 1) Special depth encoder: avoids modal conflicts caused by mixing depth and RGB features, and enhances accurate capture of spatial information;

[0171] 2) Reduce cumbersome three-dimensional reconstruction: saves large-scale acquisition and processing of multi-view data, and avoids high computational cost and error accumulation.

[0172] 2. Comprehensive ability to support multi-step spatial reasoning.

[0173] Advantages: The present application introduces a reinforcement learning fine-tuning (RFT) stage after supervised fine-tuning (SFT), explicitly decomposes and performs multi-step reasoning. This strategy breaks through the previous limitation of mainly focusing on single-step spatial understanding, and can better handle complex tasks involving multiple spatial constraints or sequential logic.

[0174] Reasons:

[0175] 1) Explicit reasoning process labeling: labeling and measuring the reasoning process in the RFT stage, enabling the model to deduce complex "spatial reference" chains step by step;

[0176] 2) Metric-sensitive reward function: The application designs a metric-sensitive reward function for inference accuracy, guiding the model to continuously correct the predicted point position in multi-step reasoning, reducing error accumulation and improving overall accuracy.

[0177] 2) Stronger open-world generalization and accuracy.

[0178] Advantages: Through the above improved three-dimensional perception and multi-step reasoning ability, the application has achieved leading performance on both existing single-step spatial understanding test sets (such as CV-Bench, BLINK) and specially constructed multi-step spatial task test sets (RefSpatial-Bench). The results show that the application can maintain high-level prediction accuracy and generalization performance in multiple scenarios and tasks.

[0179] Reasons:

[0180] 1) Single-step and multi-step reasoning advantages: not only performs well in basic spatial target recognition tasks, but also can handle complex scenes with multiple conditions and reasoning decomposition;

[0181] 2) Multi-type data and real scene evaluation: RefSpatial-Bench covers multiple real images and up to five-step reasoning depth requirements, which can fully test the robustness of the model in long-term and multi-constraint scenarios.

[0182] 4) Adaptation and expandability to multiple downstream tasks.

[0183] Advantages: The application not only can be used as a general visual assistant, but also can provide high-level spatial planning instructions for specific robot platforms (such as robotic arms, humanoid robots), including grasping, placing, navigation path planning and other operations. Multi-module collaboration can flexibly adapt to different task requirements, with high portability and expansion potential.

[0184] Reasons:

[0185] 1) Universal spatial reference expression: through clear spatial constraint semantics and multi-step reasoning process, the application has the potential to be easily extended to multiple hardware systems;

[0186] 2) Multi-strategy control interface: seamlessly connects between different robot control strategies (such as operation arm path planning, mobile platform navigation), further improving the overall efficiency of the system.

[0187] 5) Balance of high performance and safety

[0188] Advantages: While significantly enhancing spatial perception, reasoning and generalization ability, the application also takes into account the safety considerations in complex operation situations. Through explicit tracking and evaluation of the reasoning process, the decision can be optimized or safety threshold can be set when necessary, reducing the potential risks in practical applications.

[0189] Reasons:

[0190] 1) Reasoning interpretability: multi-step reasoning makes the decision-making process transparent, facilitating auditing or adding safety redundancy in safety-related applications.

[0191] 2) Open development and safety evaluation: after the model is opened to the relevant research community, more usage feedback and safety testing can be continuously obtained, improving the overall reliability of the system.

[0192] The three-dimensional space reference reasoning device provided by the application is described below, and the three-dimensional space reference reasoning device described below can be referred to each other with the three-dimensional space reference reasoning method described above.

[0193] Figure 3 is a structural schematic diagram of the three-dimensional space reference reasoning device provided by the application, as Figure 3 indicated, the three-dimensional space reference reasoning device provided by the application comprises:

[0194] The acquisition module 301 is configured to acquire RGB-D image data of a target scene and natural language instructions containing spatial constraints; wherein the RGB-D image data is multi-modal image data containing color vision information and depth information;

[0195] The reasoning module 302 is configured to input the RGB-D image data and the natural language instructions into a pre-trained visual language large model, and output text containing an explicit reasoning process and target point coordinates conforming to the spatial constraints;

[0196] The visual language large model is obtained by joint training of two-stage supervised learning fine-tuning enhanced by depth alignment and spatial understanding and reinforcement learning fine-tuning based on the explicit reasoning process; the visual language large model comprises an independent depth encoder, and the depth encoder is configured to process the depth information.

[0197] The three-dimensional space reference reasoning device provided by the application comprises a target scene RGB-D image data acquisition module, a natural language instruction acquisition module, a reasoning module and a target point coordinate output module.

[0198] In the embodiment, the visual language large model further comprises a picture encoder, a multi-modal projection layer and a large language model.

[0199] The reasoning module 302 is specifically configured to:

[0200] The picture encoder is configured to extract image features of the RGB-D image data, and the depth encoder is configured to extract depth features of the RGB-D image data.

[0201] The image features and the depth features are respectively mapped to a feature space matched with the large language model through the picture projection layer and the depth projection layer.

[0202] The large language model is configured to output text containing an explicit reasoning process and target point coordinates conforming to a space constraint based on the projected image features, the projected depth features and text features extracted according to the natural language instruction.

[0203] In the embodiment, the process of the two-stage supervised learning fine-tuning comprises:

[0204] A training data set with RGB-D annotation is acquired, and the depth projection layer parameters are updated based on the training data set to realize depth alignment.

[0205] obtaining a dataset containing three-dimensional annotations and explicit reasoning processes and a general instruction following dataset, and jointly updating the picture encoder parameters, the depth encoder parameters, the projection layer parameters, and the large language model parameters based on the dataset containing three-dimensional annotations and explicit reasoning processes and the general instruction following dataset to achieve spatial understanding enhancement.

[0206] Based on the above embodiment, in this embodiment, the process of reinforcement learning fine-tuning includes:

[0207] obtaining historical RGB-D image data and historical natural language instructions, and inputting the historical RGB-D image data and the historical natural language instructions into the supervised learning fine-tuned visual large language model to generate a plurality of candidate reasoning paths; wherein each candidate reasoning path in the plurality of candidate reasoning paths includes: text containing an explicit reasoning process and target point coordinates conforming to spatial constraints;

[0208] For each candidate reasoning path in the plurality of candidate reasoning paths, a pre-designed reward function is used for evaluation to obtain a reward value;

[0209] Based on the reward value of each candidate reasoning path, the supervised learning fine-tuned visual large language model is optimized by a grouping relative policy optimization (GRPO) algorithm to obtain a trained visual language large model.

[0210] Based on the above embodiment, in this embodiment, the reward function includes: a result class reward function and a process class reward function;

[0211] The device further includes an evaluation module, specifically configured to:

[0212] Based on each candidate reasoning path in the plurality of candidate reasoning paths, a format reward and a coordinate prediction accuracy reward are obtained through the result class reward function; wherein the format reward and the coordinate prediction accuracy reward are used to evaluate the compliance of the output format and the closeness of the target point coordinates to the real target point;

[0213] Based on each candidate reasoning path in the plurality of candidate reasoning paths, an explicit step format and a key step accuracy are obtained through the process class reward function; wherein the explicit step format and the key step accuracy are used to evaluate the correctness of the explicit reasoning process;

[0214] Based on the format reward and the coordinate prediction accuracy reward, and the explicit step format and the key step accuracy, the reward value is obtained.

[0215] Based on the above embodiment, in this embodiment, the device further includes an evaluation module, specifically configured to:

[0216] normalizing the reward value of each candidate reasoning path to obtain a relative advantage value of each candidate reasoning path;

[0217] performing policy updating based on the relative advantage value of each candidate reasoning path, and adding a KL divergence regularization term in the policy updating process to limit the offset degree between the new policy and the reference policy, to obtain the trained visual language large model.

[0218] Figure 4 An example of an entity structure diagram of an electronic device is shown in FIG. 1, which can be a robot or other electronic device. The electronic device can include a processor 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other through the communications bus 440. The processor 410 can invoke logical instructions in the memory 430 to execute a three-dimensional space reference reasoning method, including: Figure 4

[0219] obtaining RGB-D image data of a target scene and a natural language instruction containing spatial constraints; wherein the RGB-D image data is multi-modal image data containing color vision information and depth information;

[0220] inputting the RGB-D image data and the natural language instruction into a pre-trained visual language large model to output text containing an explicit reasoning process and target point coordinates conforming to spatial constraints;

[0221] wherein the visual language large model is obtained by joint training of two-stage supervised learning fine-tuning enhanced by depth alignment and spatial understanding and reinforcement learning fine-tuning based on the explicit reasoning process; the visual language large model includes an independent depth encoder, and the depth encoder is used to process the depth information.

[0222] ​Moreover, the logic instructions in the memory 430 described above can be implemented in the form of software functional units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0223] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor to enable a computer to execute the three-dimensional space reference reasoning method provided by the above-mentioned methods, comprising:

[0224] obtaining RGB-D image data of a target scene and a natural language instruction containing spatial constraints; wherein the RGB-D image data is multi-modal image data containing color vision information and depth information;

[0225] inputting the RGB-D image data and the natural language instruction into a pre-trained visual language large model to output text containing an explicit reasoning process and target point coordinates conforming to the spatial constraints;

[0226] wherein the visual language large model is obtained by joint training of two-stage supervised learning fine-tuning enhanced by depth alignment and spatial understanding and reinforcement learning fine-tuning based on the explicit reasoning process; the visual language large model comprises an independent depth encoder, and the depth encoder is used to process the depth information.

[0227] In another aspect, the present application also provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the three-dimensional space reference reasoning method provided by the above-mentioned methods, comprising:

[0228] obtaining RGB-D image data of a target scene and a natural language instruction containing spatial constraints; wherein the RGB-D image data is multi-modal image data containing color vision information and depth information;

[0229] input the RGB-D image data and the natural language instruction into a pre-trained visual language large model, output text containing an explicit reasoning process and target point coordinates conforming to spatial constraints;

[0230] The visual language large model is obtained through joint training of two-stage supervised learning fine-tuning enhanced by depth alignment and spatial understanding and reinforcement learning fine-tuning based on the explicit reasoning process; the visual language large model comprises an independent depth encoder configured to process the depth information.

[0231] The device embodiments described above are merely illustrative, wherein the units illustrated as separate components can or can not be physically separated, and the components illustrated as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0232] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and necessary universal hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0233] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A three-dimensional spatial referential reasoning method, characterized in that, The method comprises the following steps: obtaining RGB-D image data of a target scene and a natural language instruction containing spatial constraints; wherein the RGB-D image data is multi-modal image data containing color vision information and depth information; inputting the RGB-D image data and the natural language instruction into a pre-trained visual language large model to output text containing an explicit reasoning process and target point coordinates conforming to spatial constraints; wherein the visual language large model is obtained through joint training of two-stage supervised learning fine-tuning enhanced by depth alignment and spatial understanding and reinforcement learning fine-tuning based on an explicit reasoning process; the visual language large model comprises an independent depth encoder for processing the depth information; the visual language large model further comprises a picture encoder, a multi-modal projection layer and a large language model; wherein the multi-modal projection layer comprises a picture projection layer and a depth projection layer; the process of the two-stage supervised learning fine-tuning comprises: obtaining a training data set with RGB-D annotation, and updating the depth projection layer parameters based on the training data set to realize depth alignment; obtaining a data set containing three-dimensional annotations and an explicit reasoning process and a general instruction following data set, and jointly updating the picture encoder parameters, the depth encoder parameters, the projection layer parameters and the large language model parameters based on the data set containing three-dimensional annotations and an explicit reasoning process and the general instruction following data set to realize spatial understanding enhancement; the process of the reinforcement learning fine-tuning comprises: obtaining historical RGB-D image data and historical natural language instructions, and inputting the historical RGB-D image data and the historical natural language instructions into the visual large model after supervised learning fine-tuning to generate a plurality of candidate reasoning paths; wherein each candidate reasoning path in the plurality of candidate reasoning paths comprises text containing an explicit reasoning process and target point coordinates conforming to spatial constraints; for each candidate reasoning path in the plurality of candidate reasoning paths, a pre-designed reward function is used for evaluation to obtain a reward value; based on the reward value of each candidate reasoning path, the supervised learning fine-tuned visual large model is optimized through a group relative policy optimization (GRPO) algorithm to obtain the trained visual language large model.

2. The three-dimensional space reference reasoning method according to claim 1, wherein the inputting the RGB-D image data and the natural language instruction into the pre-trained visual language large model to output text containing an explicit reasoning process and target point coordinates conforming to spatial constraints comprises: extracting image features of the RGB-D image data through the picture encoder and extracting depth features of the RGB-D image data through the depth encoder; mapping the image features and the depth features to a feature space matched with the large language model through the picture projection layer and the depth projection layer, respectively. ​ Based on the projected image features, the projected depth features, and the text features extracted according to the natural language instruction, the large language model outputs text containing an explicit reasoning process and target point coordinates conforming to spatial constraints. 3.The three-dimensional space reference reasoning method of claim 1, wherein, The reward function includes a result type reward function and a process type reward function. The reward value of each candidate reasoning path is obtained by evaluating the pre-designed reward function. Based on each candidate reasoning path, the result type reward function is used to obtain a format reward and a coordinate prediction accuracy reward, which are used to evaluate the compliance of the output format and the proximity of the target point coordinates to the real target point. Based on each candidate reasoning path, the process type reward function is used to obtain an explicit step format and a key step accuracy, which are used to evaluate the correctness of the explicit reasoning process. The reward value is obtained based on the format reward, the coordinate prediction accuracy reward, the explicit step format, and the key step accuracy.

4. The three-dimensional space reference reasoning method according to claim 1, wherein, Based on the reward value of each candidate reasoning path, the supervised learning fine-tuned visual large language model is optimized by a grouping relative strategy optimization (GRPO) algorithm to obtain a trained visual language large model. The reward value of each candidate reasoning path is normalized to obtain a relative advantage value of each candidate reasoning path. Based on the relative advantage value of each candidate reasoning path, the strategy is updated, and a KL divergence regularization term is added during the strategy updating process to limit the offset degree between the new strategy and the reference strategy, thereby obtaining the trained visual language large model.

5. A three-dimensional space reference reasoning apparatus characterized by comprising: The method comprises: An acquisition module is configured to acquire RGB-D image data of a target scene and a natural language instruction containing spatial constraints; the RGB-D image data is multi-modal image data containing color vision information and depth information; A reasoning module is configured to input the RGB-D image data and the natural language instruction into a pre-trained visual language large model to output text containing an explicit reasoning process and target point coordinates conforming to spatial constraints. The visual language large model is obtained by joint training of two-stage supervised learning fine-tuning enhanced by depth alignment and spatial understanding and reinforcement learning fine-tuning based on an explicit reasoning process; the visual language large model includes an independent depth encoder for processing the depth information. The visual language large model further includes a picture encoder, a multi-modal projection layer, and a large language model; the multi-modal projection layer includes a picture projection layer and a depth projection layer. The process of the two-stage supervised learning fine-tuning comprises: A training data set with RGB-D labels is obtained, and the depth projection layer parameters are updated based on the training data set to realize depth alignment. obtaining a dataset containing three-dimensional annotations and explicit reasoning processes and a general instruction following dataset, and jointly updating the picture encoder parameters, the depth encoder parameters, the projection layer parameters, and the large language model parameters based on the dataset containing three-dimensional annotations and explicit reasoning processes and the general instruction following dataset to achieve spatial understanding enhancement; The process of reinforcement learning fine-tuning includes: obtaining historical RGB-D image data and historical natural language instructions, and inputting the historical RGB-D image data and the historical natural language instructions into the supervised learning fine-tuned visual large language model to generate a plurality of candidate reasoning paths; wherein each candidate reasoning path in the plurality of candidate reasoning paths includes text containing an explicit reasoning process and target point coordinates conforming to spatial constraints; For each candidate reasoning path in the plurality of candidate reasoning paths, a pre-designed reward function is used for evaluation to obtain a reward value; Based on the reward value of each candidate reasoning path, the supervised learning fine-tuned visual large language model is optimized by a grouping relative policy optimization (GRPO) algorithm to obtain a trained visual language large model.

6. The three-dimensional spatial reference reasoning device according to claim 5, wherein The reasoning module is specifically configured to: extract image features of the RGB-D image data through the picture encoder and depth features of the RGB-D image data through the depth encoder; map the image features and the depth features to a feature space matched with the large language model through the picture projection layer and the depth projection layer respectively; output text containing an explicit reasoning process and target point coordinates conforming to spatial constraints through the large language model based on the projected image features, the projected depth features, and text features extracted according to the natural language instructions.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the three-dimensional spatial reference reasoning method according to any one of claims 1 to 4.

8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the three-dimensional spatial reference reasoning method according to any one of claims 1 to 4.

Images