Method and device for determining spatial relationship of large model target based on mind tree guidance

Abstract

The present invention provides a method and device for determining the target spatial relationship of a large model based on a mind tree, which relates to the field of artificial intelligence technology and is used to solve the technical problems that the existing large model is not interpretable when processing complex spatial reasoning tasks and the data processing efficiency is low. The method comprises: obtaining a spatial reasoning task to be processed; parsing the task description features corresponding to the spatial reasoning task according to the spatial reasoning task, wherein the task description features include story features and question features that characterize the spatial relationship of each object; inputting the task description features into a pre-trained large language model, and performing the following operations: splitting the question features into a mind tree structure with multiple nodes, wherein each node represents a mind step; according to each mind step, identifying the potential spatial relationship features between the source object and the target object in the story features; and determining the spatial relationship between the source object and the target object according to the potential spatial relationship features.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and more specifically, to a method and device for determining a large model target spatial relationship based on a mind tree guidance. Background Art

[0002] Spatial reasoning is an important cognitive ability that involves understanding and inferring information such as the position, direction, and shape of an object in three-dimensional space. This is widely needed in many practical applications, such as robot navigation, assisted driving, and augmented reality. However, spatial reasoning tasks often have some technical bottlenecks, such as: difficulty in modeling complex geometric relationships and physical dynamics, insufficient integration of spatial information and semantic reasoning, and insufficient learning ability for abstract concepts and reasoning rules. In order to solve these problems, large model technology has brought new opportunities to spatial reasoning tasks in recent years.

[0003] Large models can use massive amounts of visual and textual data to learn rich visual-linguistic knowledge representations through self-supervised learning, and can solve many logical reasoning tasks that are critical to human communication. However, complex spatial reasoning tasks usually involve multiple sub-problems, which require layer-by-layer reasoning and comprehensive analysis to arrive at the final answer. As a result, large models still have some bottlenecks when directly processing complex spatial reasoning tasks, including: black-box and weak interpretability problems, as well as limited generalization capabilities and lack of structured reasoning mechanisms, resulting in low data processing efficiency for large models. Summary of the invention

[0004] In view of this, the present invention provides a method and device for determining the target spatial relationship of a large model based on a mind tree guidance, so as to solve the technical problems that the existing large models have poor interpretability and low data processing efficiency when processing complex spatial reasoning tasks.

[0005] One aspect of the present invention provides a method for determining a target spatial relationship of a large model based on a mind tree, comprising: obtaining a spatial reasoning task to be processed; parsing a task description feature corresponding to the spatial reasoning task according to the spatial reasoning task, wherein the task description feature includes a story feature and a question feature that characterize the spatial relationship of each object; inputting the task description feature into a pre-trained large language model, and performing the following operations: splitting the question feature into a mind tree structure with multiple nodes, wherein each node represents a mind step; according to each mind step, identifying a latent spatial relationship feature between a source object and a target object in the story feature, wherein the latent spatial relationship feature includes all possible paths that can be connected to the target object starting from the source object; determining the spatial relationship between the source object and the target object according to the latent spatial relationship feature.

[0006] According to an embodiment of the present invention, according to each thinking step, identifying the latent spatial relationship features between the source object and the target object in the story features includes: taking the source object as the root node, executing each thinking step, and identifying all nodes that the source object can reach in the next step; in response to the source object reaching the target sub-node corresponding to the target object after one of the thinking steps is completed, establishing a connection relationship between the root node and the target sub-node; based on the connection relationship, generating the latent spatial relationship features between the source object and the target object.

[0007] According to an embodiment of the present invention, each thinking step has a one-to-one corresponding tree state, wherein the tree state represents the state of the spatial reasoning task at the current thinking step.

[0008] According to an embodiment of the present invention, the large language model includes: a thought generator, which is used to generate multiple candidates for the next thought step for each tree state according to the thought strategy.

[0009] According to an embodiment of the present invention, thinking strategies include: a first thinking strategy, configured to generate multiple candidate items independently and identically distributed by adopting a thinking chain reasoning method; a second thinking strategy, configured to generate multiple candidate items in a sequential prompting method.

[0010] According to an embodiment of the present invention, the large language model also includes: a state evaluator, which is used to evaluate the execution state of each candidate item through a preset state evaluation function, wherein, in response to the evaluation value of the state evaluation function exceeding the state threshold, the execution of the thinking step is stopped.

[0011] According to an embodiment of the present invention, identifying the potential spatial relationship features between the source object and the target object in the story features according to each thinking step includes: using a priority search algorithm to identify the potential spatial relationship features between the source object and the target object in the story features according to each thinking step, wherein the priority search algorithm includes a depth-first search algorithm and a breadth-first search algorithm.

[0012] According to an embodiment of the present invention, determining the spatial relationship between a source object and a target object based on latent spatial relationship features includes: extracting a thought chain between the source object and the target object based on the latent spatial relationship features; based on the thought chain, utilizing a spatial relationship mapping model to calculate a position offset of the target object to determine the spatial relationship between the source object and the target object.

[0013] According to an embodiment of the present invention, determining the spatial relationship between the source object and the target object based on the potential spatial relationship features further includes: performing consistency check on the thought chain to ensure the correctness of each thought step.

[0014] Another aspect of the present invention provides a large model target spatial relationship determination device based on thinking tree guidance, the device comprising: an acquisition module, used to acquire the spatial reasoning task to be processed; a parsing module, used to parse out the task description features corresponding to the spatial reasoning task according to the spatial reasoning task, wherein the task description features include story features and question features that characterize the spatial relationship of each object; an execution module, used to input the task description features into a pre-trained large language model, and perform the following operations: split the question features into a thinking tree structure with multiple nodes, wherein each node represents a thinking step; according to each thinking step, identify the potential spatial relationship features between the source object and the target object in the story features, wherein the potential spatial relationship features include all possible paths that can be connected to the target object with the source object as the starting point; determine the spatial relationship between the source object and the target object according to the potential spatial relationship features.

[0015] Another aspect of the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above method.

[0016] Another aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.

[0017] Another aspect of the present invention provides a computer program product, the computer program product comprising computer executable instructions, and the instructions are used to implement the above method when being executed.

[0018] Compared with the prior art, the method and device for determining the spatial relationship of a large model target based on the guidance of a mind tree provided by the present invention have at least the following beneficial effects:

[0019] (1) The method and device for determining the target spatial relationship of a large model based on the guidance of a thinking tree provided by the present invention decomposes complex spatial reasoning problems into thinking steps with intermediate processes by introducing the TOT (Tree-of-Thought) paradigm, and performs systematic search and backtracking through a tree structure. This paradigm enables the large model to perform thinking evaluation more rigorously, thereby enhancing the interpretability and flexibility of the large model.

[0020] (2) The method and device for determining the spatial relationship of a large model target based on the guidance of a thought tree provided by the present invention generates potential execution thoughts through a thought generator, and heuristically evaluates the effect of each state execution through a state evaluator. This combination enables the model to focus on local spatial features and semantic information, improving performance and data efficiency on specific tasks.

[0021] (3) The method and device for determining the target spatial relationship of a large model based on the guidance of a thinking tree provided by the present invention introduces a series of thinking chains through the TOT prompt scheme to connect input and output, and enhances the large model to consider different paths through the guidance of the thinking chain. The application of this prompt scheme enables the model to more effectively identify the spatial connection relationship between the source object and the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0023] Figure 1 A flowchart of a method for determining a large model target spatial relationship based on a mindset tree according to an embodiment of the present invention is schematically shown;

[0024] Figure 2 The structure block diagram of a large model target spatial relationship determination device based on mind tree guidance according to an embodiment of the present invention is schematically shown;

[0025] Figure 3 A structural block diagram of an electronic device suitable for implementing a method for determining a large model target spatial relationship based on a mindset tree guidance according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0026] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0027] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0028] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0029] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0030] In the embodiments of the present invention, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage and other aspects of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0031] Spatial reasoning is an important cognitive ability that involves understanding and inferring information such as the position, direction, and shape of an object in three-dimensional space. This is widely needed in many practical applications, such as robot navigation, assisted driving, and augmented reality. However, spatial reasoning tasks often have some technical bottlenecks, such as: difficulty in modeling complex geometric relationships and physical dynamics, insufficient integration of spatial information and semantic reasoning, and insufficient learning ability for abstract concepts and reasoning rules. In order to solve these problems, large model technology has brought new opportunities to spatial reasoning tasks in recent years.

[0032] Large models can use massive amounts of visual and textual data to learn rich visual-linguistic knowledge representations through self-supervised learning, and can solve many logical reasoning tasks that are critical to human communication. However, complex spatial reasoning tasks usually involve multiple sub-problems, which require layer-by-layer reasoning and comprehensive analysis to arrive at the final answer. As a result, large models still have some bottlenecks when directly processing complex spatial reasoning tasks, including: black-box and weak interpretability problems, as well as limited generalization capabilities and lack of structured reasoning mechanisms, resulting in low data processing efficiency for large models.

[0033] In order to solve the above-mentioned problems, TOT (Tree-of-Thought) technology can be used as an effective solution. TOT can help large models establish a hierarchical and explainable reasoning structure, enhance generalization capabilities and data efficiency to cope with this complexity.

[0034] Specifically, TOT technology can decompose a complex spatial reasoning problem into a tree structure, with each node representing a sub-problem or an intermediate reasoning step. This hierarchical structure has the following advantages, including: clear reasoning process: TOT can show the complete reasoning chain, so that the input, operation and output of each step are clear at a glance, greatly enhancing the interpretability; flexible problem decomposition: the original problem can be divided into smaller and easier to solve sub-problems according to the specific task. This flexibility helps to better capture the characteristics of the task; local feature utilization: TOT allows the model to focus on local spatial features and semantic information, thereby improving performance and data efficiency on specific tasks; knowledge reuse: the reasoning system established by TOT can be reused and transferred between different but related spatial tasks, improving the overall generalization ability. The TOT-guided model can enable large models to achieve significant performance improvements in spatial reasoning tasks, greatly improve the machine's understanding and inference ability of the three-dimensional world, and help develop more advanced, intuitive and user-friendly AI systems, playing an important role in various spatial perception and decision-making scenarios.

[0035] Based on the above concept, the present invention provides a method and device for determining the target spatial relationship of a large model based on the guidance of a thinking tree, which is used to solve the technical problems that the existing large model is not interpretable when processing complex spatial reasoning tasks, and the data processing efficiency is low. The method includes: obtaining the spatial reasoning task to be processed; according to the spatial reasoning task, parsing the task description features corresponding to the spatial reasoning task, wherein the task description features include story features and question features that characterize the spatial relationship of each object; inputting the task description features into a pre-trained large language model, and performing the following operations: splitting the question features into a thinking tree structure with multiple nodes, wherein each node represents a thinking step; according to each thinking step, identifying the potential spatial relationship features between the source object and the target object in the story features, wherein the potential spatial relationship features include all possible paths that can be connected to the target object starting from the source object; according to the potential spatial relationship features, determining the spatial relationship between the source object and the target object.

[0036] Figure 1 A flowchart of a method for determining a large model target spatial relationship based on a mindset tree guidance according to an embodiment of the present invention is schematically shown.

[0037] like Figure 1 As shown, the method for determining the spatial relationship of a large model target based on the guidance of a mind tree in this embodiment may include operations S1 to S5, for example.

[0038] In operation S1, a spatial reasoning task to be processed is obtained.

[0039] In operation S2, task description features corresponding to the spatial reasoning task are parsed according to the spatial reasoning task, wherein the task description features include story features and question features that characterize the spatial relationship between objects.

[0040] Input the task description features into the pre-trained large language model and perform operations S3 to S5:

[0041] In operation S3, the problem feature is split into a thinking tree structure having a plurality of nodes, wherein each node represents a thinking step.

[0042] In operation S4, according to each thinking step, a latent spatial relationship feature between the source object and the target object is identified in the story feature, wherein the latent spatial relationship feature includes all possible paths starting from the source object and being able to connect to the target object.

[0043] In operation S5, the spatial relationship between the source object and the target object is determined according to the latent spatial relationship features.

[0044] In this embodiment, after obtaining the spatial reasoning task to be processed, the task description features corresponding to the spatial reasoning task are first parsed out, and the task description features include story features and problem features that characterize the spatial relationship between objects.

[0045] The task description usually refers to answering the relationship between two query objects (source object and target object) given a sentence that defines the spatial relationship between various objects. For example, given a spatial reasoning task, the task description is as follows:

[0046] story:

[0047] 1. Q is on the right side of O and on the same horizontal plane.

[0048] 2. Q is slightly off to the upper left, and M is slightly off to the lower right.

[0049] 3. X and E are adjacent, X is on top and E is on the bottom.

[0050] 4. O sits to the upper right of E.

[0051] 5. W is to the right and below M.

[0052] Question: What is the relationship between object W and object E?

[0053] By inputting the task description features into a pre-trained large language model, TOT converts the thinking framework of a given problem into a search on a tree, where each node represents a thinking step.

[0054] Execute each thinking step to identify the latent spatial relationship feature between the source object and the target object in the story feature. The latent spatial relationship feature includes all possible paths that can connect to the target object starting from the source object. Finally, determine the spatial relationship between the source object and the target object based on the latent spatial relationship feature.

[0055] Human thinking is represented by a coherent sequential language for solving problems. In other words, this sequential language is the intermediate step in solving problems. The thinking tree model combines the ability of the big model to generate and evaluate thinking with the breadth / depth first search algorithm, guiding the big model to systematically explore forward and trace back its own thinking process, so that the big model can perform more rigorous thinking evaluation.

[0056] The method for determining the target spatial relationship of a large model based on the guidance of a thinking tree provided by the present invention decomposes complex spatial reasoning problems into thinking steps with intermediate processes by introducing the TOT (Tree-of-Thought) paradigm, and performs systematic search and backtracking through a tree structure. This paradigm enables the large model to perform thinking evaluation more rigorously, thereby enhancing the interpretability and flexibility of the large model.

[0057] According to an embodiment of the present invention, operation S4 may include, for example, identifying the potential spatial relationship features between the source object and the target object in the story features according to each thinking step:

[0058] Taking the source object as the root node, execute each thinking step to identify all the nodes that the source object can reach in the next step; when the source object reaches the target sub-node corresponding to the target object after one of the thinking steps is completed, a connection relationship between the root node and the target sub-node is established; based on the connection relationship, the latent spatial relationship features between the source object and the target object are generated.

[0059] According to an embodiment of the present invention, each thinking step has a one-to-one corresponding tree state, which represents the state of the spatial reasoning task under the current thinking step, that is, each node is a tree state. , where x represents the task description feature, z represents the thinking steps, and i represents the number of thinking steps.

[0060] In this embodiment, the specific implementation plan of TOT includes the following four steps: decomposing the given problem features into thinking steps with intermediate processes; generating potential execution thinking for each tree state; heuristically evaluating the effect of the execution of each state; and determining the specific search algorithm to be used.

[0061] TOT designs and disassembles intermediate thinking steps by analyzing the properties of the problem. Generally, a thinking step should be broken down into small enough steps so that the large model can generate more diverse and coherent sample results.

[0062] According to an embodiment of the present invention, the large language model includes: a thought generator , which is used to generate multiple candidates for the next thinking step for each tree state according to the thinking strategy.

[0063] In this embodiment, given a tree state , two thinking strategies are used to generate k candidates for the next thinking step, where the two thinking strategies include: a first thinking strategy and a second thinking strategy.

[0064] The first thinking strategy is configured to generate k candidate items independently and identically distributed by using the thinking chain reasoning method, wherein the generation method of the first thinking strategy is:

[0065]

[0066] Among them, s represents the tree state, represents the independent and identically distributed based on the thought chain, and j represents one of the K candidates.

[0067] The second thinking strategy is configured to generate k candidate items in a "sequential prompt" manner, wherein the generation method of the second thinking strategy is:

[0068]

[0069] in, represents independent and identically distributed based on sequential cues.

[0070] According to an embodiment of the present invention, the large language model also includes: a state evaluator, which is used to evaluate the execution state of each candidate item through a preset state evaluation function, wherein, in response to the evaluation value of the state evaluation function exceeding the state threshold, the execution of the thinking step is stopped.

[0071] In this embodiment, the state evaluation function can be set to , evaluate the status of solving the problem according to the current task and solution. If the evaluation value of the status evaluation function has exceeded the threshold of the problem, the solution can be terminated.

[0072] According to an embodiment of the present invention, operation S4 includes identifying the potential spatial relationship features between the source object and the target object in the story features according to each thinking step:

[0073] By using a priority search algorithm, according to each thinking step, the potential spatial relationship features between the source object and the target object are identified in the story features, wherein the priority search algorithm includes a depth-first search algorithm and a breadth-first search algorithm.

[0074] In this embodiment, the depth-first search (DFS) or breadth-first search (BFS) can be selected according to the decomposed thinking steps to explore the modeled thinking tree steps. During the exploration process, the evaluation value calculated by the state evaluator deviates from the current threshold. , the search algorithm will backtrack to the parent node to continue exploring other steps.

[0075] The method for determining the target spatial relationship of a large model based on the guidance of a thinking tree provided by the present invention generates potential execution thoughts through a thinking generator, and heuristically evaluates the effect of the execution of each state through a state evaluator. This combination enables the model to focus on local spatial features and semantic information, thereby improving performance and data efficiency on specific tasks.

[0076] According to an embodiment of the present invention, operation S5 of determining the spatial relationship between the source object and the target object according to the potential spatial relationship feature includes:

[0077] According to the characteristics of latent spatial relationship, the thought chain between the source object and the target object is extracted; based on the thought chain, the spatial relationship mapping model is used to calculate the position offset of the target object, determine the spatial relationship between the source object and the target object, and perform consistency check on the thought chain to ensure the correctness of each thinking step.

[0078] In this embodiment, the core idea of ​​TOT is to introduce a series of thinking chains. , used to connect the input x and output y, where i represents the i-th step.

[0079] In a reasoning task, x includes the task description (story and question) and a few-shot case hints, and y represents the relationship between the queried target object (from the source object o s To the target object o t ) of the relationship, each chain of thought is to identify the source object o s and target object o t The connection relationship between them.

[0080] The guidance of thought chains can enhance the large model to consider different paths, but it is easy to be distracted in the process of searching for source objects and target objects, that is, the selection of the path sequence from the source object to the target object is prone to errors, so it is crucial to successfully reach the target object from the source object in the correct sequence. If the large model mistakenly tracks an incorrect sequence, it may fall into a dead end that leads to incorrect reasoning conclusions, such as "the story does not provide direct spatial information."

[0081] The algorithm starts by prompting the big model to set an initial tree state s0, using the input x, which includes a story and a question.

[0082] The initial tree state s0 is of the form " , unused R", where R represents all connections between objects in the story, in the form of object1-object2. Next, it continues to iterate from the source object o s To the target object o t Construct a connection relationship, where for the i-th step, the large model needs to consider the tree state s established in this step i-1 , if s i-1 The state does not contain the ability to reach o t , the large model is prompted for each substate s in the current tree state j i-1 Generate k candidate thoughts (where k=2).

[0083] Thought generators mainly work by suggesting relationships that are never used by the big model. Explore and current object i-1 Potential objects to connect to i possibility, and simultaneously perform valid candidate checks to ensure that the proposed valid candidate extensions are non-duplicate ones.

[0084] The state evaluator evaluates each candidate connection using a state evaluation function that is fed back by the prompt, i.e., an evaluation of whether the chain can reach the target object (definite / possible / impossible). If the chain has reached the target, it is "definite". If the unused relationship includes the current object, it is "possible". If the unused relationship does not include the current object, it is "impossible". The algorithm then calls the state evaluation function to calculate the score of the potential connection and determine whether the connection is related to o i Connect and Update Relationship table, in order to connect to the target object as soon as possible t .

[0085] In this embodiment, the priority search algorithm is based on the depth of the candidate tree and the width of the candidate tree, and comprehensively considers the use of the depth-first search DFS model to select each s i The best b-tree is evaluated comprehensively in . The criterion for stopping the search is set when the link reaches the target object. t Status f When , the hint chain constructed by tree chain connection will be used to form the final connection L.

[0086] TOT provides the source object o s To the target object o t Based on the formed construction chain, the spatial relationship between objects is calculated by using spatial relationship mapping, position offset calculation and thinking chain consistency verification, and finally the source object o is formed. s To the target object o t Reasoning about the spatial relationship between .

[0087] In this embodiment, possible relationships in the spatial reasoning task include: overlap, above, below, left, right, upper left, upper right, lower left, and lower right. If the sentence in the story describes clockwise information, 12 represents above, 1 and 2 represent upper right, 3 represents right, 4 and 5 represent lower right, 6 represents below, 7 and 8 represent lower left, 9 represents left, and 10 and 11 represent upper left. If the sentence describes cardinal directions, north represents above, east represents right, south represents below, and west represents left. In all spatial relationships, it is assumed that all agents occupy positions on grid points that are equally spaced in the vertical and horizontal directions, and the agent occupies the nearest grid point that is consistent with the spatial relationship.

[0088] To make this embodiment more detailed, the following will describe in detail the subsequent operations through a specific spatial reasoning task, including starting a connection chain, identifying potential relationships between target objects, and listing all links between objects from a statement.

[0089] story:

[0090] 1. Q is on the right side of O and on the same horizontal plane.

[0091] 2. Q is slightly off to the upper left, and M is slightly off to the lower right.

[0092] 3. X and E are adjacent, X is on top and E is on the bottom.

[0093] 4. O sits to the upper right of E.

[0094] 5. W is to the right and below M.

[0095] Question: What is the relationship between object W and object E?

[0096] Thinking about potential next steps:

[0097] chain: W->, target object: E, unused relationship set: 1.Q->O, 2Q->M, 3.X->E, 4.O->E, 5.W->M.

[0098] TOT thinking chain guides the answer:

[0099] Please list all potential extension chains through the relationships in the unused relationship set, considering the unused relationships directly connected to the last target object.

[0100] Assume that W is at (0,0). The relationship of W to E can be analyzed by following the chain of links and considering step by step the relationships provided in the story, starting with W.

[0101] According to 5, "W is to the right of M and below M." This means that M is above and to the left of W. M = W + offset (upper left) = (0,0) + (-1,1) = (-1,1). Therefore, M is at (-1,1).

[0102] Then we come to M. According to 2, "Q is slightly off-center to the upper left, while M is slightly off-center to the lower right." This means that Q is above and to the left of M. Q = M + offset (upper left) = (-1, 1) + (-1, 1) = (-2, 2). Therefore, Q is at (-2, 2).

[0103] Then we come to Q. According to 1, "Q is to the right of O and on the same horizontal plane." This means that O is to the left of Q. O = Q + offset (left) = (-2,2) + (-1,0) = (-3,2). Therefore, O is at (-3,2).

[0104] Then we come to O. According to 4, "O sits at the upper right position of E." This means that E is at the lower left of O. E = O + offset (lower left) = (-3,2)+(-1,-1)=(-4,1). Therefore, E is at (-4,1).

[0105] Finally arrived at E.

[0106] So, considering W(0,0) and E(-4,1), W is below and to the right of E.

[0107] The thought chain consistency check is used synchronously to trace back and verify that the relative coordinate positions between objects are correct, thus confirming the answer.

[0108] Answer: Bottom right

[0109] The method for determining the target spatial relationship of a large model based on the guidance of a thinking tree provided by the present invention introduces a series of thinking chains for connecting input and output through a TOT prompt scheme, and enhances the large model's consideration of different paths through the guidance of the thinking chain. The application of this prompt scheme enables the model to more effectively identify the spatial connection relationship between the source object and the target object.

[0110] Figure 2 The structural block diagram of the device for determining the spatial relationship of a large model target guided by a mind tree according to an embodiment of the present invention is schematically shown.

[0111] like Figure 2 As shown, the large model target spatial relationship determination device 200 based on the guidance of the mind tree according to the embodiment of the present invention includes: an acquisition module 210, a parsing module 220, and an execution module 230.

[0112] The acquisition module 210 is used to acquire the spatial reasoning task to be processed.

[0113] The parsing module 220 is used to parse out task description features corresponding to the spatial reasoning task according to the spatial reasoning task, wherein the task description features include story features and question features that characterize the spatial relationship between objects.

[0114] The execution module 230 is used to input the task description features into the pre-trained large language model and perform the following operations:

[0115] Split the problem features into a thinking tree structure with multiple nodes, where each node represents a thinking step;

[0116] According to each thinking step, a latent spatial relationship feature between a source object and a target object is identified in the story feature, wherein the latent spatial relationship feature includes all possible paths starting from the source object and being able to connect to the target object;

[0117] According to the latent spatial relationship features, the spatial relationship between the source object and the target object is determined.

[0118] The large model target spatial relationship determination device based on the guidance of the thinking tree provided by the present invention decomposes complex spatial reasoning problems into thinking steps with intermediate processes by introducing the TOT (Tree-of-Thought) paradigm, and performs systematic search and backtracking through a tree structure. This paradigm enables the large model to perform thinking evaluation more rigorously, thereby enhancing the interpretability and flexibility of the large model.

[0119] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware, and firmware, or by a proper combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be executed.

[0120] For example, any multiple of the acquisition module 210, the parsing module 220, and the execution module 230 can be combined in one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present invention, at least one of the acquisition module 210, the parsing module 220, and the execution module 230 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware or in a suitable combination of any of them. Alternatively, at least one of the acquisition module 210 , the parsing module 220 , and the execution module 230 may be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function may be executed.

[0121] It should be noted that the part of the large model target spatial relationship determination device based on the guidance of the thinking tree in the embodiment of the present invention corresponds to the part of the large model target spatial relationship determination method based on the guidance of the thinking tree in the embodiment of the present invention. The description of the large model target spatial relationship determination device based on the guidance of the thinking tree specifically refers to the part of the large model target spatial relationship determination method based on the guidance of the thinking tree, which will not be repeated here.

[0122] Figure 3 A structural block diagram of an electronic device suitable for implementing a method for determining a large model target spatial relationship based on a mindset tree guidance according to an embodiment of the present invention is schematically shown. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0123] like Figure 3 As shown, the electronic device 300 according to an embodiment of the present invention includes a processor 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage part 308 to a random access memory (RAM) 303. The processor 301 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 301 may also include an onboard memory for caching purposes. The processor 301 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0124] In RAM 303, various programs and data required for the operation of electronic device 300 are stored. Processor 301, ROM 302 and RAM 303 are connected to each other through bus 304. Processor 301 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 302 and / or RAM 303. It should be noted that the program can also be stored in one or more memories other than ROM 302 and RAM 303. Processor 301 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0125] According to an embodiment of the present invention, the electronic device 300 may further include an input / output (I / O) interface 305, which is also connected to the bus 304. The electronic device 300 may further include one or more of the following components connected to the input / output (I / O) interface 305: an input portion 306 including a keyboard, a mouse, etc.; an output portion 307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 308 including a hard disk, etc.; and a communication portion 309 including a network interface card such as a LAN card, a modem, etc. The communication portion 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the input / output (I / O) interface 305 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as needed, so that a computer program read therefrom is installed into the storage portion 308 as needed.

[0126] According to an embodiment of the present invention, the method flow according to an embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 309, and / or installed from the removable medium 311. When the computer program is executed by the processor 301, the above-mentioned functions defined in the system of the embodiment of the present invention are executed. According to an embodiment of the present invention, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0127] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0128] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device.

[0129] For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 302 and / or the RAM 303 described above and / or one or more memories other than the ROM 302 and the RAM 303 .

[0130] An embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the method provided by the embodiment of the present invention.

[0131] When the computer program is executed by the processor 301, the above functions defined in the system / device of the embodiment of the present invention are executed. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0132] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 309, and / or installed from the removable medium 311. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0133] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0134] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways, even if such a combination or combination is not explicitly recorded in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways. All these combinations and / or combinations fall within the scope of the present invention.

[0135] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for determining the spatial relationship of a large model target based on a mind tree, characterized in that: The method comprises: Get the spatial reasoning tasks to be processed; According to the spatial reasoning task, a task description feature corresponding to the spatial reasoning task is parsed, wherein the task description feature includes a story feature and a problem feature that characterize the spatial relationship between each object; Input the task description features into the pre-trained large language model and perform the following operations: Splitting the problem feature into a thinking tree structure having multiple nodes, wherein each node represents a thinking step; According to each thinking step, identifying a potential spatial relationship feature between a source object and a target object in the story feature, wherein the potential spatial relationship feature includes all possible paths starting from the source object and being able to connect to the target object; The spatial relationship between the source object and the target object is determined according to the potential spatial relationship feature.

2. The method according to claim 1, characterized in that According to each thinking step, identifying the potential spatial relationship features between the source object and the target object in the story features includes: Taking the source object as the root node, execute each thinking step to identify all nodes that the source object can reach in the next step; In response to the source object reaching the target child node corresponding to the target object after one of the thought steps is executed, a connection relationship between the root node and the target child node is established; According to the connection relationship, a potential spatial relationship feature between the source object and the target object is generated.

3. The method according to claim 1, characterized in that Each thinking step has a one-to-one corresponding tree state, wherein the tree state represents the state of the spatial reasoning task at the current thinking step.

4. The method according to claim 3, characterized in that The large language model includes: A thought generator is used to generate multiple candidates for the next thought step for each tree state according to the thought strategy.

5. The method according to claim 4, characterized in that The thinking strategies include: The first thinking strategy is configured to generate multiple candidate items independently and identically distributed by using the thinking chain reasoning method; The second thinking strategy is configured to generate multiple candidate items in a sequential prompting manner.

6. The method according to claim 4, characterized in that The large language model also includes: A state evaluator is used to evaluate the execution state of each candidate item by using a preset state evaluation function, wherein, in response to an evaluation value of the state evaluation function exceeding a state threshold, the execution of the thinking step is stopped.

7. The method according to claim 1, characterized in that According to each thinking step, identifying the potential spatial relationship features between the source object and the target object in the story features includes: Using a priority search algorithm, according to each thinking step, the potential spatial relationship features between the source object and the target object are identified in the story features, wherein the priority search algorithm includes a depth-first search algorithm and a breadth-first search algorithm.

8. The method according to claim 1, characterized in that The determining the spatial relationship between the source object and the target object according to the potential spatial relationship feature comprises: extracting a thought chain between the source object and the target object according to the latent spatial relationship feature; Based on the thought chain, a spatial relationship mapping model is used to calculate the position offset of the target object to determine the spatial relationship between the source object and the target object.

9. The method according to claim 8, characterized in that The determining the spatial relationship between the source object and the target object according to the potential spatial relationship feature further comprises: The above thought chain is checked for consistency to ensure the correctness of each thought step.

10. A device for determining spatial relationship of large model targets based on mind tree guidance, characterized in that: The device comprises: An acquisition module, used to acquire the spatial reasoning tasks to be processed; A parsing module, configured to parse out task description features corresponding to the spatial reasoning task according to the spatial reasoning task, wherein the task description features include story features and question features that characterize the spatial relationship between objects; An execution module is used to input the task description features into a pre-trained large language model and perform the following operations: Splitting the problem feature into a thinking tree structure having multiple nodes, wherein each node represents a thinking step; According to each thinking step, identifying a potential spatial relationship feature between a source object and a target object in the story feature, wherein the potential spatial relationship feature includes all possible paths starting from the source object and being able to connect to the target object; The spatial relationship between the source object and the target object is determined according to the potential spatial relationship feature.

Images