A large model execution method, system and medium based on embodied retrieval enhancement

Abstract

The present application relates to the field of embodied intelligence technology, and specifically discloses a large model execution method, system and medium based on embodied retrieval enhancement. The execution method mainly includes: obtaining an embodied intelligent body execution example, the execution example is a task instruction I and an action sequence A based on the task instruction I; constructing a retrieval library C with the task instruction I as the key and the action sequence A based on the task instruction I as the value; calculating the relevance score Score; constructing an execution logic set with the task instruction I and the action sequence A according to the relevance score Score, and constructing output content according to the execution logic set. The present application adaptively retrieves relevant execution examples by obtaining execution examples, and then extracts key information from these examples as part of the context, so that the model can have more reliable learning reference content, thereby improving the model's ability to understand semantics and reason about common sense, and ultimately being able to infer more correct execution actions.

Description

Technical Field

[0001] The present application relates to the field of embodied intelligent technology, and in particular to a large model execution method based on embodied retrieval enhancement, a large model execution system based on embodied retrieval enhancement, and a readable storage medium. Background Art

[0002] Embodied execution is one of the core elements of embodied intelligence. It is responsible for translating basic natural language instructions into actual actions that robots can understand and execute. The key lies in how to map the cognition of the virtual world into actual operations and how to deal with uncertainties and changes in the execution process.

[0003] At present, the embodied execution method based on large models mainly stimulates its capabilities based on artificially pre-designed prompt templates and predicts execution actions based on given task instructions.

[0004] However, in the process of implementing relevant technical solutions, it was found that there are at least the following technical problems: when performing tasks, the model needs to be able to generate a specific action sequence that conforms to the task instructions, and plan to ensure that the generated actions can be successfully executed. However, when faced with complex tasks that require semantic understanding or common sense reasoning, due to the lack of semantic understanding and common sense reasoning, large models often produce fantasies, resulting in the generated actions being out of touch with the actual situation, which is not conducive to the execution of embodied intelligent bodies. Summary of the invention

[0005] The present application provides a retrieval-enhanced model execution method, system and medium to solve the problem in the prior art that when facing complex tasks that require semantic understanding or common sense reasoning, large models often generate fantasies due to the lack of semantic understanding ability and common sense reasoning ability, resulting in the generated actions being out of touch with the actual situation. The application provides the model with more reliable learning reference content, thereby improving the model's ability to understand semantics and common sense reasoning ability, and ultimately being able to infer more correct execution actions.

[0006] The present application provides a large model execution method based on embodied retrieval enhancement, which includes the following steps:

[0007] S1. Obtain an embodied agent execution example, where the execution example is a task instruction I and an action sequence A based on the task instruction I;

[0008] S2, constructing a retrieval library C with task instruction I as the key and action sequence A based on task instruction I as the value;

[0009] S3, compare the given task instruction with the task instruction I and action sequence A in the retrieval library C and calculate the relevance score Score; the relevance score Score includes the relevance score ScoreTF-IDf Relevance score Score S-BERT ;

[0010] Calculate the relevance score TF-IDf The methods include:

[0011] S311, obtain the given task instruction I t The keyword frequency TF and inverse document frequency IDF, TF is: The IDF is: Among them, w i Represents the keyword, n(w i ,I t ) represents the keyword w i Appears in Mission Directive I t The frequency in, L represents the task instruction I t The number of keywords in |C| represents the number of candidate tasks in the retrieval library C;

[0012] S312. Calculate the correlation score of keyword frequency TF and inverse document frequency IDF TF-IDf , Score TF-IDf Score TF-IDf (C j ,I t )=∑TF(w i ,I t )×IDF(C j ,w i );

[0013] Calculate the relevance score S-BERT The methods include:

[0014] S321. Obtaining task instruction I according to SentenceBERT model t The characterization vector h p and h q ,h p For: h p =Encoder(I t ), h q For: h q =Encoder(C q );

[0015] S322, calculate the representation vector h p and h q Relevance score Score S-BERT , Score S-BERT for: Among them, Encoder() represents the SentenceBERT encoder, C qRepresents the qth candidate task and its action instructions;

[0016] S4, constructing an execution logic set with the task instruction I and the action sequence A according to the relevance score Score, wherein represents the execution logic, the execution logic includes the instruction I and the action sequence A based on the task instruction I, and K represents the number of execution logics;

[0017] S5. Build output content according to the execution logic set.

[0018] By obtaining execution examples, we adaptively retrieve related execution examples, and then use these examples as part of the context to extract key information, so that the model can have more reliable learning reference content, thereby improving the model's ability to understand semantics and reason about common sense, and ultimately be able to infer more correct execution actions.

[0019] Furthermore, in S1, obtaining the embodied intelligent body execution examples specifically includes: using a crawler program to collect embodied intelligent execution examples on the Internet.

[0020] Furthermore, in constructing a retrieval library C with task instruction I as a key and action sequence A based on task instruction I as a value, task instruction I and action sequence A based on task instruction I are filtered execution examples.

[0021] Furthermore, the screening method is: calculating the similarity S of each pair of execution examples after data preprocessing, deleting any execution example in each pair of execution examples whose similarity S is greater than a preset threshold; running the execution examples and deleting the execution examples that failed to run.

[0022] Furthermore, the similarity S is: Among them, X and Y represent the vector representation of the two execution examples respectively, and ‖X‖ and ‖Y‖ represent the modulus lengths of the two execution example vectors respectively.

[0023] Furthermore, according to the relevance score Score, the execution logic set is constructed with the task instruction I and the action sequence A Specifically, the execution logic set is constructed by using the specified number of task instructions I with the highest relevance score and the action sequence A based on the task instruction I.

[0024] In a second aspect, the present application provides a large model execution system based on embodied retrieval enhancement, which uses the large model execution method based on embodied retrieval enhancement of the first aspect, and includes:

[0025] A collection module, which is used to obtain an embodied agent execution example, wherein the execution example is a task instruction I and an action sequence A based on the task instruction I;

[0026] A retrieval library construction module, which is used to construct a retrieval library C with the task instruction I as a key and the action sequence A based on the task instruction I as a value;

[0027] The relevance score calculation module is used to compare the given task instruction with the task instruction I and action sequence A in the retrieval library C and calculate the relevance score Score; the relevance score Score includes the relevance score Score TF-IDf Relevance score Score S-BERT ;

[0028] Calculate the relevance score TF-IDf The methods include:

[0029] S311, obtain the given task instruction I t The keyword frequency TF and inverse document frequency IDF, TF is: The IDF is: Among them, w i Represents the keyword, n(w i ,I t ) represents the keyword w i Appears in Mission Directive I t The frequency in, L represents the task instruction I t The number of keywords in |C| represents the number of candidate tasks in the retrieval library C;

[0030] S312. Calculate the correlation score of keyword frequency TF and inverse document frequency IDF TF-IDf , Score TF-IDf Score TF-IDf (C j ,I t )=∑TF(w i ,I t )×IDF(C j ,w i );

[0031] Calculate the relevance score S-BERT The methods include:

[0032] S321. Obtaining task instruction I according to SentenceBERT model t The characterization vector h p and h q ,h p For: h p =Encoder(I t ), h q For: h q =Encoder(C q );

[0033] S322, calculate the representation vector h p and h q Relevance score Score S-BERT , Score S-BERT for: Among them, Encoder() represents the SentenceBERT encoder, C q Represents the qth candidate task and its action instructions;

[0034] A correlation task construction module is used to construct an execution logic set with task instruction I and action sequence A according to the correlation score Score, wherein represents the execution logic, and the execution logic includes instruction I and action sequence A based on task instruction I, and K represents the number of execution logics;

[0035] The output module is used to construct output content according to the execution logic set.

[0036] The technical solution provided by this application has at least the following technical effects or advantages:

[0037] 1. The present invention obtains execution examples on the Internet and builds a related database. The model can adaptively retrieve relevant execution examples according to task instructions, and then use these examples as part of the context to extract key information. This effectively solves the problem that when faced with complex tasks that require semantic understanding or common sense reasoning, large models often generate fantasies, resulting in generated actions that are out of touch with the actual situation due to the lack of semantic understanding and common sense reasoning. This allows the model to have more reliable learning reference content, thereby improving the model's ability to understand semantics and reason about common sense, and ultimately being able to infer more correct execution actions.

[0038] 2. The present invention adopts a two-way recall method. Through surface semantic matching and deep logic matching execution examples, it can recall several tasks with the highest relevance to the task to be executed and their corresponding action execution sequences in the retrieval library. These matching results are organized into defined contextual learning task templates to provide the model with richer external knowledge to guide learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flowchart of performing example collection and organization in Embodiment 1 of the present application;

[0040] Figure 2 This is a flow chart of a large model execution method based on embodied retrieval enhancement in Embodiment 1 of the present application;

[0041] Figure 3 This is a module diagram of the large model execution system based on embodied retrieval enhancement in Example 2 of the present application. DETAILED DESCRIPTION

[0042] This application obtains human knowledge priors as context examples through multimodal retrieval, and uses these examples as additional prompts during large model training and reasoning, so that it can have more reliable learning reference content, thereby improving the model's ability to understand semantics and reason about common sense, enabling the model to infer more correct execution actions.

[0043] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0044] like Figure 1-Figure 2 As shown, this embodiment provides a large model execution method based on embodied retrieval enhancement, which includes the following steps:

[0045] S1. Obtain an embodied intelligent agent execution example, wherein the execution example is a task instruction I and an action sequence A based on the task instruction I.

[0046] The specific steps of obtaining the embodied intelligent agent execution examples are as follows: a crawler program is used to collect embodied intelligent execution examples on the Internet, each example includes a task instruction I and an action sequence A, where A is: A = {a 1 ,a 2 ,...,a i}, a i Represents the i-th execution action a. The crawler program can use crawler frameworks such as Scrapy, Beautiful Soup, Selenium, etc. The obtained examples can be organized as Here, M represents the number of instances.

[0047] S2, constructing a retrieval library C with task instruction I as the key and action sequence A based on task instruction I as the value;

[0048] In constructing a retrieval library C with task instruction I as the key and action sequence A based on task instruction I as the value, task instruction I and action sequence A based on task instruction I are the filtered execution examples. When filtering, the acquired execution examples are firstly preprocessed, including removing examples with incorrect formats, unifying the formats, removing special symbols, etc., and then identifying repeated and redundant execution examples.

[0049] The screening method is: calculating the similarity S of each pair of execution examples after data preprocessing, deleting any execution example in each pair of execution examples whose similarity S is greater than a preset threshold; running the execution examples and deleting the execution examples that failed to run.

[0050] In this embodiment, cosine similarity is used to identify repeated and redundant execution examples, and each execution example is represented as a vector, where each dimension of the vector represents a feature or term. The cosine similarity S between each pair of execution examples is calculated. According to the set threshold, it is determined which execution examples have similarities exceeding the threshold, so as to identify repeated and redundant examples. The similarity S is: Among them, X and Y represent the vector representation of the two execution examples respectively, and ‖X‖ and ‖Y‖ represent the modulus lengths of the two execution example vectors respectively.

[0051] S3, the given task instruction I t Compare with the task instruction I and action sequence A in the retrieval library C and calculate the relevance score Score.

[0052] When a task instruction is given, the present invention adopts a dual-path recall retrieval method to compare all tasks I and execution action sequences A in the retrieval library C, fully considering the surface semantic correlation and deep execution logic correlation between task I and action A.

[0053] The relevance score Score includes the relevance score Score TF-IDf , calculate the relevance score Score TF-IDf The methods include:

[0054] S311, obtain the given task instruction I t The keyword frequency TF and inverse document frequency IDF, TF is: The IDF is: Among them, w i Represents the keyword, n(w i ,I t ) represents the keyword w i Appears in Mission Directive I t The frequency in, L represents the task instruction I t The number of keywords in |C| represents the number of candidate tasks in the retrieval library C;

[0055] S312. Calculate the correlation score of keyword frequency TF and inverse document frequency IDF TF-IDf , Score TF-IDf For: Score TF-IDf (C j ,I t )=∑TF(w i ,I t )×IDF(C j ,w i ).

[0056] The surface semantic relevance mainly considers the matching degree of keywords. The TF-IDF method is used to obtain the keyword frequency TF and inverse document frequency IDF of the task instructions, and then multiply them to calculate the relevance score.

[0057] The relevance score Score includes the relevance score Score S-BERT , calculate the relevance score Score S-BERT The methods include:

[0058] S321. Obtaining task instruction I according to SentenceBERT model t The characterization vector h p and h q ,h p For: h p =Encoder(I t ), h q For: h q =Encoder(C q );

[0059] S322, calculate the representation vector h p and h q Score of relevance S-BERT , Score S-BERT for: Among them, Encoder() represents the SentenceBERT encoder, C q Represents the qth candidate task and its action instructions.

[0060] In addition, the deep execution logic relevance mainly considers the overall matching degree of the sentence. The present invention uses a deep neural network to represent the task sentence, encodes it to obtain a hidden vector, and then recalls relevant examples based on the matching degree of the sentence vector. Specifically, the SentenceBERT model is used to obtain the representation vector of the sentence, and the dot product is performed to calculate the relevance score.

[0061] S4. Construct an execution logic set based on the relevance score Score with the task instruction I and the action sequence A Among them, C j represents a task, which includes instruction I and an action sequence A based on task instruction I, and K represents the number of execution logics.

[0062] The relevance score Score can be obtained by the relevance score Score TF-IDf and relevance score Score S-BERT The comprehensive result first needs to be the relevance score Score TF-IDf and relevance score Score S-BERTNormalize them so that their values ​​are between 0 and 1. TF-IDf and relevance score Score S-BERT After normalization, they are expressed as: Score TF-IDf ′ and Score S-BERT ′, then Score TF-IDf ′ and Score S-BERT ′ are respectively:

[0063] Among them, Score TF-IDfmax Represents the relevance score Score TF-IDf The maximum value, Score TF-IDfmin Represents the relevance score Score TF-IDf The minimum value of

[0064] Among them, Score S-BERTmax Represents the relevance score Score S-BERT The maximum value, Score S-BERTmin Represents the relevance score Score S-BERT The minimum value of

[0065] Finally, calculate the correlation score Score, which can be: Score = k × Score TF-IDf ′+(1-k)×Score S-BERT ', where k is a weighting factor used to balance the contribution of keyword matching and sentence overall matching to similarity. In this embodiment, k can be 0.5.

[0066] According to the relevance score Score, the execution logic set is constructed with the task instruction I and the action sequence A.

[0067] Specifically, the execution logic set is constructed by using the specified number of task instructions I with the highest relevance score and the action sequence A based on the task instruction I. K represents the number of execution logics.

[0068] Through the above method, the C in C in the search library can be q Sort the relevance and recall several tasks to be performed I t The most relevant task I and its corresponding action execution sequence A. These matching results are organized into defined context learning task templates to provide the model with richer external knowledge and guide learning.

[0069] S5. Based on the execution logic set Build output.

[0070] The present invention adopts the paradigm of efficient fine-tuning of instructions to train the embodied execution large model. Specifically, through the collection and organization of S1-S4 execution actions a, the matching and retrieval of task instructions I, a set of tasks that match the given task instructions I is successfully extracted. t Highly related task instructions I and their corresponding execution logic These elements are integrated into a framework and work in parallel with the task description, scene description and predefined prompt templates to form the input content. Specifically, the loss function of the large model is: LOSS is: Among them, s i+1 Represents the i+1th character, s <i represents a word less than i, and θ represents the parameters of the model.

[0071] Through retrieval-enhanced contextual example fine-tuning learning, the embodied execution large model can reduce hallucinations and generate a series of precise execution actions, making the embodied execution large model widely applicable to new scenarios and new tasks.

[0072] [Table 1]

[0073]

[0074]

[0075] As shown in Table 1, Table 1 shows the performance comparison between the algorithm of the present invention and the commonly used algorithms in the world under known scenarios.

[0076] [Table 2]

[0077]

[0078] As shown in Table 2, Table 2 shows the performance comparison between the algorithm of the present invention and the commonly used algorithms in the world in new scenarios and new tasks.

[0079] Embodiment 2

[0080] like Figure 3 As shown, the present application provides a large model execution system based on embodied retrieval enhancement, which uses the large model execution method based on embodied retrieval enhancement in Example 1, which includes: a collection module, a retrieval library construction module, a relevance score calculation module, a relevance task construction module and an output module.

[0081] The acquisition module is used to obtain the embodied agent execution example, wherein the execution example is a task instruction I and an action sequence A based on the task instruction I; the retrieval library construction module is used to construct a retrieval library C with the task instruction I as the key and the action sequence A based on the task instruction I as the value; the relevance score calculation module is used to convert the given task instruction I into a retrieval library C.t Compare with the task instruction I and action sequence A in the retrieval library C and calculate the relevance score Score; the relevance score Score includes the relevance score Score TF-IDf Relevance score Score S-BERT ;

[0082] Calculate the relevance score TF-IDf The methods include:

[0083] S311, obtain the given task instruction I t The keyword frequency TF and inverse document frequency IDF, TF is: The IDF is: Among them, w i Represents the keyword, n(w i ,I t ) represents the keyword w i Appears in Mission Directive I t The frequency in, L represents the task instruction I t The number of keywords in |C| represents the number of candidate tasks in the retrieval library C;

[0084] S312. Calculate the correlation score of keyword frequency TF and inverse document frequency IDF TF-IDf , Score TF-IDf For: Score TF-IDf (C j ,I t )=∑TF(w i ,I t )×IDF(C j ,w i );

[0085] Calculate the relevance score S-BERT The methods include:

[0086] S321. Obtaining task instruction I according to SentenceBERT model t The characterization vector h p and h q ,h p For: h p =Encoder(I t ), h q For: h q =Encoder(C q );

[0087] S322, calculate the representation vector h p and h q Relevance score Score S-BERT , ScoreS-BERT for: Among them, Encoder() represents the SentenceBERT encoder, C q Represents the qth candidate task and its action instructions;

[0088] The correlation task construction module is used to construct an execution logic set based on the correlation score Score with the task instruction I and the action sequence A. Among them, C j represents the execution logic in the keyword search library C, K represents the number of execution logics; the output module is used to Build output.

[0089] The retrieval enhancement system of the embodied execution big model based on context learning in this embodiment has the advantages of the retrieval enhancement method of the embodied execution big model based on context learning in Example 1, and the system in this embodiment can automate the big model execution method.

[0090] Embodiment 3

[0091] This embodiment provides a readable storage medium, in which computer program instructions are stored. When the computer program instructions are read and executed by a processor, the steps of the large model execution method based on embodied retrieval enhancement of the first embodiment are executed.

[0092] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0093] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps of the functions specified in a box or multiple boxes. Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they know the basic creative concept. Therefore, the attached claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0094] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0095] What has been described above are only preferred specific implementation methods of the embodiments of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can make equivalent substitutions or changes based on the technical solution and concept of the present application within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application.

Claims

1. A large model execution method based on embodied retrieval enhancement, characterized in that: It includes the following steps: S1. Obtain an embodied agent execution example, where the execution example is a task instruction I and an action sequence A based on the task instruction I; S2, constructing a retrieval library C with task instruction I as the key and action sequence A based on task instruction I as the value; S3, compare the given task instruction with the task instruction I and action sequence A in the retrieval library C and calculate the relevance score Score; The relevance score Score includes the relevance score Score TF-IDf Relevance score Score S-BERT ; Calculate the relevance score TF-IDf The methods include: S311, obtain the given task instruction I t The keyword frequency TF and inverse document frequency IDF, TF is: The IDF is: Among them, w i Represents the keyword, n(w i ,I t ) represents the keyword w i Appears in Mission Directive I t The frequency in, L represents the task instruction I t The number of keywords in |C| represents the number of candidate tasks in the retrieval library C; S312. Calculate the correlation score of keyword frequency TF and inverse document frequency IDF TF-IDf , Score TF-IDf Score TF-IDf (C j ,I t )=∑TF(w i ,I t )×IDF(C j ,w i ); Calculate the relevance score S-BERT The methods include: S321. Obtaining task instruction I according to SentenceBERT model t The characterization vector h p and h q ,h p For: h p =Encoder(I t ), h q For: h q =Encoder(C q ); S322, calculate the representation vector h p and h q Relevance score Score S-BERT , Score S-BERT for: Among them, Encoder() represents the SentenceBERT encoder, C q Represents the qth candidate task and its action instructions; S4, constructing an execution logic set with the task instruction I and the action sequence A according to the relevance score Score, wherein represents the execution logic, the execution logic includes the instruction I and the action sequence A based on the task instruction I, and K represents the number of execution logics; S5. Build output content according to the execution logic set.

2. The large model execution method based on embodied retrieval enhancement as claimed in claim 1, characterized in that: In S1, obtaining the embodied intelligent body execution examples specifically includes: using a crawler program to collect embodied intelligent execution examples on the Internet.

3. The large model execution method based on embodied retrieval enhancement as claimed in claim 1, characterized in that: In constructing a retrieval library C with task instruction I as a key and action sequence A based on task instruction I as a value, task instruction I and action sequence A based on task instruction I are filtered execution examples.

4. The large model execution method based on embodied retrieval enhancement as claimed in claim 3, characterized in that: The screening method is: calculating the similarity S of each pair of execution examples after data preprocessing, and deleting any one execution example in each pair of execution examples whose similarity S is greater than a preset threshold; Run the execution examples and delete the execution examples that failed to run.

5. The large model execution method based on embodied retrieval enhancement as claimed in claim 4, characterized in that: The similarity S is: Among them, X and Y represent the vector representation of the two execution examples respectively, and ‖X‖ and ‖Y‖ represent the modulus lengths of the two execution example vectors respectively.

6. The large model execution method based on embodied retrieval enhancement as claimed in claim 1, characterized in that: According to the relevance score Score, the execution logic set is constructed with the task instruction I and the action sequence A. Specifically, the execution logic set is constructed by using the specified number of task instructions I with the highest relevance score and the action sequence A based on the task instruction I.

7. A large model execution system based on embodied retrieval enhancement, which uses the large model execution method based on embodied retrieval enhancement as described in claims 1-6, characterized in that: It includes: A collection module, which is used to obtain an embodied agent execution example, wherein the execution example is a task instruction I and an action sequence A based on the task instruction I; A retrieval library construction module, which is used to construct a retrieval library C with the task instruction I as a key and the action sequence A based on the task instruction I as a value; The relevance score calculation module is used to compare the given task instruction with the task instruction I and action sequence A in the retrieval library C and calculate the relevance score Score; the relevance score Score includes the relevance score Score TF-IDf Relevance score Score S-BERT ; Calculate the relevance score TF-IDf The methods include: S311, obtain the given task instruction I t The keyword frequency TF and inverse document frequency IDF, TF is: The IDF is: Among them, w i Represents the keyword, n(w i ,I t ) represents the keyword w i Appears in Mission Directive I t The frequency in, L represents the task instruction I t The number of keywords in |C| represents the number of candidate tasks in the retrieval library C; S312. Calculate the correlation score of keyword frequency TF and inverse document frequency IDF TF-IDf , Score TF-IDf Score TF-IDf (C j ,I t )=∑TF(w i ,I t )×IDF(C j ,w i ); Calculate the relevance score S-BERT The methods include: S321. Obtaining task instruction I according to SentenceBERT model t The characterization vector h p and h q ,h p For: h p =Encoder(I t ), h q For: h q =Encoder(C q ); S322, calculate the representation vector h p and h q Relevance score Score S-BERT , Score S-BERT for: Among them, Encoder() represents the SentenceBERT encoder, C q Represents the qth candidate task and its action instructions; A correlation task construction module is used to construct an execution logic set with task instruction I and action sequence A according to the correlation score Score, wherein represents the execution logic, and the execution logic includes instruction I and action sequence A based on task instruction I, and K represents the number of execution logics; The output module is used to construct output content according to the execution logic set.

8. A readable storage medium, characterized in that: The readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the steps of the large model execution method based on embodied retrieval enhancement are executed as described in any one of claims 1-6.

Images