Context-aware hierarchical instruction learning method and device for large language model

Abstract

The application discloses a context-aware hierarchical instruction learning method and device for a large language model, and relates to the technical field of artificial intelligence. The method comprises the following steps: distinguishing the instruction roles of an initial training data for a large language model, and constructing training data according to the instruction roles; constructing instruction role labels for the instruction roles in the training data, determining instruction segment segmentation features and instruction segment query features according to the instruction role labels, instruction segment segmentation embeddings and instruction segment query embeddings; inputting the instruction segment segmentation features, the instruction segment query features and Token embedding features into a context-aware module to obtain sample input data; inputting the sample input data into the large language model to perform supervised fine-tuning training on a reply model, and obtaining a trained reply model; and inputting a text to be replied into the trained reply model to obtain dialogue reply data. By using the application, the execution priority of a key role instruction of a large language model can be improved, and the safety and stability of a dialogue application can be ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a context-aware hierarchical instruction learning method and device for large language models. BACKGROUND

[0002] Large language models (LLM) have shown great potential in many fields in recent years, especially in intelligent agents, automatic decision-making, educational tools, medical assistance, etc. LLM can handle complex language tasks, generate natural language text, answer questions, and execute instructions. However, existing large language model architectures process all tokens uniformly when processing input, lacking a mechanism to distinguish and prioritize instructions based on their roles or importance. This deficiency makes the model vulnerable to malicious operations, as attackers can inject malicious instructions to override normal instructions, leading to security risks and resulting in irresponsible or dangerous responses from the model.

[0003] To address this issue, a method can be used to explicitly distinguish between system instructions, user prompts, and data inputs, and assign a following order according to priority, with system instructions having higher priority than other instructions, to ensure that the model can correctly and safely perform functions according to the designer's intentions and provide reliable assistance to users.

[0004] To achieve the above functions and enhance the security of large language models, many studies have proposed different solutions. Existing technologies such as OpenAI's data-driven model fine-tuning method use hierarchical instruction datasets to train models, ensuring that instructions between different roles follow priority. However, this method fails to explicitly distinguish and prioritize Token-level semantic information for all tokens, resulting in the model being unable to effectively respond to malicious instructions. The instruction segmentation embedding (ISE) technology proposes an instruction segment embedding method that sums the features of the instruction segment and Token embedding to uniformly perform semantic shift on each role segment, but this method ignores the fine-grained semantic information at the Token level, limiting the model's accurate understanding of hierarchical instructions. SUMMARY

[0005] To address the technical problem of existing technologies lacking Token-level semantic information when explicitly modeling instruction role features, resulting in the model's inaccurate understanding of hierarchical instructions, the present application provides a context-aware hierarchical instruction learning method and device for large language models. The technical solution is as follows:

[0006] On the one hand, a context-aware hierarchical instruction learning method for large language models is provided. This method is implemented by a context-aware hierarchical instruction learning device for large language models, and includes:

[0007] S1. Obtain initial training data, distinguish the instruction roles of the large language model in the initial training data, construct training data according to the instruction roles, and determine the instruction segment segmentation embedding, instruction segment query embedding, and token embedding of the input sequence in the training data.

[0008] S2. Construct instruction role labels for instruction roles in the training data. Determine instruction segment segmentation features based on instruction role labels and instruction segment segmentation embeddings. Determine instruction segment query features based on instruction role labels and instruction segment query embeddings. Determine token embedding features based on token embeddings.

[0009] S3. Input the instruction segmentation features, instruction segment query features, and token embedding features into the context awareness module to obtain sample input data;

[0010] S4. Input the sample input data into the large language model, and perform supervised fine-tuning training on the response model composed of the context-aware module and the large language model to obtain the trained response model.

[0011] S5. Obtain the text to be replied to, input the text to be replied to into the trained response model, and obtain the dialogue response data of the text to be replied to.

[0012] On the other hand, a context-aware hierarchical instruction learning device for large language models is provided. This device is applied to a context-aware hierarchical instruction learning method for large language models. The device includes:

[0013] The construction unit is used to acquire initial training data, distinguish the instruction roles of the large language model in the initial training data, construct training data based on the instruction roles, and determine the instruction segment segmentation embedding, instruction segment query embedding, and token embedding features of the input sequence in the training data.

[0014] The determination unit is used to construct instruction role labels for instruction roles in the training data, determine instruction segment segmentation features based on instruction role labels and instruction segment segmentation embeddings, and determine instruction segment query features based on instruction role labels and instruction segment query embeddings.

[0015] The context-aware unit is used to input instruction segmentation features, instruction segment query features, and token embedding features into the context-aware module to obtain sample input data.

[0016] The fine-tuning training unit is used to input sample input data into the large language model and perform supervised fine-tuning training on the response model composed of the context-aware module and the large language model to obtain a trained response model.

[0017] The response unit is used to acquire the text to be responded to, input the text to be responded to into the trained response model, and obtain the dialogue response data of the text to be responded to.

[0018] On the other hand, a context-aware hierarchical instruction learning device for large language models is provided. The context-aware hierarchical instruction learning device for large language models includes: a processor; a memory, wherein computer-readable instructions are stored in the memory, and when the computer-readable instructions are executed by the processor, any one of the methods described above for context-aware hierarchical instruction learning methods for large language models is implemented.

[0019] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the above-described methods of context-aware hierarchical instruction learning for large language models.

[0020] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0021] This invention implements a context-aware instruction hierarchy priority. By interacting with information between instruction segments of different roles and throughout the entire context, it effectively distinguishes different instruction roles and trains the model to correctly learn the instruction hierarchy based on dialogue data. This improves the execution priority of the large language model for instructions of key roles, prevents the model from generating insecure responses when vulnerable low-priority instructions are attacked, enhances model security, and maintains the instruction compliance capability of the large language model, i.e., maintains model effectiveness. At the same time, distinguishing different instruction roles also provides flexible extensions for the large language model as the foundation for dialogue systems and intelligent agent applications, ensuring the security and stability of such dialogue applications. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1-1 This is a flowchart of a context-aware hierarchical instruction learning method for large language models provided by an embodiment of the present invention;

[0024] Figure 1-2This is a computational schematic diagram of a context-aware hierarchical instruction learning method for large language models provided in an embodiment of the present invention;

[0025] Figure 2 This is a block diagram of a context-aware hierarchical instruction learning device for large language models provided in an embodiment of the present invention;

[0026] Figure 3 This is a schematic diagram of the structure of a context-aware hierarchical instruction learning device for large language models provided in an embodiment of the present invention. Detailed Implementation

[0027] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0028] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0029] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0030] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0031] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0032] This invention provides a context-aware hierarchical instruction learning method for large language models. This method can be implemented by a context-aware hierarchical instruction learning device for large language models, which can be a terminal or a server. Figure 1-1 The flowchart shown is for a context-aware hierarchical instruction learning method for large language models. Figure 1-2 The diagram shown illustrates the computation of a context-aware hierarchical instruction learning method for large language models. The processing flow of this method may include the following steps:

[0033] S1. Obtain initial training data, distinguish the instruction roles of the large language model in the initial training data, construct training data according to the instruction roles, and determine the instruction segment segment embedding, instruction segment query embedding, and token embedding of the input sequence in the training data.

[0034] In this context, based on existing large language model methods, determining token embeddings (i.e., lexical embeddings) refers to constructing a token embedding matrix. ,in For word list length, For model vector dimensions. Determining the instruction segmentation embedding and instruction segment query embedding of the input sequence in the training data refers to constructing two additional matrices according to the token embedding matrix construction method: the instruction segmentation embedding matrix and the instruction segmentation embedding matrix. Instruction segment query embedding matrix , This represents the number of character categories.

[0035] In the large language model, the instruction roles are pre-defined with different priorities, and different priorities indicate different instruction execution orders.

[0036] In one feasible implementation, the roles of large language model instructions are distinguished. This means that during the data cleaning and annotation stages, different roles are manually classified when the large language model interacts with the user, such as system, user, tool, and model response, and their functions in the dialogue are determined according to their roles.

[0037] Specifically, typical dialogue roles include: System, responsible for guiding the dialogue and providing basic functions, usually with the highest instruction priority, such as system instructions or constraints defining model behavior; User, interacting by inputting questions or requests, whose instructions usually follow the rules set by the system and, in most scenarios, have lower priority than system instructions; Tool, referring to auxiliary tools used in the dialogue, such as external knowledge bases, databases, computing modules, etc. Tool instructions are usually used to assist the model in performing tasks, but have lower priority than system and user instructions; Model / Agent / Assistant: as the entity that executes instructions, generating responses or performing specific tasks based on the received instructions. For non-inference models, this is the response content; for inference models, it includes both inference content and response content. These roles typically handle the interaction between the system and the user, and the response results participate in subsequent dialogues as dialogue history, continuing to perform tasks based on the model's own preset rules and algorithms.

[0038] In a large language model, command roles are pre-defined with different priorities. This means that command roles are categorized into hierarchical levels for command compliance. Specifically, a priority hierarchy is established based on different command roles, with higher-priority roles executing commands first (e.g., system commands), and lower-priority roles executing commands according to the constraints of higher-priority roles (e.g., tool commands). In this structure: system commands have the highest priority and must always be executed first to ensure the model follows core rules when performing tasks; user commands typically have lower priority than system commands but still need to be followed; tool commands are executed based on actual task requirements and are usually constrained by both system and user commands; model commands represent the model's response history and serve as a reference for subsequent dialogues. During implementation, this hierarchical division of command roles ensures that various commands are processed according to their priorities, preventing interference from malicious or inappropriate commands.

[0039] Optionally, the constructed training data includes aligned dialogue data and unaligned dialogue data, wherein:

[0040] Aligned dialogue data represents dialogue between instruction roles that follow preset priority rules; unaligned dialogue data represents dialogue between instruction roles that do not follow priority rules or have priority conflicts.

[0041] In one feasible implementation, aligned and unaligned dialogue data based on instruction roles are represented as follows: Dialogue data between different roles is constructed using role information in the training dataset. Aligned dialogue data refers to dialogues between roles that follow preset rules and priorities. For example, the system provides task instructions to the model, the user asks questions according to the framework provided by the system, tools provide auxiliary information, and the execution of all role instructions follows a priority hierarchy. Unaligned dialogue data refers to dialogues between roles that do not strictly follow the rules or contain instruction conflicts. For example, instruction injection attacks or jailbreaking attacks can allow low-priority instructions to bypass system instructions, thus affecting the normal execution of the model. By constructing unaligned data, the model can learn how to handle instruction priority conflicts, ensuring the correct execution of various instructions in a multi-role dialogue environment. During subsequent fine-tuning training, responses from aligned dialogue data should correctly follow instruction priorities and execute all tasks given by all roles. Responses from unaligned dialogue data should identify conflicts between instructions and execute only the task given by the highest-priority role. The model is then trained using these two types of dialogue data to correctly handle instruction priorities.

[0042] S2. Construct instruction role labels for instruction roles in the training data. Determine instruction segment segmentation features based on instruction role labels and instruction segment segmentation embeddings. Determine instruction segment query features based on instruction role labels and instruction segment query embeddings.

[0043] Optionally, command role labels are constructed for command roles in the training data, including:

[0044] S21. Mark the role information corresponding to each Token in the input sequence of the training data to form an instruction role tag sequence.

[0045] In one feasible implementation, a different instruction token is assigned to each instruction role (such as system, user, data, output, etc.), and the role information corresponding to each token is marked in the input sequence. That is, for the input token sequence... The command role marker sequence is formed as follows (1):

[0046] (1)

[0047] in , This represents the number of character categories.

[0048] Optionally, instruction segmentation features are determined based on instruction role markers and instruction segment segmentation embeddings, including:

[0049] S22. Embedding based on instruction segment segmentation , the character command sequence By using a table lookup operation, Mapped to The i-th dimension vector is used to map the instruction role label to a feature vector, thus obtaining the instruction segment segmentation feature.

[0050] Based on the instruction role marker and instruction segment query embedding, the characteristics of the instruction segment query are determined, including:

[0051] S23. Query embedded information based on instruction segment , the character command sequence By using a table lookup operation, Mapped to The i-th dimension vector is used to map the instruction role marker to a feature vector, thus obtaining the instruction segment query feature.

[0052] In one feasible implementation, learnable instruction segment segments can be embedded using... This indicates that the instruction segment query will be embedded using... This indicates that the command role marker sequence will be used. Mapped to feature vectors, the instruction segmentation features of the input sequence are used... This indicates that the query feature of the instruction segment is used. This represents the sequence length, where N is the model vector dimension, and d is used for token embedding. It means that among them Given the vocabulary length, and based on existing large language model token processing techniques, the token sequence x is mapped to token embedding features. .

[0053] S3. Input the instruction segmentation features, instruction segment query features, and token embedding features into the context awareness module to obtain sample input data.

[0054] Optionally, the context-aware module includes an intra-segment summary module and a context aggregation module.

[0055] S3 inputs instruction segmentation features, instruction segment query features, and token embedding features into the context-aware module to obtain sample input data, including:

[0056] S31. Input the instruction segment query features and token embedding features into the segment summary module, and fuse the instruction segment query features and token embedding features through the cross-attention mechanism of the segment summary module to obtain the fused features.

[0057] In one feasible implementation, the two features, instruction segment query features and token embedding features, are first projected into the attention space, as shown in equation (2):

[0058] (2)

[0059] in These are different, learnable weight parameters. Indicates the token embedding feature; Indicates query embedding. Indicates key, Represents the value;

[0060] Then, the two features are fused through cross-attention, as shown in equation (3):

[0061] (3)

[0062] Where d represents the dimension of the query embedding, and M is the intra-segment attention mask matrix. The intra-segment attention mask matrix M is represented as follows: the corresponding position in the mask matrix for tokens within the same segment is 0, and the corresponding position in the mask matrix for tokens from different segments is... This forms a block-shaped diagonal matrix. The result of the segment summary module is the fusion feature, expressed by the following formula (4):

[0063] (4)

[0064] in It contains semantic and hierarchical information within the segment.

[0065] S32. Input the fused features into the context aggregation module, and aggregate all context information through the self-attention mechanism to obtain the context-level learning output.

[0066] In one feasible implementation, the fusion feature is... Projected onto the attention space, as shown in equation (5):

[0067] (5)

[0068] in, , and These are different, learnable weight parameters.

[0069] The entire contextual information is aggregated through a self-attention mechanism as shown in equation (6):

[0070] (6)

[0071] in, It represents the learning output at the context level, integrating the global interaction of the overall token between instruction segments, and contains the semantic and hierarchical information of the entire text.

[0072] S33. Integrate the instruction segmentation features, token embedding features, and context-level learning output to obtain sample input data.

[0073] Optionally, S33 integrates instruction segmentation features, token embedding features, and context-level learning output to obtain sample input data, including:

[0074] The sample input data is obtained by adding the product of the instruction segmentation features, token embedding features, and context-level learning output with the balancing parameters.

[0075] In one feasible implementation, the integration method is as follows (7):

[0076] (7)

[0077] in These are learnable equilibrium parameters.

[0078] The embodiments of the present invention improve the semantic understanding and context awareness of role commands in large language models by constructing a context-aware hierarchical instruction learning method for large language models.

[0079] S4. Input the sample input data into the large language model, and perform supervised fine-tuning training on the response model composed of the context-aware module and the large language model to obtain the trained response model.

[0080] Typical large language models include LLaMa, DeepSeek, and Qwen. Large language models refer to large-scale pre-trained language models based on deep neural networks, especially the Transformer architecture. They can be trained on massive amounts of text data and achieve excellent performance in natural language processing tasks. Through learning from massive corpora, large language models gain a deep understanding of language and can generate coherent text, perform contextual reasoning, answer questions, and translate languages. In this invention, large language models are used to handle complex language tasks, adapting to various application scenarios through fine-tuning or with limited contextual information, generating fluent, accurate, and secure dialogue responses.

[0081] In one feasible implementation, the overall model is trained using instruction role dialogue data through a supervised fine-tuning method. The model learns the priority of hierarchical instruction following through aligned and unaligned data.

[0082] Supervised fine-tuning methods train the model using data with explicit priority labels. Specifically, supervised fine-tuning uses dialogue data with standard responses to optimize the model's behavior, enabling it to learn to follow instructions from different roles based on instruction priority and correct responses. Typical training methods include instruction fine-tuning and preference optimization. Instruction fine-tuning helps the model learn how to handle instructions according to priority across different roles by providing explicit instruction and response examples. Preference optimization introduces positive and negative samples, constructing preference pairs using instruction priority labels. For aligned dialogue data, positive samples indicate that the model correctly responds to aligned dialogues and has better response quality, while negative samples indicate that the model misunderstands instruction priority or has poor response quality. For misaligned dialogue data, positive samples indicate that the model correctly identifies instruction priority conflicts in the dialogue and rejects misaligned dialogues, while negative samples indicate that the model fails to identify instruction priority conflicts or fails to reject responses to misaligned dialogues.

[0083] Compared to existing mainstream methods, the embodiments of this invention can explicitly model and perform neural network calculations on the characteristics of instruction roles, and can explicitly learn the characteristics of instruction roles based on the semantics of instructions, and then input them into a large language model to utilize the capabilities of the basic model, so as to make the hierarchical structure of instruction roles clearer, thereby achieving better role instruction compliance priority and providing better security and application performance for large language model dialogue applications.

[0084] S5. Obtain the text to be replied to, input the text to be replied to into the trained response model, and obtain the dialogue response data of the text to be replied to.

[0085] In one feasible implementation, after obtaining the text to be replied to, the instruction role of the text to be replied to is determined, and the instruction segment segment embedding, instruction segment query embedding, and token embedding of the text to be replied to are determined. Based on the instruction role, instruction segment segment embedding, and instruction segment query embedding of the text to be replied to, the instruction segment segmentation features, instruction segment query features, and token embedding of the text to be replied to are obtained respectively. The specific processing flow can refer to step S2 above, and will not be elaborated here.

[0086] The instruction segmentation features, instruction segment query features, and token embedding features of the text to be replied to are input into the trained context-aware module to obtain the context-level learning output of the text to be replied to. The processing flow of this module can refer to step S3 above, and will not be repeated here. The instruction segmentation features, token embedding features, and context-level learning output are integrated, and the integrated features are input into the trained large language model to obtain the dialogue reply data of the text to be replied to.

[0087] This invention implements a context-aware instruction hierarchy priority. By interacting with information between instruction segments of different roles and throughout the entire context, it effectively distinguishes different instruction roles and trains the model to correctly learn the instruction hierarchy based on dialogue data. This improves the execution priority of the large language model for instructions of key roles, prevents the model from generating insecure responses when vulnerable low-priority instructions are attacked, enhances model security, and maintains the instruction compliance capability of the large language model, i.e., maintains model effectiveness. At the same time, distinguishing different instruction roles also provides flexible extensions for the large language model as the foundation for dialogue systems and intelligent agent applications, ensuring the security and stability of such dialogue applications.

[0088] Figure 2 This is a block diagram of a context-aware hierarchical instruction learning device for large language models provided in an embodiment of the present invention. This device is used for a context-aware hierarchical instruction learning method for large language models. (Refer to...) Figure 2 The device includes a construction unit 210, a determination unit 220, a context-aware unit 230, a fine-tuning training unit 240, and a response unit 250. Wherein:

[0089] The construction unit 210 is used to acquire initial training data, distinguish the instruction roles of the large language model in the initial training data, construct training data according to the instruction roles, and determine the instruction segment segmentation embedding, instruction segment query embedding and token embedding of the input sequence in the training data.

[0090] The determination unit 220 is used to construct instruction role labels for instruction roles in the training data, determine instruction segment segmentation features based on instruction role labels and instruction segment segmentation embeddings, determine instruction segment query features based on instruction role labels and instruction segment query embeddings, and determine token embedding features based on token embeddings.

[0091] The context-aware unit 230 is used to input the instruction segmentation features, instruction segment query features and token embedding features into the context-aware module to obtain sample input data.

[0092] The fine-tuning training unit 240 is used to input sample input data into the large language model and perform supervised fine-tuning training on the response model composed of the context-aware module and the large language model to obtain a trained response model.

[0093] The response unit 250 is used to acquire the text to be responded to, input the text to be responded to into the trained response model, and obtain the dialogue response data of the text to be responded to.

[0094] Figure 3 This is a schematic diagram of the structure of a context-aware hierarchical instruction learning device for large language models provided in an embodiment of the present invention, as shown below. Figure 3 As shown, a context-aware hierarchical instruction learning device for large language models can include the above-mentioned... Figure 2 The illustrated context-aware hierarchical instruction learning device for large language models. Optionally, the context-aware hierarchical instruction learning device 310 for large language models may include a first processor 2001.

[0095] Optionally, the context-aware hierarchical instruction learning device 310 for large language models may also include a memory 2002 and a transceiver 2003.

[0096] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0097] The following is combined Figure 3 The following is a detailed introduction to the various components of the context-aware hierarchical instruction learning device 310 for large language models:

[0098] The first processor 2001 is the control center of the context-aware hierarchical instruction learning device 310 for large language models. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0099] Optionally, the first processor 2001 can perform various functions of the context-aware hierarchical instruction learning device 310 for large language models by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0100] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 3 CPU0 and CPU1 are shown in the diagram.

[0101] In a specific implementation, as one example, the context-aware hierarchical instruction learning device 310 for large language models may also include multiple processors, for example... Figure 3 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0102] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0103] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be accessed through the interface circuit of the context-aware hierarchical instruction learning device 310 for large language models. Figure 3 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0104] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0105] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 3 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0106] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and can be connected to the interface circuit of the context-aware hierarchical instruction learning device 310 for large language models. Figure 3 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0107] It should be noted that, Figure 3 The structure of the context-aware hierarchical instruction learning device 310 for large language models shown in the figure does not constitute a limitation on the router. Actual knowledge structure recognition devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0108] Furthermore, the technical effectiveness of the context-aware hierarchical instruction learning device 310 for large language models can be referenced from the technical effectiveness of the context-aware hierarchical instruction learning method for large language models described in the above method embodiments, and will not be repeated here.

[0109] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0110] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0111] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0112] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0113] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0114] It should be understood that, in various embodiments of the present invention, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0115] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0116] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0117] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0118] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0119] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0120] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0121] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A context-aware hierarchical instruction learning method for large language models, characterized in that, The method includes: S1. Obtain initial training data, distinguish the instruction roles of the large language model in the initial training data, construct training data according to the instruction roles, and determine the instruction segment segmentation embedding, instruction segment query embedding, and token embedding of the input sequence in the training data. S2. Construct instruction role labels for instruction roles in the training data. Determine instruction segment segmentation features based on instruction role labels and instruction segment segmentation embeddings. Determine instruction segment query features based on instruction role labels and instruction segment query embeddings. Determine token embedding features based on token embeddings. S3. Input the instruction segmentation features, instruction segment query features, and token embedding features into the context awareness module to obtain sample input data; S4. Input the sample input data into the large language model, and perform supervised fine-tuning training on the response model composed of the context-aware module and the large language model to obtain the trained response model. S5. Obtain the text to be replied to, input the text to be replied to into the trained response model, and obtain the dialogue response data of the text to be replied to.

2. The context-aware hierarchical instruction learning method for large language models according to claim 1, characterized in that, The instruction roles in the large language model are pre-defined with different priorities, and different priorities indicate different instruction execution orders.

3. The context-aware hierarchical instruction learning method for large language models according to claim 1, characterized in that, The constructed training data includes aligned dialogue data and unaligned dialogue data; The aligned dialogue data represents the dialogue between instruction roles that follows a preset priority rule; The unaligned dialogue data represents dialogues between command roles that do not follow priority rules or have priority conflicts.

4. The context-aware hierarchical instruction learning method for large language models according to claim 1, characterized in that, The process of constructing instruction role tags for instruction roles in the training data includes: The role information corresponding to each Token is marked in the input sequence of the training data to form an instruction role tag sequence.

5. The context-aware hierarchical instruction learning method for large language models according to claim 4, characterized in that, The step of determining instruction segmentation features based on instruction role markers and instruction segment segmentation embeddings includes: Based on the instruction segment segmentation embedding, the instruction role markers are mapped to feature vectors to obtain the instruction segment segmentation features; The step of determining the command segment query features based on the command role marker and command segment query embedding includes: The instruction segment query features are obtained by mapping the instruction role markers to feature vectors based on the instruction segment query embedding.

6. The context-aware hierarchical instruction learning method for large language models according to claim 1, characterized in that, The context-aware module includes an intra-segment summary module and a context aggregation module; The S3 inputs the instruction segmentation features, instruction segment query features, and token embedding features into the context-aware module to obtain sample input data, including: S31. Input the instruction segment query features and token embedding features into the segment summary module, and fuse the instruction segment query features and token embedding features through the cross attention mechanism of the segment summary module to obtain the fused features. S32. Input the fused features into the context aggregation module, and aggregate all context information through the self-attention mechanism to obtain the context-level learning output; S33. Integrate the instruction segmentation features, token embedding features, and context-level learning output to obtain sample input data.

7. The context-aware hierarchical instruction learning method for large language models according to claim 6, characterized in that, S33 integrates the instruction segmentation features, token embedding features, and context-level learning output to obtain sample input data, including: The sample input data is obtained by adding the product of the instruction segmentation features, token embedding features, and context-level learning output with the balancing parameters.

8. A context-aware hierarchical instruction learning device for large language models, wherein the context-aware hierarchical instruction learning device for large language models is used to implement the context-aware hierarchical instruction learning method for large language models as described in any one of claims 1-7, characterized in that, The device includes: The construction unit is used to acquire initial training data, distinguish the instruction roles of the large language model in the initial training data, construct training data according to the instruction roles, and determine the instruction segment segmentation embedding, instruction segment query embedding, and token embedding of the input sequence in the training data. The determination unit is used to construct instruction role labels for instruction roles in the training data, determine instruction segment segmentation features based on instruction role labels and instruction segment segmentation embeddings, determine instruction segment query features based on instruction role labels and instruction segment query embeddings, and determine token embedding features based on token embeddings. The context-aware unit is used to input instruction segmentation features, instruction segment query features, and token embedding features into the context-aware module to obtain sample input data. The fine-tuning training unit is used to input sample input data into the large language model and perform supervised fine-tuning training on the response model composed of the context-aware module and the large language model to obtain a trained response model. The response unit is used to acquire the text to be responded to, input the text to be responded to into the trained response model, and obtain the dialogue response data of the text to be responded to.

9. A context-aware hierarchical instruction learning device for large language models, characterized in that, The context-aware hierarchical instruction learning device for large language models includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 7.

Images